About a year ago, Tanagram was granted its first direct (meaning we weren’t a sub-contractor) DARPA research funded program. Our goal was to improve situational awareness for soldiers deployed in contested (dangerous) cultures. During this program we proposed creating a complete server / client based architecture that responded to the program needs by providing digital augmentation of the surrounding environment. Tons of research, some prototype concepts, and a few patents later we are now cleared to share it with you.
We’ll talk more about the technology stack and client platform(s) in future posts, but for now you may read our final report.
If you invested 26 minutes into the video we posted here, then you already know where we are going with this post. Just as augmented reality (AR) is only one facet of an embedded interface, touch is only a portion of the entire vision for Natural User Interface (NUI). We are seeing a lot of touch and AR discussions today mostly because there have been advances in technology that are commercially (to we the consumer) available. Not to be a nag, but isn’t about time the problem drives the solution and not technology? Just as we learned during the age of artificial intelligence, technology (specifically improvements in processor speed in this case) will never solve the problem. Complexity must be replaced with simple and elegant solutions. Human’s with dreams must drive these solutions.
What is Natural User Interface (NUI)
So what is Natural User Interface? Natural user interface is the concept that you can touch (or yell at…) and manipulate digital objects much like you would objects in the real world. It’s a hard problem because we have been using mice and keyboards for so long we’ve forgotten what natural really feels like. On a recent cavort through the interwebs, I discovered the following concept and am thrilled to see discussions standardizing the way we talk about NUI.
Unfortunately, gestures like the two-finger-tap and the two finger hold are not natural interactions. We call these types of interactions “NUI Transitional.” They are interactions that are trained/learned that allow fingers to modulate information, but they are not “Present in or produced by nature.” An exaggerated example (talking to you Apple) would be if you have to use four fingers in a swiping gesture (versus 2 or 3) to interact in a specific way, you’re really not experiencing a natural interaction. More likely the software engineers ran out of natural metaphors, right?
A more natural approach
A hold-and-act-upon gesture is a good example of a truer NUI interaction for a touch surface. Specifically imagine you have a piece of paper on your desk and you want to slide it somewhere else. You touch the paper and, while holding your finger down, drag it to another location. Simple right? If you want to draw on that paper you wouldn’t use the same gesture. You also wouldn’t look around for a state-indicating button or icon to make sure you were in draw mode and not move mode, right? Instead you would put one hand (or finger) on the paper to hold it still and the second hand (perhaps with a writing implement) would move the drawing element over the surface of the paper.
This illustration uses Ron’s iconography to illustrate a stateless natural interaction. Specifically, we have one touch point from the left hand thumb holding the page still (a state in itself), while the right hand pinches (not the gesture) three fingers together much like holding a pen and drags across the surface. The result is a hard-to-explain but intuitive feeling writing gesture. I’ll disclaim that last statement in that we have no data to prove how “intuitive” that gesture is, but when we do it ourselves (give it a try) it feels pretty good. Mileage may vary… What do you think?
We wish it was a holiday, but alas no. Instead we’ve just got quite a bit to share with regards to the domain and will be posting it throughout the week. We’ve just wrapped a cool research project, filed our patents, and are now working through architecture challenges. Hopefully you will find our work helpful and participate in the NUI discussions happening here and on the web.
Once again, Tanagram is growing. We are looking for a curious individual with a passion for the future of interaction and deep experience working with HTML, AS3, and XAML. Our newest member must also rock. This isn’t a confidence thing for you or us, we aren’t prima donna designers and don’t work well with individuals who can’t collaborate intensively. Rock to us means to be REALLY REALLY good at what you do. It means you understand the implications of aesthetics and beauty on complex functional design. It means you understand and are relevant with the “fashion” of design. It means you wake up in the morning thinking about visual and systemic gorgeousness and strive to create it. You must have a great portfolio that is relevant and shows you are actively pursuing the issues we are passionate about.
Here at Tanagram we actively look for ways to escape the world of WIMP interaction and I’m proud to say we are very very close to realizing it. We design complex interactive experiences, systems that make most designers cringe. Systems that allow users to touch and manipulate data with hands not abstractions. We abhor refresh-encumbered interaction and enjoy individuals who understand that there is a better place.
While we can’t share project specifics, our current assignments include redefining geospatial hyperlocal media planning as a SaaS product*, inventing the the future of Augmented Reality interaction (no it’s not a silly iPhone app, think heads-up display technology and daily wear), and creating / refining online communities that are meaningful and truly help people. Our clients work with us to help them create the future and we are steadfast that our future is a better one.
In our process the UX designer’s role begins at project inception. She participates in client discussions gathering information and providing references to current and leading edge design approaches (visual, haptic, etc) that may benefit the client’s requirements. She references new trends, hot topics, and examples used by like and non-like competitors. Once requirements are gathered, she works with the project team as an artist, innovator, and strategist representing concepts and prototypes that are desired by her target audiences. Design aesthetics are always as important as simplicity and relevance. Our newest member must be a communicator, actively seeking and spreading inspiration. She is constantly challenging herself and the status quo.
Experience working with Industrial Design is a plus.
If you feel like she could be you and have 10+ years of experience please send your resume and portfolio to careers@tanagram.com and we can start the discussion.
It seems every other discussion I enter as of late has something to do with privacy and or the things “they” know about us. Maybe it’s the paranoid circles I travel, or maybe we are seeing the beginning of a significant change in our human paradigm. Digital technology is maturing and enabling us to connect/interact/react in ways we never before imagined. Human nature drives us to interact (you remember that “No man is an island” junk from High School?) and Digital technology is superpowering our ability.
Remember life before Facebook? How many old friends have you lost touch with? How many viable relationships could you maintain before Facebook? How many can you maintain now? (I have 181)
It’s a little overwhelming to think that we are actively crafting a Digital overlay that will enable Digitally enabled beings to interact with other Digitally enabled beings or objects across space (think telekinesis), time (think time-based stitched panoramas made from FLICKR images chronologically navigated), and wordless communication (telepathy?). Imagine being able to see and hear and interact with objects from amazing distances. Imagine wearing a digital outfit. Imagine.
As Augmented Reality, the superimposition of the Datasphere onto reality, becomes more viable we will gain access to vast amounts of real-time data and there are a few issues we need face very soon.
Privacy – The ‘Millenials‘ have already succombed to the collective and have given away the illusion of privacy. Social media products like Facebook, MySpace, and Twitter have become platforms for their near-narcisistic need to self-proclaim. They communicate outwardly, freely and without fear. The rest of us, especially those who remember the 60′s are going to face challenging times over the next 10 years. Data availability and detail will continue to improve and we will all be faced with a choice: Does data serve or punish me?
What does it mean that marketers know where I live and what products I’m inclined to buy? Is my cellphone constantly broadcasting my location accurate to the meter? Am I being monitored for selfish reasons? Ok, you can sit down and stop shaking…. It’s not as bad as you think. We at Tanagram firmly believe Digital will serve us in powerful and meaningful ways and we spend a great deal of time imagining what those might be.
The truth – You ARE being watched. There, I said it, and nothing exploded. The fact is the Marketing Industry is facing one of the most challenging times since its inception. Gone are the days when they could ‘tell people what to believe.’ Gone are the days where they could produce a single, one-size-fits-all message that compells consumers to purchase products. Digital has made us (consumers) much smarter and because of this, we require much more sophisticated communication and interaction to even begin to consider a product relationship (that’s what it is when you become loyal to a brand). We the consumer have become extremely powerful and marketers are afraid.
So what evil things can they do with their monitoring? Well, that’s where we take an unexpected turn. You see, the only way they can sell is with the truth. They can no longer pay talking heads tell us what to think, lies are posted on Twitter for thousands to view seconds after they are broadcast. The backlash from a malformed or misguiding ad-campaign can cost millions in revenue but even worse it can cast the originator as not reputable and have significant long term effects. Everybody remembers Blackwater, right? Their brand was so destroyed by blunder and lies they had to rename as Xe. So if they can’t lie to us, how can they push their products? They have to connect with us like real people, meaningfully. To do that they grab gobs of data and look for consumers who would actually benefit from and are interested in the products they sell. They then do their best to be helpful and useful so you, the consumer, will learn they are trustworthy and legitimate. They spend fortunes on non-profits and other ‘Cause Marketing’ to be viewed as contributing members of society. Ultimately, they work very hard to maintain a real relationship with you. You see how mild mannered this once arrogant monster has become? Joking aside, they cannot take your free will. You will continue to be a fickle and informed consumer and they cannot afford for you to think that they have any intentions other than serving you the best they can. Also, there is no mind-control chip. They cannot hypnotise you into stupid purchase habits (for me this is a non-issue, I only buy stupidly) and if they could, they wouldn’t because the consumer backlash would destroy them.
So what do we do with these needy, shivering, whelp marketers? We give them our data. Why, because they will serve us better if they know our likes and dislikes. They can’t afford to annoy us anymore and they don’t want to waste money trying to sell to those who are disinclined. Broadcast advertising is going away. Advertising of the future will take place at a personal level in meaningful ways. Here’s an example: Perhaps you are driving home, late from work and you haven’t prepared the family meal. Your local preferred local grocer sends you a text message offering a discount on cooked chicken dinners (or your favorite dish) and saves the day.
That is where we are going, we just have to make the choice.
New and exciting concepts that merge the physical and digital are showing up every day. You know how excited we are by the premise but perhaps these will get you juiced as well.
Playstation 3 uses AR to enhance physical EYE OF JUDGEMENT (imagine deep movie voice guy reading that name) trading card game.
Another strange fetish-like avatar whose name translates to Cyber Figure Alice (ya, I know… CYBER?) from GeishaTokyo.com
Dennou Coil is an interesting Anime series that illustrates an augmented world. Finding the videos online is like a video game where you have to dodge popups and malware (be careful). Here’s the first episode.
If you do nothing else today spend 5 minutes and go to this site. You’ll need a printer, a pdf viewing application, a computer with a webcam (new Macs come with these by default), and a web browser with Flash 10 installed (you will get a prompt to install if you don’t have it).
Click on the “Launch Turbine” or “Launch Solar” link and permit Flash to access your camera (you’ll get a dialog box)
Hold up the printed PDF so it shows up in your camera view and prepare to be amazed.
This demo begins to show the promise of Augmented Reality on embedded display technology. What I mean is as displays get cheaper it will be more common for them to be placed in public or even private areas for your benefit. Imagine dressing room mirrors that can show you what you will look like without changing cloths. Imagine wearing digital accessories visible only to those who view you through a screen or through a Head Mounted Display (HMD). The possibilities are endless as we start to embed virtual objects in our reality.
Note: Tanagram has been awarded (we’re in the ‘negotiation’ phase) a grant to design a user interface concept for Augmented Reality. More on that soon!
I may have had a drink or two or perhaps the interviewer was particularly charming. Either way I opened up for the folks at Level 3 and shared our dreams and visions for where Digital is going to take us in the coming years. I wish you could have seen the interviewer’s face. These were not answers he was prepared to address. Tell me the truth… Do I sound like a nut? Check out the other interviewees here. Great thinking and commentary from all.
I recently attended the Information Architecture Institute Idea 2008 Conference, October 7-8 in Chicago. The theme of the conference was ” . . . on designing complex information spaces of all kinds.”
What was intriguing about this particular conference was the diversity of people, both in professional and geographic terms. There were graphic designers, interaction designers, technical leads, managers, and oh yes . . . “information architects.” What was interesting about the attending information architects was that they came from so many backgrounds to become an information architect. There were actual classically trained architects that became IA’s, there were designers that were IA’s and so on.
When Richard Saul Wurman coined the term Information Architect in the late 1980′s, he was an architect that was designing travel books (Access Press) and came up with a term for designers that created information intensive artifacts. Since we cannot really agree on what the exact meaning of “information” and “architect”, I have come to the conclusion that merging the words into a concept would be difficult having hired several IA’s professionally. There is little agreement of what an IA is, and even what their outputs are. This was evident at the Idea conference, that there was no attempt in defining the term.
The first speaker was blogger David Armano who spoke on “Micro-Interactions in a 2.0 World.” A well-known and dynamic speaker, David took participants down a very rapid terrain of design, marketing and business through technological innovations. His central premise is that we are moving from passive consumers to active participants through existing social architecture technologies – not custom applications. Since all of our devices are internet enabled, the notion of a traditional browser experience is giving way to smaller more intimate digital apps that do one or two things. We as users cross-link these apps together. He used the term “life streams” to name this process of “engage, enable, and empower” our actions through a model of “usefulness, utility and ubiquity.” David also articulated new digital ecosystems such as the Nike Touch which uses “engagement” of “deposits” and “withdrawls” with several micro-functionalities bundled together. Social networks by their very nature amplify communications and he asked the audience what their “passion point” was.
Elliott Malkin, an artist from New York discussed “Information in Space.” His passionate and precise presentation went down a very indirect route that got me very excited. His initial metaphor was the hassidic concept of an eruv, or a physical demarkation between a secular world and a religious world using the same space. He referred to this psychographic space as having strong conceptual power for the intended group and for what for most people would not even notice. Unfortunately, I thought he was going to bring the metaphor back to digital technology and social architecture, but instead he discussed using digital technology to create a virtual eruv that could be monitored without rabbi’s going out to check if the eruv physical demarcations were intact. The implications of this metaphor in discussing how a shared space could have unique “functionalities” for different groups at the same time holds great promise.
Jesse James Garrett of Adaptive Path presented “Envisoning the Future of the Web.” They worked with the Mozilla Foundation in exploring the future of web browsers. Their concept is called Project Aurora. Now, when dealing with the future of anything, especially technology, it is difficult since we tend to use established conventions and behaviors and link it to a future that people can understand. Star Trek did it best by taking human behaviors and linking them to technologies that did not necessarily need to describe their inner workings. As viewers, especially hooked viewers, we understood the galactic federation model and the value system of the show to put the expressions and technologies in context.
With the future of a browser, Adaptive Path focused on augmented reality, or the overlay of digital information on the real world where there will be data abundance and the question will be how to we visualize, focus and manage all of it cognitively and socially. Processing power, storage capacity, bandwith and graphic capabilities of computers will impact how we interact with each other through the digital cloud. “Context awareness,” “natural interaction,” and “continuity” would allow for more natural collaborations. Each of us would have a semantic profile and with geolocation, would allow for very rich interactions between people where ever they are. Two main questions arose from the audience. What was the time horizon of Project Aurora? Jesse stated they had a 10 year window into the future (this would be the equivalent of 40 years in technological terms). He said they had to balance “compelling” with “plausible” in their vision. My view was that their vision was too contemporary and linked to current “plausible” scenarios. The second question was that their concept could be viewed that the browser was an operating system. Jesse made it clear that they did not want to address the operating system vs. browser question, but in my mind the two converge in their scenario.
Chris Crawford a former game designer for Atari presented an interesting perspective on “Linguistic User Interfaces.” His perspective on intelligent systems is that smart computers that could interact with humans using extensive language patterns is not realistic. This is due to the Sapir/Wharf hypothesis that inside the human mind language and reality exist together. Chris’ interesting take is that with games, a model for a computer to interact with humans is much more manageable since the worlds are much smaller. He further elaborated that software, verbs define the program and is core to the human/computer interface. With most current software, as the verb count increases, accessibility and expectability reduces. 100 verbs is the limit for most users. Chris is currently developing a linguistic user interface (LUI) for programs that can create stories. I found his perspective very compelling.
Alberto Canas, of the Institute for Human and Machine Cognition (IHMC) presented a surprising presentation called “From Meaningful Learning to a Network of Knowledge Builders.” IHMC has created the popular language mapping program CMAP, which I have on my computer. What was informative about his presentation on CMAP were examples of its application (pun intended). Knowledge essentially are concepts that are linked together with prepositions to make a relationship (this was a wonderfully simple definition). Humans have created written language to describe concepts that cannot easily be illustrated. CMAP are concept maps linked by phrases to form propositions. I already knew this in principle. Alberto then showed the power of CMAP through an ongoing project with the Panamanian government giving school children CMAP to describe their lives. The power of CMAP is that users can link images and web page addresses to their maps and can also link concept maps to other concept maps. They are essentially mini-websites which are non-linear. I will not look at CMAP the same way and believe that its potential is not fully understood by a large cross section of users.
Jason Fried, founder of 37signals presented a lucid lecture called “Getting Real.” I had not heard Jason before, but am a heavy user of Basecamp and a real fan of its simplicity and how reliable it is. Jason is a real visionary and their development process flies in the face of every convention that most digital consultancies use. They do not “plan” anything, do no “specification” documents, and do not use “actor or personas.” They focus on building things and figure out how to do it over time. Keep things small, use sharpie markers as the finest resolution when sketching ideas, and only have a core set of functions (a simple core). Part of me was aghast, but I quickly saw the logic to their process. If you are designing for yourselves and then find users, the 37signals model is perfect. Unfortunately, if you collaborate with clients to define the problem and then facilitate understanding, then the 37signals model will not work. However (you knew there was going to be a “but”) I totally agree with his concept of “scratching your itch” and doing things with passion.
Aradhana Goel of Ideo gave one of the most thought provoking presentations called “Emerging Trends, Design Thinking, Service Innovation.” We have all heard of IDEO and it is one of the most influential innovative firms (along with Pentagram) merging design and engineering. Aradhana was trained as an architect and has only recently become involved with service design. Her perspective on human factors was clear and in alignment with my understanding. What was powerful was her ideas around linking human factors with trend factors. Human factors focus on digging into context, while trend factors find the context. She went on to compare and contrast these two areas and how service design is a logical next step in productizing intangible experiences.
Bill DeRouchey of Ziba Design gave a very direct and engaging discussion on “The Language of Interaction.” His deconstruction of everyday visual clues that we take for granted and their constant reinterpretation and reapplication to other situations was informative, common sense, but insightful.
Overall, there were several key themes that all speakers seemed to focus upon:
1) Browsers are giving way to other internet enabled experiences
2) Windows, Icon, Mouse, Pointer system is under stress
3) Transaction is more than money
4) Link several apps, not one killer app
5) Focus on experiences, not just interactions
Upon reflection, I found this conference very fulfilling and reinforced certain convictions, challenged others and provided a very positive mental workout for me (which is what good conferences should do). I would like to compliment IAI for the organization of the conference and the lunches with different groups of people was enjoyable. There were twelve speakers in two days, interspersed with group lunches around the Chicago loop.
The IAI did not go down an exististential vortex of what an information architect is or is not (though it came close at times), which would have been a divisive and somewhat unimaginative exercise. Instead they linked together several strains of interesting ideas and left the participants to decide what it meant to them.
Those crazy cool kids at Mozilla Labs have just announced the release of their experimental plugin Geode. Using the draft W3C Geolocation standard they plan on enabling your browser to dynamically determine its location to provide cool value-adds like location-switching RSS feeds and emergency restaurant location. For us the ubiquity of geolocation services is a natural progression towards augmented reality but we can’t help finding ourselves a little amused at an application that politely asks permission to know where it is and then offers the user a choice of resolutions (exact, neighborhood, city, nothing) for each page using the service. That sure makes for a transparent experience! Sheesh.
Recent Comments