A little over a month ago we announced our work developing an Augmented Reality Platform for the U.S. Government. While we can’t talk about contract negotiations as they are lengthy, sensitive, and tricky, we CAN share that our Bloom server is currently in an Alpha state and is already doing some amazing things. We are modeling, modeling, modeling from video and still photography and the output is amazing. The video above is a model of the front of my home. The video above represents only a single (but very important) step in the Bloom “Gathering Process.”
We are also in the process of optimizing the code to further improve efficiencies. As our effort continues we will be entering the second phase of our program that will include the development of the user interface for the iARM platform. That user interface will be a multichannel input / output leveraging gesture (hands/hand symbols), speech, and context. Our first evaluation will be the gesture based system. There are a number of open-source and closed platforms (this, and all of these…) that will likely play a part in this system but we are always looking for experts.
Please take the time to share if you have ideas you think would benefit this vision.
ARE2010 did a lot of things for Tanagram. First and foremost it taught us that we aren’t the only crazy people out there. There were many hundred equally passionate (and equally insane) individuals working diligently to realize a vision that has been foretold for many many years. Attending were 400 enthusiasts, 90 speakers, and 40 sponsors.
The conference also re-affirmed that Augmented Reality (AR) is a new frontier. The discussions were widely diverse from immediate implementations of marketing gimmicks to future visions of integrated experiences and super-powered humans. Keynote speakers, Bruce Sterling, Will Wright, and Blaise Agüera y Arcas, and Jesse Schell were all brilliant. They taught us to be humble, focused, human-centered, and reminded us that this isn’t the first new frontier and to remember (and reuse) the past. Jesse Schell presented a humorous and sobering view of privacy and what AR could mean for it (his point was it was already too late…).
I personally had an opportunity to meet and discuss ideas with some of my all-time hero’s including Bruce Sterling and Blaise Agüera y Arcas.
On the technology front I can share that there is a definite future for head mounted display technology. We saw proposals for 150deg. FOV systems. It’s going to be a very exciting decade!
As soon as the ARE folks post the videos I encourage you to browse the sessions. I didn’t attend one that was not fascinating. The ARE event will be held next year in Santa Clara and we will definitely be there.
P.S. Through this experience I was introduced to Bruce Sterling’s Wired Blog – Beyond the Beyond. Bruce admits it is eclectic as hell and is really anything that interests him. He saw that as a weakness, but I see it as a strength. I really look forward to the “random” posts. They are always head-stretching. I also encourage you to review the twitter feed for the conference. Great stuff in there!
This is an exciting illustration of the power of “collective memory” a concept we use to describe the knowledge management-like capabilities presented by spatialized AR data. Depending on the amount of stored detail one could rewind a specific view through multitudes of perspectives of a single area to see information from yesterday or a few centuries ago. This concept is a key component in the architecture of our iARM ecology.
Because our system is vision based, we have an extensive amount historical footage for areas of interest and the daily use of our system continuously adds to our model making for an amazingly powerful captured dimensional recording of our world.
If we caught your attention with our last post but you’re having a difficult time understanding what’s so exciting about a little research and some images narrating a potential future rest assured we have a lot to talk about.
The first thing we need to address is one of the “Hows” of the visualization technology. Even though we have several patents pending, we are still too sensitive to expose the ‘whole tomato’ to the masses, but suffice it to say we have cracked the Augmented Reality visual registration nut. The following is our first delving into the details that are under development.
What is this nut? Let me explain…
If you’ve ever used the current lineup of “AR Apps” available on Android or iPhone platform you’ve seen a neat portal into the world of Augmented Reality. You typically see a camera view of the world around you presented on a hardware screen with digital information presented (most often clumsily) on top of the view. As you move the phone around you see that information move accordingly as if it’s locked to some physical location. Sometimes you see that information move or ‘drift’ for unknown reasons and that is really the problem with current tech AR.
Registration is the accuracy by which the computing platform can “lock” the digital world onto the physical world and up to today, that technology has been susceptible to all sorts of errors and interference. One of the reasons registration is still immature is because developers are relying on hardware improvements to solve their problems. We believe that while hardware, especially heavily funded hardware (thank you Apple), may eventually get accurate enough to solve the registration problem we don’t have the time or patience to wait. We instead spent our time researching and testing an elegant and simple solution that would work with today’s hardware capabilities.
If you look at the narrative slides below you can see our User Experience Model relies heavily on pixel-accurate digital registration. We can “paint” anything including highlighting objects, manually marking the environment and even embedded three-dimensional models. I can assure you these aren’t the crazy vision of a designer with no insight as to how the technology works but are based on our registration system (U.S. Patent Pending).
Our system is primarily vision-based, but leverages gross measurements (with error detection) of GPS, Cell/WIFI triangulation, and digital compass hardware. That means that it uses cameras, models, and some other fancy wizardry to rapidly identify the observed view and paint objects within that domain accordingly.
That’s all for now but we’ll be presenting some more details on the system at ARE2010 and are excited to continue this discussion.
About a year ago, Tanagram was granted its first direct (meaning we weren’t a sub-contractor) DARPA research funded program. Our goal was to improve situational awareness for soldiers deployed in contested (dangerous) cultures. During this program we proposed creating a complete server / client based architecture that responded to the program needs by providing digital augmentation of the surrounding environment. Tons of research, some prototype concepts, and a few patents later we are now cleared to share it with you.
We’ll talk more about the technology stack and client platform(s) in future posts, but for now you may read our final report.
If you invested 26 minutes into the video we posted here, then you already know where we are going with this post. Just as augmented reality (AR) is only one facet of an embedded interface, touch is only a portion of the entire vision for Natural User Interface (NUI). We are seeing a lot of touch and AR discussions today mostly because there have been advances in technology that are commercially (to we the consumer) available. Not to be a nag, but isn’t about time the problem drives the solution and not technology? Just as we learned during the age of artificial intelligence, technology (specifically improvements in processor speed in this case) will never solve the problem. Complexity must be replaced with simple and elegant solutions. Human’s with dreams must drive these solutions.
What is Natural User Interface (NUI)
So what is Natural User Interface? Natural user interface is the concept that you can touch (or yell at…) and manipulate digital objects much like you would objects in the real world. It’s a hard problem because we have been using mice and keyboards for so long we’ve forgotten what natural really feels like. On a recent cavort through the interwebs, I discovered the following concept and am thrilled to see discussions standardizing the way we talk about NUI.
Unfortunately, gestures like the two-finger-tap and the two finger hold are not natural interactions. We call these types of interactions “NUI Transitional.” They are interactions that are trained/learned that allow fingers to modulate information, but they are not “Present in or produced by nature.” An exaggerated example (talking to you Apple) would be if you have to use four fingers in a swiping gesture (versus 2 or 3) to interact in a specific way, you’re really not experiencing a natural interaction. More likely the software engineers ran out of natural metaphors, right?
A more natural approach
A hold-and-act-upon gesture is a good example of a truer NUI interaction for a touch surface. Specifically imagine you have a piece of paper on your desk and you want to slide it somewhere else. You touch the paper and, while holding your finger down, drag it to another location. Simple right? If you want to draw on that paper you wouldn’t use the same gesture. You also wouldn’t look around for a state-indicating button or icon to make sure you were in draw mode and not move mode, right? Instead you would put one hand (or finger) on the paper to hold it still and the second hand (perhaps with a writing implement) would move the drawing element over the surface of the paper.
This illustration uses Ron’s iconography to illustrate a stateless natural interaction. Specifically, we have one touch point from the left hand thumb holding the page still (a state in itself), while the right hand pinches (not the gesture) three fingers together much like holding a pen and drags across the surface. The result is a hard-to-explain but intuitive feeling writing gesture. I’ll disclaim that last statement in that we have no data to prove how “intuitive” that gesture is, but when we do it ourselves (give it a try) it feels pretty good. Mileage may vary… What do you think?
We wish it was a holiday, but alas no. Instead we’ve just got quite a bit to share with regards to the domain and will be posting it throughout the week. We’ve just wrapped a cool research project, filed our patents, and are now working through architecture challenges. Hopefully you will find our work helpful and participate in the NUI discussions happening here and on the web.
Once again, Tanagram is growing. We are looking for a curious individual with a passion for the future of interaction and deep experience working with HTML, AS3, and XAML. Our newest member must also rock. This isn’t a confidence thing for you or us, we aren’t prima donna designers and don’t work well with individuals who can’t collaborate intensively. Rock to us means to be REALLY REALLY good at what you do. It means you understand the implications of aesthetics and beauty on complex functional design. It means you understand and are relevant with the “fashion” of design. It means you wake up in the morning thinking about visual and systemic gorgeousness and strive to create it. You must have a great portfolio that is relevant and shows you are actively pursuing the issues we are passionate about.
Here at Tanagram we actively look for ways to escape the world of WIMP interaction and I’m proud to say we are very very close to realizing it. We design complex interactive experiences, systems that make most designers cringe. Systems that allow users to touch and manipulate data with hands not abstractions. We abhor refresh-encumbered interaction and enjoy individuals who understand that there is a better place.
While we can’t share project specifics, our current assignments include redefining geospatial hyperlocal media planning as a SaaS product*, inventing the the future of Augmented Reality interaction (no it’s not a silly iPhone app, think heads-up display technology and daily wear), and creating / refining online communities that are meaningful and truly help people. Our clients work with us to help them create the future and we are steadfast that our future is a better one.
In our process the UX designer’s role begins at project inception. She participates in client discussions gathering information and providing references to current and leading edge design approaches (visual, haptic, etc) that may benefit the client’s requirements. She references new trends, hot topics, and examples used by like and non-like competitors. Once requirements are gathered, she works with the project team as an artist, innovator, and strategist representing concepts and prototypes that are desired by her target audiences. Design aesthetics are always as important as simplicity and relevance. Our newest member must be a communicator, actively seeking and spreading inspiration. She is constantly challenging herself and the status quo.
Experience working with Industrial Design is a plus.
If you feel like she could be you and have 10+ years of experience please send your resume and portfolio to careers@tanagram.com and we can start the discussion.
It seems every other discussion I enter as of late has something to do with privacy and or the things “they” know about us. Maybe it’s the paranoid circles I travel, or maybe we are seeing the beginning of a significant change in our human paradigm. Digital technology is maturing and enabling us to connect/interact/react in ways we never before imagined. Human nature drives us to interact (you remember that “No man is an island” junk from High School?) and Digital technology is superpowering our ability.
Remember life before Facebook? How many old friends have you lost touch with? How many viable relationships could you maintain before Facebook? How many can you maintain now? (I have 181)
It’s a little overwhelming to think that we are actively crafting a Digital overlay that will enable Digitally enabled beings to interact with other Digitally enabled beings or objects across space (think telekinesis), time (think time-based stitched panoramas made from FLICKR images chronologically navigated), and wordless communication (telepathy?). Imagine being able to see and hear and interact with objects from amazing distances. Imagine wearing a digital outfit. Imagine.
As Augmented Reality, the superimposition of the Datasphere onto reality, becomes more viable we will gain access to vast amounts of real-time data and there are a few issues we need face very soon.
Privacy – The ‘Millenials‘ have already succombed to the collective and have given away the illusion of privacy. Social media products like Facebook, MySpace, and Twitter have become platforms for their near-narcisistic need to self-proclaim. They communicate outwardly, freely and without fear. The rest of us, especially those who remember the 60’s are going to face challenging times over the next 10 years. Data availability and detail will continue to improve and we will all be faced with a choice: Does data serve or punish me?
What does it mean that marketers know where I live and what products I’m inclined to buy? Is my cellphone constantly broadcasting my location accurate to the meter? Am I being monitored for selfish reasons? Ok, you can sit down and stop shaking…. It’s not as bad as you think. We at Tanagram firmly believe Digital will serve us in powerful and meaningful ways and we spend a great deal of time imagining what those might be.
The truth – You ARE being watched. There, I said it, and nothing exploded. The fact is the Marketing Industry is facing one of the most challenging times since its inception. Gone are the days when they could ‘tell people what to believe.’ Gone are the days where they could produce a single, one-size-fits-all message that compells consumers to purchase products. Digital has made us (consumers) much smarter and because of this, we require much more sophisticated communication and interaction to even begin to consider a product relationship (that’s what it is when you become loyal to a brand). We the consumer have become extremely powerful and marketers are afraid.
So what evil things can they do with their monitoring? Well, that’s where we take an unexpected turn. You see, the only way they can sell is with the truth. They can no longer pay talking heads tell us what to think, lies are posted on Twitter for thousands to view seconds after they are broadcast. The backlash from a malformed or misguiding ad-campaign can cost millions in revenue but even worse it can cast the originator as not reputable and have significant long term effects. Everybody remembers Blackwater, right? Their brand was so destroyed by blunder and lies they had to rename as Xe. So if they can’t lie to us, how can they push their products? They have to connect with us like real people, meaningfully. To do that they grab gobs of data and look for consumers who would actually benefit from and are interested in the products they sell. They then do their best to be helpful and useful so you, the consumer, will learn they are trustworthy and legitimate. They spend fortunes on non-profits and other ‘Cause Marketing’ to be viewed as contributing members of society. Ultimately, they work very hard to maintain a real relationship with you. You see how mild mannered this once arrogant monster has become? Joking aside, they cannot take your free will. You will continue to be a fickle and informed consumer and they cannot afford for you to think that they have any intentions other than serving you the best they can. Also, there is no mind-control chip. They cannot hypnotise you into stupid purchase habits (for me this is a non-issue, I only buy stupidly) and if they could, they wouldn’t because the consumer backlash would destroy them.
So what do we do with these needy, shivering, whelp marketers? We give them our data. Why, because they will serve us better if they know our likes and dislikes. They can’t afford to annoy us anymore and they don’t want to waste money trying to sell to those who are disinclined. Broadcast advertising is going away. Advertising of the future will take place at a personal level in meaningful ways. Here’s an example: Perhaps you are driving home, late from work and you haven’t prepared the family meal. Your local preferred local grocer sends you a text message offering a discount on cooked chicken dinners (or your favorite dish) and saves the day.
That is where we are going, we just have to make the choice.
New and exciting concepts that merge the physical and digital are showing up every day. You know how excited we are by the premise but perhaps these will get you juiced as well.
Playstation 3 uses AR to enhance physical EYE OF JUDGEMENT (imagine deep movie voice guy reading that name) trading card game.
Another strange fetish-like avatar whose name translates to Cyber Figure Alice (ya, I know… CYBER?) from GeishaTokyo.com
Dennou Coil is an interesting Anime series that illustrates an augmented world. Finding the videos online is like a video game where you have to dodge popups and malware (be careful). Here’s the first episode.
Recent Comments