• Register
• Log in

Spill

Expert User Acceptance & The Poorly Architected Application

Pilots are interesting creatures that offer us a lot of opportunities to learn if we observe them in their natural habitat. I recently spent several weeks locked in a bunker covered in camouflage observing their ritual behaviors. Ok… not really, but I have been working closely with Naval pilot Subject Matter Experts (SMEs) and during a recent user testing exercise I was faced with an interesting discovery. Boy I should have seen this one coming.

Pilots don’t like innovation.

When we started our charter to evolve the pilot cockpit interface I had heard the infamous stories of the unanimous rejection of life-saving technologies based simply on the fact that they didn’t look cool. I can’t say why I didn’t pay more attention to these stories except that I thought they were talking about electrodes on the skull and giant diaper shaped gravity suits. Surely a pilot would eagerly accept a new and better way to read the information his cockpit was providing him. I am here to say this is not the case. Pilots are the quintessential expert user. They spend no less than 2000 hours of training to become experts of an information system that is a metaphorical nightmare. It follows logically that they are a little apprehensive to reinvest their time in a new, albeit seemingly simpler, set of informational metaphors. During our first concept reviews for our Ev1 prototype we presented the “Combined Metaphor,” an interface that took all of the various information displays in the pilot cockpit and combined them into a single congruent presentation. Simply put, we removed gauges that tipped opposite directions to illustrate the same event and placed them all into a single top-down display of the craft in its environment surrounded with performance statistics and trend indications. What could be better, right? I knew I was in trouble when a pilot asked me if I had my instrument rating (I don’t yet, but am more than half-way finished with the training). He went on to tell me that the gauges we combined were showing completely different things and that they were fine the way they were.

I was dumbfounded… How could he not see that this system was better by far than the hundred year old aggregate in production today?

It was simple really, he couldn’t see it because he had spent 2000 hours learning to love the old interface. He, like every other expert user we encounter, has made an investment in learning the vocabulary of the system we hope to evolve. The difficulty by which his learning came is directly proportional to the resistance he has for its change. Believe me, there aren’t a lot of users that have a larger investment than pilots.

But I wasn’t ready to give up. I had to figure out what our new interface could offer pilots that they truly desired. A little ethnography and a whole lot of interviewing later I had my answer.

Pilots Love Workload Reduction.

Don’t get me wrong, pilots aren’t lazy but sometimes they have to do a lot of work to get something done. A perfect example is the process of entering waypoint coordinates into a flight plan. You’d be surprised how many attributes are used to describe a waypoint in a flight plan and entering or editing them can be tedious even when you’re not being fired upon. When we showed that our combined metaphor could facilitate single touch flight plan correction we got their attention. Even more exciting to them were our cognitive assistance features. When we discovered that pilots were often tasked with being at a specific location at an extremely precise period of time and were tasked with repeatedly calculating their performance to meet that objective, it was easy for us to design a system that did the same calculations dynamically. User testing recently proved that pilots showed a significant improvement in acquisition of Time On Target (ToT) when using our Ev2 interface. We had pilots laughing with joy as they executed their scenarios because it was so easy to monitor their performance. Best of all, once they saw how the new Ev2 interface could make their life easier they very quickly got over the metaphor shifts. Several of our recent flight testers would pout (in a tough and cool way of course) when they had to execute a mission on the “old” interface.

Mission successful, users engaged.

So what can we learn from this? You don’t need to be a pilot to make an investment in an interactive system and see that investment as valuable. As we architect we must always consider the costs of adoption and make sure we involve the user in helping us find their solutions. In doing so we’ll make some friends and maybe even fans.

---

Posted by Joseph Juhnke on September 4, 2007

---

---

---

---