Prototyping for Discovery: Determining Fidelity for User Testing
The advantages of prototyping during a discovery phase are obvious and well documented. If you’re comparing prototypes to actual builds, they are quicker to make, don’t have to rely on a designer’s coding ability, and save a ton of time and money by drastically reducing risks involved in launching a product. They help product teams understand and shape what the product is when it is still in a flexible, moldable form, and test out how it performs in simulated situations. We use prototypes as a way of learning by doing — a way to put our ideas out there for users to react to so we know what’s worth pursuing and what isn’t.
There is no magic formula to follow for creating a testable prototype, but there are definitely some guidelines we follow to make sure we’re learning what we want to know, when we want to know it. The reasons for starting with a lo-fi prototype from a product team’s perspective are numerous, but it can also be disheartening to put something lo-fi into a user’s hands and watch them struggle with it and call your beautiful baby ugly. If they could only just see how it moves! If they could see it with the right animations, the beautiful final images, they would love it so much more! And yes, there is a time and a place for getting user love for your products, and that time is post launch, when you’re asking users to rate you on the app store. The purpose of user-testing a prototype is to learn.
Follow the “5 Planes” of UX Structure
Polished prototypes are time consuming, and quick sketches don’t act like real apps, and therefore won’t have the same effect as the final version. So how do you decide what fidelity to test at? Jesse James Garrett, one of the early thought leaders of User Experience methodology said that there were 5 planes of user experience that you move through. It basically starts with 1.) user and business goals, and moves through 2.) requirements and function, 3.) information architecture, 4.) interface, information and navigation design, and 5.) visual design. If you’re practicing waterfall, you move from one stage into the next, successively. But for those who are practicing agile and lean methodologies, it gets a little blurry. We move quickly as a team around the different planes, back and forth, refining, pivoting, and iterating as we go, but we never get too far ahead without establishing the lower planes first.
I would argue prototype fidelity should follow a similar pattern. In the lower planes when there is still so much to discover, fidelity should be low and fluid enough to make big changes to the structure without much additional work. If you go too far into the higher planes of refining the experience, you risk going into unvalidated territory. You might get designer’s tunnel vision, and start solving problems for yourself instead of your user, which is a whole separate issue I’m not going into.
Avoid Polish Diversion
Psychologically speaking, when you test with a higher fidelity prototype it’s also perceived differently. Users are less likely to criticize it, and isn’t that the point of user testing? To find problems, and validate all the assumptions you have about your user that led you to make what you did? Let’s say for example, you’ve built the shiniest prototype of them all. It’s got snappy animations, beautiful design, and real content. The user may fumble with the central navigation, yet tell you they like the app. Design has a strong impact on the perceived value, and something that looks nice is assumed to also work well. This is what I call “polish diversion”—but when you strip away all the pretty stuff, the user can only focus on the navigation, or the content, or whatever else you’re testing. It’s like putting blinders on a carriage horse to prevent distractions. You’re limiting the number of things that can affect their perception of the app, and bring that one thing you want to learn about into focus.
Use the Scientific Process
Think of the scientific process. If you were to run an experiment, you’d have your control group, and you’d have one main variable. You would run the tests in identical environments, and measure the change. If all goes as planned, you know that the one variable is responsible for any change in the outcome. Of course, humans are much too unpredictable and un-scientific for that, but there are definitely some things we can do to make sure our prototype testing results come from logical decisions. Just like you wouldn’t put too many variables in an experiment, going hi-fi with your prototype too early means you’re opening yourself up for biased results. If a user responds in an unexpected way, you can’t really be sure of why. Was it the content? Was the design too subtle? Did they fail to understand the central navigation? Or was the app just not helpful to them?
So in short, the prototype should match what you’re trying to learn about. Granted there are other reasons for making a hi-fi prototype, such as selling the concept to stakeholder or investor, or later in the testing process when you’re doing more nitty-gritty usability testing. But when you don’t yet understand the product, the fidelity should match so you can test the foundation and structure first. The earlier you start testing, when the architecture of the app is still fluid, the more you can be sure of what variable caused a certain reaction, and the more confident you can be that you’re building up on a solid foundation. When you finally do get to put on the polish, you not only have something super functional and need-fulfilling, but it looks and behaves beautifully too.