I’ve spent a little too much time thinking about the introduction of this article. Looking for a way to say that a good design solution needs to connect and resolve both business and user needs but emphasise how important it is not to lose sight of what real users want. I would end that argument by introducing our hero — user research — as an essential tool for keeping design solutions grounded and focused on the user experience.

But I decided that nobody needs to be sold on how important user research is. I’ll focus on how I get the most value out of it instead.

1. Ask the Right Questions

The first step for planning is to define goals. What do I want to get from doing user research? The most common mistake is to have a goal that looks like this: I need to make some personas for this app I’m working on.

But personas are just a tool — a means to an end — an artifact of the research that can help me achieve my real goal. So what outcome am I looking for? What’s the end game? Maybe something like this: I want to have the best UX ever in any app since the beginning of time.

No, that won’t work either. I went a bit too far and, while the enthusiasm and conviction are commendable, this is too abstract and too ambiguous to test. What I need to do is ask a question that can be answered through user research. Something concrete that can lead to an outcome that is actionable.

How about something like this: I introduced a new checkout flow in the app and I want to see if users understand it and are able to complete tasks easier. Yes, this is something I can test.

To summarize asking the right questions means:

• Focusing on the outcome, not on output.
• Asking concrete questions to get concrete answers.
• Research artifacts are just a means to an end.

2. Choose the Right Research Tool

Now that I have my question I can work with the researcher to decide what’s the appropriate research method for this case. First I should decide if I need qualitative or quantitative research.

Quantitative research (gathered through surveys or through analytics data) is best suited, for example, when I need to define user segments, assess preferences or size up intent for a particular feature. Anything that can be answered with numbers. But if I need to understand the intent behind those segments or preferences, to understand motivation and context behind user choices, or if I need to test the usability issues of a certain feature, I’ll choose qualitative research instead.

While quantitative data is most accurate, because it represents a large number of users and it comes from the natural use of the product, the question I asked — on the usability of a checkout flow — is best answered by qualitative research. Qualitative data can be collected in different ways:

• natural use of the product (e.g. ethnographic study, diary study…)
• scripted use of the product (e.g. lab-based or remote, moderated study)
• decontextualized research — not using the product (e.g. user interview).

For the current example, the best choice would be to run a moderated, lab-based, study — where I have control of the scenario I want to test.

3. Be there

There’s simply no reason for (at least one of) the designers working on the product being tested to not be involved in user research. Any excuses I can think of: time, priorities, distance… are just that — excuses.

Just be there.

The value I get, as a designer, from being involved in a user research session, is much bigger than what I could ever get from any research artifact handed to me. If I’m not there no findings report can give me the motivation I can get from seeing real users struggle with the product. Even the best-made personas can’t bring the kind of empathy that comes from coming face to face with real users needs and struggles.

Regardless of how much experience I have with user research — maybe it’s none — I’m still the expert on something, the product being tested. But, unless I’m experienced with all aspects of conducting a research session, trying to be there in place of the researcher would also ruin the outcome.

This is what a research team looks like:

This is the expert. She will help me choose the best research method for my current needs and she will drive and facilitate the research sessions. Most often she will also conduct the interviews.

The hint is in the name: observers are there to observe. In most scripted, lab-based, studies the researcher won’t be able to moderate and take notes. It’s the job of the observers to do that. It’s important to have at least one product designer as an observer but that doesn’t mean all observers need to be designers. Empathy and user-focus are not the sole prerogatives of designers. Anyone in the product team — from product managers to developers — can benefit from and can contribute to user research.

It’s hard to underestimate how important selecting the right participants is. Even for moderated, lab-based, research. To get accurate results the scenario should be as real as possible — real users, or real prospects, that fit the necessary profile will make (or break) the test.

4. Capture Everything

I will invite 7 users for the study. I only really need 6 but I will schedule 7 because at least one will cancel. I know that from experience. 5 could work but if 2 people cancel… 7 it is.

It’s best to outsource the job of finding them. Without a large database of people willing to participate I don’t think I could ever find 7 people that fit the profile I deed for my scenario.

The next step is finding the right space. For the best results it needs to be as close to the natural use of the product as possible. For example, at booking.com, we furnished our in-house lab like a living room. Participants can choose if they want to sit at the desk or on the couch — depending also on the device tested. Getting a desktop computer and an external display on the couch has proven a little tricky.

The space also needs to have good internet, the recording and tracking equipment necessary and a good space for the observers. Since we’ll spend all day in there taking notes snacks and water are top priorities.

Next, the researcher will ask some context questions and introduce the scenario. She’s really careful not to ask leading questions or set up the conversation in a biasing way. When the user begins she will ask them to think out loud and give us some insight into their reasoning and motivation. Sometimes there’s strong dissonance between user’s actions and their words —in this case what they do, not what they say, is more important.

In another room the observers… we’re observing. This can be a video feed from the next room or from a different continent. Or, set-up with the vibe of a thrilling police interrogation, through a one-way mirror. Pro tip: if I’m behind a one-way mirror I’ll cover the logo of my MacBook — it’s not as one way as one might think.

I’ll use a spreadsheet to take in the moment notes and use the breaks between sessions to add more detailed observations.

• The first column in the spreadsheet will automatically capture a timestamp. This way I can focus on what’s happening and come back later to pay closer attention details like eye tracking or to find the exact moment in the video recording.
• The second column is for what is happening: what are participants doing? What are they thinking or feeling? These should be concrete and objective — there’s room for my own thoughts in the last column.
• The third column is for notes: ideas, improvements, notes. I keep these concise but I write them all down immediately — with all that’s happening I don’t want to count on my memory for this.

5. Generate Testable Hypotheses

At the end of the day, everyone — researcher and observers — will share their notes and collaborate do draw the first conclusions. Usually, this means comparing notes and observations, grouping common findings, sorting discoveries by importance and severity, and outlining some actionable insights to follow up on.

If everything went well the final artefacts (findings reports, personas, customer journeys, etc.) contain some answers to my initial question. OK, answers may be the wrong word — these come from just 6 users — they’re still assumptions until an A/B test can prove (or disprove) their value.

But I’ll stop here since the next steps are outside the scope of this article. How to formulate these assumptions as testable hypotheses? How to choose the right metrics? How to run an A/B test? I’ll write about that next so be sure to follow me if you’re interested. But there’s a more important, question left unanswered: did you spot the dad joke?