In helping product design professionals and business leaders to leverage user research, a powerful truth-finding tool, to make better business and design decisions, I will discuss user research best practices through the lens of driving business benefits.
Guided by business objectives
Whereas user research is a practice associated with User Centered Design, besides driving better user experience, it is also an instrument to improve business results. Therefore, researchers should always try hard to use business objectives, in addition to UX considerations, in guiding test planning and the interpretation of results.
For example, when determining the types of users to interview, in addition to thinking of the users that are most impacted by the product, we should also understand the business opportunities associated with the different user segments. This understanding can help us prioritize the user segments to recruit.
Here’s another example. When proposing design recommendations, whereas certainly we should apply UX principles, we should also consider what design approach brings more business benefits. For example, among a few equally valid design solutions, we should intentionally choose one that leads to more business benefits (e.g., encouraging purchase behavior).
Conversations about user research should happen early, really early
User Centered Design is an integral part of product design and development – there is no business success to speak of if we don’t develop products that speak to user needs. So whether or not we plan to conduct formal user research, a conversation about user research should always, always happen at the earliest stage of product development. It should happen even before there is a design initiative – we should already talk about research at the point where business strategy is formulated.
The right way of introducing early-stage conversations around user research is not to frame the discussion around “research”. Rather, we can simply discuss with business owners about user needs, competitive landscape, business metrics, and perceived pain points. If there is a knowledge gap around these topics, then the need for user research naturally surfaces.
Thinking about success measures at the very beginning
Given that most user research intends to drive certain product design and/or business outcome, it is of great importance for the researcher to think about how design success is to be measured and incorporate such advice in the research report. The success measure should be derived based on in-depth discussions with the product managers and designers so that the whole team is committed to driving success based on the same criteria. This helps demonstrate the value of user research to business leaders and ultimately drive measurable improvements.
Success can be measured qualitatively
Measuring success comes with two flavors. One, undoubtedly, is quantitative in nature. For example, click-through rate, tracaction revenues, registration rate. On the other hand, not all user experience success can be measured quantitatively, and because numbers can be mis-interpreted, qualitative measurement can sometimes better reflect true improvements. For example, it is very easy to notice qualitative improvements in user experience when conducting user research studies — users are much less confused about the UI and are much more able to complete the same tasks. Whereas due to limited sample sizes, these observations are not statistically significant, they should be treated as an important indication of UX improvements.
Research can and should be done in an agile manner
User research is perceived by many as a rather formalized and systemic procedure. As such, it is often shun by organizations that focus on moving forward quickly with deliverables. Whereas there’s certainly truth in this notion, research can be done in many different ways, some of which are very flexible and can be applied to the so-called Agile development process.
A good example is the RITE (Rapid Iterative Testing and Evaluation) method, which supports multiple design-research iterations during just a few days of testing. Another example is persona development. Whereas there has to be a large, systemic effort in coming up with the initial personas, we can quickly update and modify the personas based on up-to-date user insights and the specific needs of the present design project. This means we don’t always need a formalized approach in developing personas.
1 + 1 > 2: Merging qualitative and quantitative methods
In providing a 360-degree understanding of user behavior, a great starting point is to think of research in quantitative and qualitative terms: qualitative research provides in-depth understanding of user behavior and drives effective UX solutions; quantitative research provides large-sample-size findings and lends lots of credibility to the insights. If we are ever able to do more than one study on the same topic, we should try to have at least one qualitative study and one quantitative study planned, because a combination of these two methods gives us a very powerful truth-finding tool.
A typical way of combining these two methods is to conduct qualitative study in understanding user behavior in an open-ended, exploratory manner, and then formulate a few hypotheses to be validated by a follow-up quantitative study.
There are many different ways of combining the methods. And even within one study, we can easily combine both approaches. For example, eyetracking is typically viewed as a hardcore quantitative method because its explanatory power comes from aggregation of user behavior across a large sample size. However, one can always incorporate think-aloud protocol after the eyetracking session in probing user motivations. Take focus group for another example, whereas it’s certainly a qualitative technique, by aggregating responses from many groups and leveraging quantitative techniques such as survey, we can derive some large-sample-size-based conclusions.
Synthesizing findings from multiple studies
Whereas research studies are done one at a time, no one study can truly provide a 360-degree understanding of user behavior. On the other hand, insights derived from individual studies can and should be synthesized in providing a more comprehensive knowledge base around our users. For example, by conducting, say, two rounds of usability studies of a particular product, we should be able to derive lots of understanding about users’ overall workflows and usage scenarios from these studies, assuming we build in the test script a few questions probing about these extra dimensions. These holistic insights can be integrated, again, with insights derived from analyses of Web metrics and quantitative surveys, and lead to the development of personas that reflect the full range of users’ psychological and behavioral traits.
Validating target users and use cases
Oftentimes, the target users and use cases of the product to be examined have already been identified by the product managers and/or business stakeholders. However, it is user researchers’ responsibility to review, challenge, and propose new ideas about the users and use cases during the interviews. For instance, let’s say the product managers think that large eBay sellers use eBay Web applications in their selling process, and in fact many such sellers hire someone else to deal with the computer stuff and have little direct knowledge of the Web applications used for selling. The implication of this finding is that eBay should design the Web applications to the needs of those that actually use the software rather than the eBay sellers themselves. This kind of learning helps the business align product development efforts to the needs of the true users and use cases.
Having the team participate while still maintaining control
It is very important to have the product team observe the research sessions in which users are interviewed. No research report can replace the understanding of users that one gains by observing first hand how users interact with the product.
Besides, after each research session, the team can discuss learnings based on what they just saw and come up with innovative solutions collectively. The product manager and the designers are likely to have deep knowledge of the product domain or design expertise. Incorporating the team’s feedback will help the researcher come up with design recommendations that make more design and business sense.
This, however, does not suggest that the analysis of research findings is a democratic process: The researcher is always the owner of the interpretation and the driver of any brainstorming sessions – he just leverages the team discussions to enrich and facilitate his interpretation of the results.
Planning the design before conducting the research
To effectively leverage user research in informing design decisions, it’ll be better if we do some preliminary work on the design front. For example creating a few alternative designs that illustrate competing design philosophies is a great way for us to provide more design insights through research. In doing so, researchers and designers need to figure out what key elements of differentiation between the design alternatives (e.g., different color schemes, different navigational models, different page layout) and collaborate on creating prototypes that accurately reflect these elements.
Trying to answer strategic questions
Certainly, many user research studies are conducted to mainly answer tactical questions posed by stakeholders. On the other hand, to truly deliver user and business benefits, we as researchers should always, always think of the bigger picture when planning the research and proposing recommendations.
For example, even when we conduct usability studies that have very well-defined objectives (e.g., evaluating the ease of use and content discoverability in relation to specific user tasks), we can use this as an opportunity to validate the overall design approach (e.g., is the UI work flow the most effective one in meeting user expectation) and business model (e.g., despite having great usability, does the UI effectively guide user behavior in driving conversion, registration, and repeat visits?)
This, requires researchers to do more than they’re asked. But this kind of value-added activities really elevates the quality of the research work to a different level.
Leveraging HCI and psychological principles
How well we as user researchers influence stakeholder behavior has lot to do with whether our insights are trustworthy. We can certainly use our extensive professional experience to make arguments. But to improve the quality and depth of our recommendations, we should reference to known human-computer-interaction principles and psychological mechanisms whenever appropriate.
For example, in explaining why we need to spell out call to action on the UI, we can reference to the fact that humans are better at recognition (i.e., seeing the action spelled out on the screen) than recall (i.e., trying to remember what action one should take without being reminded on the screen). This helps us connect observations of user behavior to scientific findings and lends depth and authority to our recommendations.
Keeping in mind actions to take as a result of research
We do not conduct research for its own sake — there are always some actions to take based on the research findings. This sounds like common sense, but unfortunately much too often none of us have a clear understanding of how exactly the research can help. For example, are we able to influence the entire design direction? Are we able to improve the detailed interaction design? Are we able to influence the visual identity of the brand? Or are we just trying to feel comfortable with our design but having no intention to really improve it?
All of these questions should be thought through before the research is conducted and a consensus must be achieved among team members in terms of the actions to take. And throughout all stages of research planning and execution, the researcher should keep in mind the actions to take, and in so doing they can create more targeted interview guide, are better able to follow up with in-depth probing questions during the interviews, and develop more actionable UX recommendations.
By Frank Guo