In my last post, I discussed why you should conduct lean UX research in order to induce great user experience through Agile development process.
Here, I’ll explain how to conduct lean UX research in dealing with the tremendous timeline and planning pressure posed by the Agile process — that requires you to be creative and leverage alternative user research methods. Let me go through them one by one.
Conduct UX research to complement A/B tests
Given that a big part of the Agile process is test-and-learn – test here typically refers to A/B testing – we can conduct UX research to complement A/B testing. A typical way to do this is to conduct a usability study on the different variations currently being A/B tested. Given that the product is already live with the different variations, it’s very easy for us to test the product, as there’s no need to do prototyping or wireframing in preparing for the usability study.
Conducted in conjunction with A/B testing, the usability study can tell us “why” one variation is better than the other, and if a better solution outside of the variations tested exist.
At one point I was asked to conduct a usability study to evaluate the variations of a live-site A/B test in order to encourage users’ shopping behavior. Whereas the A/B test gave us some early indication of which design would fare better, I used the usability study to provide in-depth UI and content recommendations, pointing out solutions that exist outside of the four variations A/B tested. End result: combining insight from the UX research with data from the A/B test, I helped the client create an experience in which users were much more likely to go through the shopping flow, and we saw a truly dramatic lift in revenue as a result.
Conduct UX research throughout product development cycle
Traditionally UX research is supposed to be conducted before the development stage. Given the rapid iterations of product development, that leaves little room for conducting research. The thing is, exactly due to the iterative nature of the Agile development process, you can conduct research during the development phase (through a QA environment) or after a product is launched. Why’s that? Well, because the insight will turn into action for the next release, which could be just weeks away.
Another benefit is, testing QA versions or live products saves us lots of time on creating prototypes for testing, and can yield better insight because the product is fully functional and users can explore it in their natural ways to give context-relevant feedback.
In an Agile world, there’s no wrong time for conducting UX research!
Much of the UX research I conducted for a client recently happened after the product launches. The findings and insights went a long way in driving the improvements of the next release, and the following release, and so on – due to the rapid development cycle, we went through the incrementally improved releases quickly, absorbing user insight collected from the usability studies. During the process, we didn’t slow down a bit for the development to wait for the research to happen as the studies were conducted on the live site. End result: Through multiple rounds of testing and product releases, I helped the client launch a very successful new feature on their website.
Develop ongoing user research program
You can put in place an ongoing research program with a pre-determined testing schedule and ongoing recruiting effort. For example, your company can conduct user interviews on a monthly basis, with different topics for each month. Because the schedule is fixed and known in advance, the team is able to plan and recruit participants with a lot of lead time. In an Agile environment, due to the speediness of development and design changes, you will always find something worth testing every month (see the graph below).
I’ve conducted such ongoing UX research for a client. Each month, we received requests from various product teams in terms of topics to test and we planned accordingly. After the monthly user interviews, we shared the findings immediately after the user interviews to give the team quick feedback. The end result: The program, while being very cost effective, had informed the company of up-to-date usability issues and user concerns on an ongoing basis – and led to quick solutions and many new product ideas to fill the gaps.
Leverage existing data and user insight
What if you really, really don’t have any resource for conducting any UX research? That does happen if there’s little organizational support for conducting research or if you’re a very limited budget or experience for such research.
Well, another research alternative is instead of conducting a separate study, we can leverage existing industry analysis and research reports to extract consumer insight and develop guidelines to shape the product features and design. There’re plenty of industry analyses, both paid or free, that can be leveraged.
On the design side, Nielsen Norman Group provides a lot of free UX design guidelines, covering almost every area of UI design. On the product management and business planning side, you can leverage industry analyses from Corporate Insight, Kasina, and Forrester, all consulting firms providing syndicated research reports, to help teams focus on the right product features.
A word of caution: no existing research data can ever replace conducting customized research for your particular product. Every product is different, every target user segment is different, and every UI is designed differently. Conducting UX research based on your business context is always, always a must for developing great products that users love.
When I was leading UX strategy for a major finance company, I didn’t have all the resources to conduct customized research for all the product launches. I frequently leveraged syndicated reports from the likes of Kasina and Forrester, in conjunction with findings from past research conducted internally, to help the eBusiness organization identify product opportunities and UX solutions. End result: Our products covered key features competitors offer, and our UI design started with a solid foundation based on past usability study findings.
Be creative about process
In order to operate under tight timelines, we need to be very creative about research process. Traditionally, user research follows a somewhat rigid process, taking a long while to move through the steps: planning the study, getting stakeholder buy-in, recruiting participants, creating discussion guide, and developing a full report.
But in an agile environment, we need to drastically shorten the cycle by doing more things at once, taking out unnecessary steps, and applying alternative research approaches. For instance, you can use friends and colleagues as participants, assuming they fit your target user’s profile, you can create a high-level discussion guide that addresses key research questions but that doesn’t contain detailed verbiage used for the interview, and you can write a quick findings report rather than a fully-developed, polished report.
I was able to conduct such informal usability studies, in the form of diary-style short surveys, to influence the design requirements and improve usability of the first launch of an iPhone app for a major financial company. End result: The iPhone app, greatly benefiting from the informal user research, later won a Silver award from w3c as well as received a five-star average rating in the app store.
Read on to see why lean UX research is a quick but not dirty way to understand your users.
By Frank Guo