There are two main approaches of validating your products to see if they perform as expected and identify areas for improvements. One is user experience research, which includes usability studies as well as other forms of user interview methods, gathering user feedback through asking user questions, surveys, and direct observation of user behavior.
The other is A/B testing, also called “launch and learn”, through which we randomly present different versions of the launched product to different users and observe the differences in the resulted Web, mobile, and business metrics.
Given that both approaches are talked a lot about, based on what I’ve seen among clients I helped, there’s much confusion around choosing between the two approaches: Do we need to use both methods? Can we just do A/B testing and forget about user research? If we need to do both, then when do we use each method? Continue reading
In my previous posts, I discussed why you should conduct lean UX research to induce great user experience through Agile development and how to do this. Some of you might be concerned with a lack of quality insight in lean UX research. I’ll address the concern in this post.
“Lean” Doesn’t Mean Poor Quality
In my last post, I discussed why you should conduct lean UX research in order to induce great user experience through Agile development process.
Here, I’ll explain how to conduct lean UX research in dealing with the tremendous timeline and planning pressure posed by the Agile process — that requires you to be creative and leverage alternative user research methods. Let me go through them one by one.
Conduct UX research to complement A/B tests
Given that a big part of the Agile process is test-and-learn – test here typically refers to A/B testing – we can conduct UX research to complement A/B testing. A typical way to do this is to conduct a usability study on the different variations currently being A/B tested. Given that the product is already live with the different variations, it’s very easy for us to test the product, as there’s no need to do prototyping or wireframing in preparing for the usability study.
Conducted in conjunction with A/B testing, the usability study can tell us “why” one variation is better than the other, and if a better solution outside of the variations tested exist.
At one point I was asked to conduct a usability study to evaluate the variations of a live-site A/B test in order to encourage users’ shopping behavior. Whereas the A/B test gave us some early indication of which design would fare better, I used the usability study to provide in-depth UI and content recommendations, pointing out solutions that exist outside of the four variations A/B tested. End result: combining insight from the UX research with data from the A/B test, I helped the client create an experience in which users were much more likely to go through the shopping flow, and we saw a truly dramatic lift in revenue as a result.
Agile software development process gained tremendous popularity recently, adopted by many companies to deliver high-quality products through iterative launch and testing.
In contrast to the traditional Water Fall model, in an Agile environment the design and development teams collaborate very closely and there is little step-by-step procedure or upfront planning – decisions are made and solutions are implemented on the fly, in a highly iterative and flexible manner.
However the lack of planning and lead time in the process apparently poses a major challenge to user experience research. Remember, UX research is supposed to bring a strategic perspective into software development, helping the product team understand the big picture and focus on the right things to work on based on user insights. But the making-decisions-on-the-fly mindset underlying the Agile process makes conducting UX research seemingly hard to do and unnecessary.
So here come the questions:
Is UX research even needed any more in an Agile environment?
If so, then how do we conduct UX research in this context?
The answer: Lean UX research – conducting research in a quick-but-not-dirty way.
Test-and-Learn is Not Enough – Garbage in, Garbage out
I’ve worked on quite a few Agile-based product development projects, noteworthy among them are an iPhone app design project and a social-collaboration design project (Jive-based). Whereas we’ve achieved great user experience results through those projects, the ultimate test has yet to come – building one’s own website, where one has to constantly balance and combine all aspects of web development, such as design, user testing, coding, and content writing, within a really short time frame with very limited resource. Let’s examine the challenges and solutions: Continue reading