Category Archives: Customer Research & UX Research

Getting User and Customer Feedback: How to Understand Customer Experience Holistically?

In previous posts we discussed a bunch of ways of collecting customer feedback. Thing is, none of the approaches mentioned in itself gives us complete understanding of customers. The key always resides in synthesis–analyzing information from all sources to form a complete picture. Let’s see how we can look at the different sources of customer data and develop holistic customer insight. Continue reading

Getting User and Customer Feedback: Remote Interviews or Local Interviews?

Traditionally, customer feedback is collected in a face-to-face manner, be it usability studies, focus groups, or one-one interviews: Users come to on-campus facilities or off-site local 3rd party labs to be interviewed. The advantage of interviewing users locally is obvious: You get to observe their behavior in person, including body language, and you get to discuss with your stakeholders who are sitting in the observation room about additional questions they’d like to ask the participants.

Limitation of In-Person Interviews

Despite the numerous benefits, there’re a few issues associated with interviewing only local users: Continue reading

Validate Your Products: A/B Testing or User Experience Research?

There are two main approaches of validating your products to see if they perform as expected and identify areas for improvements. One is user experience research, which includes usability studies as well as other forms of user interview methods, gathering user feedback through asking user questions, surveys, and direct observation of user behavior.

The other is A/B testing, also called “launch and learn”, through which we randomly present different versions of the launched product to different users and observe the differences in the resulted Web, mobile, and business metrics.

Given that both approaches are talked a lot about, based on what I’ve seen among clients I helped, there’s much confusion around choosing between the two approaches: Do we need to use both methods? Can we just do A/B testing and forget about user research? If we need to do both, then when do we use each method? Continue reading

Getting User and Customer Feedback: Usability Study vs. Focus Group vs. Survey — Which Should You Choose?

When it comes to the all-important issue, listening to user/customer feedback, the first thing that comes to mind for many is focus group or survey. The tendency is so persistent that even when I conducted usability studies, the stakeholders, typically product managers and business owners, kept referring to the exercise as “focus group” or “survey”!

On the other hand, designers and programmers trust usability testing much more and don’t see or understand value of focus groups and surveys.

So, what are the differences between the three and when should we apply each technique? Continue reading

Market Research vs. User Research — Which Should I Use to Understand Customers?

We all want to collect customer feedback, right? Then you might have heard of market research and user research, methods that allow you to systematically gather and analyze customer feedback. But then again, you might be wondering, what’s the difference between the two?

For most of you, I guess, you couldn’t care less about the nomenclature as long as you get the customer insight you want. However, in the corporate world, these two functions do belong to separate departments and, as such, you do need to know which one you should turn to if you seek their help. Even if you hire independent consultants or do it by yourself, a basic understanding of the two approaches would help you get high quality customer feedback.

Market Research — Focusing on Monetization

Continue reading

Getting User and Customer Feedback: How Many Users Should We Talk to?

Most of us working on product development know the importance of getting user feedback, but how can we effectively drive actions based on user feedback? That’s the question.

One of the key questions I heard people asking is: How many users should we talk to?

The answer is, of course, it depends.

To Improve Usability, Interview as Few as Five Users Would Do

Continue reading

From Customer Journey to Business Opportunities

Ever wonder how to identify business areas you should improve based on customers’/users’ true needs rather than spend money on something that doesn’t matter to them? Looking at their end-to-end journey — across all touch points with your product/business — is a great way to start!

By Frank Guo


Lean UX Research: Quick, but Not Dirty

In  my previous posts, I discussed why you should conduct lean UX research to induce great user experience through Agile development and how to do this. Some of you might be concerned with a lack of quality insight in lean UX research. I’ll address the concern in this post.

“Lean” Doesn’t Mean Poor Quality

Lean UX research quality

Continue reading

How to Conduct Lean UX Research to Support Agile Development?

In my last post, I discussed why you should conduct lean UX research in order to induce great user experience through Agile development process.

Here, I’ll explain how to conduct lean UX research in dealing with the tremendous timeline and planning pressure posed by the Agile process — that requires you to be creative and leverage alternative user research methods. Let me go through them one by one.

Conduct UX research to complement A/B tests

Given that a big part of the Agile process is test-and-learn – test here typically refers to A/B testing – we can conduct UX research to complement A/B testing. A typical way to do this is to conduct a usability study on the different variations currently being A/B tested. Given that the product is already live with the different variations, it’s very easy for us to test the product, as there’s no need to do prototyping or wireframing in preparing for the usability study.

Conducted in conjunction with A/B testing, the usability study can tell us “why” one variation is better than the other, and if a better solution outside of the variations tested exist.

Case study:

At one point I was asked to conduct a usability study to evaluate the variations of a live-site A/B test in order to encourage users’ shopping behavior. Whereas the A/B test gave us some early indication of which design would fare better, I used the usability study to provide in-depth UI and content recommendations, pointing out solutions that exist outside of the four variations A/B tested. End result: combining insight from the UX research with data from the A/B test, I helped the client create an experience in which users were much more likely to go through the shopping flow, and we saw a truly dramatic lift in revenue as a result.

Continue reading

Is UX Research Dead in the Agile Development World?

Agile software development process gained tremendous popularity recently, adopted by many companies to deliver high-quality products through iterative launch and testing.

In contrast to the traditional Water Fall model, in an Agile environment the design and development teams collaborate very closely and there is little step-by-step procedure or upfront planning – decisions are made and solutions are implemented on the fly, in a highly iterative and flexible manner.

However the lack of planning and lead time in the process apparently poses a major challenge to user experience research. Remember, UX research is supposed to bring a strategic perspective into software development, helping the product team understand the big picture and focus on the right things to work on based on user insights. But the making-decisions-on-the-fly mindset underlying the Agile process makes conducting UX research seemingly hard to do and unnecessary.

So here come the questions:

Is UX research even needed any more in an Agile environment?

If so, then how do we conduct UX research in this context?

The answer: Lean UX research – conducting research in a quick-but-not-dirty way.

Lean UX full size

Test-and-Learn is Not Enough – Garbage in, Garbage out

Continue reading

Develop New Products that Customers Want – A Step-by-Step Guide

For entrepreneurs and product managers, developing new products and uncovering new markets pose great challenges, as they are in an uncharted territory with little guidance. That’s why it is particularly important to gather customer feedback to explore, validate, and improve the product vision and direction at a very early stage. However, I’ve seen many times entrepreneurs and product managers dived into UI design and coding without first evaluating the concept, the single most important step of customer validation.

Continue to read the full step-by-step guide

By Frank Guo


Finding Business Priorities: What Matters to Your Users? – Part II

Here is Part II of my three-part series published on UX Matters, which describes the different aspects of user experience and how we can develop better products based on the framework: More Than Usability: The Four Elements of User Experience, Part II

By Frank Guo


Finding Business Priorities: What Matters to Your Users? – Part I

Whereas user experience and usability have been used almost interchangeably in many occasions, through my conversations with many product-design professionals, I’ve found that “usability” is being increasingly used in a narrow context, in which it specifically refers to the ease of task completion and is closely associated with a “testing” connotation. On the other hand, “user experience” is used by practitioners in much broader contexts, referring to things ranging from ease of use to user engagement to visual appeal, and therefore I believe is a better term in capturing all the psychological and behavioral elements of user interactions with products. Please check out my article on UX Matters, More Than Usability: The Four Elements of User Experience, Part I

By Frank Guo


Why Is NPS a Poor Measure of Web and Mobile Customer Experience?

If you have to name one measure of customer experience that has ubiquitous acceptance among senior executives, it’s NPS, short for Net Promoter Score, a well-studied and indeed very simple way to measure customer loyalty to a brand. Lots of research has shown a strong correlation of NPS and revenue growth.

What’s the Problem, Then?

On the other hand, having conducted customer experience surveys for many years, I’ve found NPS to be a poor and misleading measure of online, mobile, and social media experience. Furthermore, whereas NPS is a great tool to help the company focus on customer loyalty through an easy-to-understand concept, it offers very little help when it comes to developing customer experience solutions. Continue reading

Choosing User Research Methods Based on Your Business Objectives

Through investigating user needs and behavior, user experience research is critical in guiding our product strategy and design. There are many techniques leveraged by user researchers. Over the many years of working in the field, I have seen a persisting tendency of choosing research methods based on techniques and not business objectives. For example, we often hear people saying things like “Why don’t we conduct an eyetracking study? It seems very exciting! For such a large re-design project, we can’t afford not doing it.” and “Let’s conduct a survey to learn about our client needs.” Continue reading