Getting User and Customer Feedback: Remote Interviews or Local Interviews?

Traditionally, customer feedback is collected in a face-to-face manner, be it usability studies, focus groups, or one-one interviews: Users come to on-campus facilities or off-site local 3rd party labs to be interviewed. The advantage of interviewing users locally is obvious: You get to observe their behavior in person, including body language, and you get to discuss with your stakeholders who are sitting in the observation room about additional questions they’d like to ask the participants.

Limitation of In-Person Interviews

Despite the numerous benefits, there’re a few issues associated with interviewing only local users:

1. Biased sample. This is a particularly serious issues in Silicon Valley, where most of the latest IT companies have their headquarters. I happen to be here and see the bias first hand — almost all users are tech savvy, have the latest models of smartphones, frequent Facebook, Twitter, LinkedIn, and other popular sites/apps. Unless your product only targets tech savvy users, this does present a highly biased picture — users are less bothered by usability issues and less intimidated by new product ideas.

2. Recruiting difficulty. Oftentimes, local users present a very limited pool, and you won’t be able to get enough users that fit your recruiting profile. For instance, in investigating new user experience towards smartphones, you might want to recruit people who don’t use smartphones but intend to try this out. Guess what, you simply can’t find those people easily in San Francisco bay area. You have to extend your search to other regions.

3. Lack of context. Interviewing users face-to-face in a facility, by definition, takes away the usage context–the users’ own computers, home environment, etc.–from your evaluation. Therefore this is not suited for contextual inquiry, a market/user research technique that focuses on interviewing users in their natural environment of doing things, such as workplace and home.

Remote Interviews

Given the limitation of local interviews, many have thought about interviewing users remotely, so that they can access uses nationwide or worldwide. That’s particularly useful if many of your target users can only be found outside of your own location. Nowadays remote conference systems such as Gotomeeting, WebEx, and Adobe Connect can allow you to 1) share screens, 2) share webcams, and 3) record the entire interview session for distribution. That works almost as good as interviewing users face to face while giving you access to people far away from your location. Plus, this gives you something in-person interviews can’t: Accessing users with more diverse background, and allow users to interact with your products in their own natural settings.

That said, remote interview does have its own drawbacks:

1. Can’t observe body language. The biggest drawback of remote interviews is that even if you talk with users through webcams, you won’t be able to see clearly their body language, which can give you additional ideas about their spontaneous, emotional reaction toward the product.

2. Hard to observe mobile UI. Whereas it’s quite easy to do screen sharing for you and the interviewee to both see Web or desktop UI, it’s harder to share mobile UI — there’re ways to project one’s mobile UI to the computer screen for sharing, but that’s indeed complicated setup and it’s hard to instruct your non-savvy interviewee to do it.

3. Harder to establish rapport. Given that users are on the other end of the internet or phone line, it just doesn’t feel like face-to-face meetings, and therefore create difficulty for you to establish rapport which is helpful for you to get users warmed up and tell you all their thoughts. That said, with more experience, you will find ways to break the ice and get through the users even through remote meetings.

Best Practice: Combining Local and Remote Interviews

Given the pros and cons of the two approach, I typically used a hybrid approach in market and user research studies I conducted: Have 1/3 or 1/2 of your sessions conducted in person, typically the first few sessions in your planned research, and have the rest of the sessions conducted remotely. By conducting the first few sessions in person, you get to observe users’ body language and have in-depth discussion with observers about what questions to explore when interviewing users. That helps you establish a baseline understanding of user behavior and refine your research questions. Then, for the rest of the sessions, you can do it remotely to reach out to a more representative sample of users outside of your own area.

Keep in mind, that whereas user interviews were traditionally conducted locally, it’s quite important for us to go outside of our own locales to reach out to more diverse users. This is especially important for Silicon Valley companies, where local users only represent a small and very tech-savvy fraction of overall user population.

To learn more about this topic:

Read more insights about how to conduct market research, evaluate usability, and gather customer feedback

By Frank Guo



Leave a Reply

Your email address will not be published. Required fields are marked *