RESEARCH IS NOT…
- A demo to sell a product
- Word for word everything the user wants, sometimes known as “Requirements Gathering”. We look for pains, gaps and needs. We dig deeper to ask why they want something, when they last needed it & build a timeline for such functionality.
RESEARCH BIASES
Biases can and will interfere with gathering valuable and accurate research. Be aware of the following biases:
- Social Desirability Bias (participant)
- Answer questions in a way that will lead to them being liked & accepted
To counteract: positive reinforcement of criticism or negative feedback
- Sponsor Bias (participant)
- Feelings about the sponsor, such as a company, client, or stakeholder, may bias their answers
To counteract: Stay neutral about sponsor feedback
- Confirmation Bias
- Using participants answer to confirm their own belief. We tend to remember points that support our hypothesis and filter out points that disprove.
To counteract: Re-evaluate participant’s expressions, perform research synthesis as a group, challenge preexisting assumptions
- Leading question and wording Bias
- Elaborating on a participant’s answer puts words into their mouth
To counteract: Use participant’s own words, avoid summarizing and ask them to summarize once more instead
RESEARCH INTERVIEW DOS AND DON’TS
Below is a list of interview dos and don’ts that you can go over with your team to help them craft questions that draw insightful information from people.
Do: Start the interview with simple questions.
Don’t start with hard questions. Ease into them.
Starting with easy topics will help people get relaxed, comfortable, and talking.
| Good Question |
Bad Question |
| Tell me a little bit about what you do? |
You answered that you hate __ in our recruiting survey. Why do you hate ___ ? |
|
|
Do: Ask open-ended questions with “What”, “How” and “Why”.
Don’t ask leading questions or lead users through tasks.
Don’t ask questions with “Did”, “Is”, or “Would” that provide only yes or no answers.
The goal is to hear people speak in their own words as much as possible. It’s very easy for people to say they would buy your product until they actually have to pay. Past behavior is the best predictor of future behavior.
| Good Question |
Bad Question |
| How are you doing? |
Are you having a good day? |
| Can you tell me the whole story of? Then what? |
Do you like this and find this helpful? |
| Why did you do that? |
|
| How did you do that? |
|
|
|
Do: Seek concrete stories, past actions & behaviors.
Don’t seek generalizations or ask about future behavior.
The goal is to understand what happened in their past to give us clues about why they think or feel things today.
| Good Question |
Bad Question |
| Can you talk about the last time you used this? |
Do you typically do this? |
| What did you do that time and why was it different? |
What do you normally do? |
| What do you think about this product? |
Would you pay for this? |
| Tell me about the last time you had a painful experience with this product. |
What do you want? |
|
|
Do: Give people time to answer the question.
Don’t fill in silence with more questions.
The goal is to give people space to think because being interviewed is difficult. You can also use silence as a way to extract even more information that people may be hesitant to give. Related, also feel free to move forward if the conversation is digressing or come back to the question if someone is having a hard time answering.
| Good Behavior |
Bad Behavior |
| Silence |
Interrupting while interviewee is speaking |
|
|
Do: Nod and say less. The interviewee is always right.
Don’t correct or contradict a research participant if they do something “wrong”.
The goal is to get people speaking as honestly as possible and to encourage them to feel it’s okay when they don’t understand how to use something in an interview. Correcting interviewees makes them feel that they are being tested and they can easily become afraid to speak their mind for fear of looking dumb.
| Good Interviewer Behavior |
Bad Interviewer Behavior |
User: This part doesn’t make sense. Is it supposed to do that?
Interviewer: What do you think it should do?
User: I think it would swap out all the pictures.
Interviewer: There is no actual answer here, we want to learn about what you expect. Thanks! |
User: This part doesn’t make sense. Is it supposed to do that?
Interviewer: Oh, it’s a slideshow and it’s just broken in the prototype. You’re supposed to tap on the arrow and the next image should enter the page.
User: Oh I didn’t see that, sorry. |
|
|
Do: Actively listen and dig deeper with “why” questions.
Don’t ignore users when they are struggling in silence, or they show surprise.
Dig deeper and ask why they are reacting a certain way.
The goal is to pay attention to the user as they are speaking and feel comfortable going off script when you see or hear something interesting. It can be easy to read straight off the script without asking follow up questions but it means that we miss out on something that may be valuable for us to know.
| Good Interviewer Behavior |
Bad Interviewer Behavior |
Interviewer: What do you think this page does?
User: Oh! I like that. Anyway, I think this page tells me my sleep duration.
Interviewer: Let’s rewind for a second, what did you mean by “Oh! I like that.”? |
Interviewer: What do you think this page does?
User: Oh! I like that. Anyway, I think this page tells me my sleep duration.
Interviewer: Great, what would you do next? |
|
|
Do: Stay neutral about the product you are showing.
Don’t talk about or sell your product.
The goal with research is to have as unbiased feedback as possible. For example, interviewees should feel . Talking about your opinion, or even telling them what the product is supposed to do will subtly encourage interviewees to agree with you. Interviewees will feel inclined to be polite and make you feel good about your product to begin with.
| Good Interviewer Behavior |
Bad Interviewer Behavior |
| Interviewer: We’re interested in having you test drive an idea that’s related to sleep cycles. |
Interviewer: We’re working on this mobile app that gives you an easy way to look at how much sleep you’re getting at night. |
|
|
Do: Wrap up with broad questions.
Don’t wrap up without having interviewees reflect on their experience.
Interviewees will have more to say. You can capture many interesting insights during the wrap-up.
| Good Interviewer Behavior |
Bad Interviewer Behavior |
| How was this experience for you? Is there anything else pertaining to sleep that you’ve been thinking about lately? |
We’ve got no more questions for you. Thanks, bye! |
|
|
INTERVIEW PRACTICE SILENT QUIZ
Here are 3 “bad” questions. Turn them into good questions.
- Was that helpful for you?
- What do you want?
- How do you like this?
INTERVIEW PRACTICE
Interview each other by only using “Tell me more about…” or “Why?”
Remember to regroup and learn from mistakes. Perfection is impossible.
RESEARCH SAMPLE STRUCTURE
Introduction
- Introduce yourself and note taker.
- Explain how long the interview will take and the general format.
- State the goal is just to have a discussion about their experience with ______ .
- No right or wrong answers to questions, the only right answer is their opinion. Let him/her know nothing they say will offend you.
- Mention they should feel free to ask questions anytime; they’ll have time at the end for questions too.
- Have you signed the Non Disclosure Agreement?
- Do you mind if we record the session?
- Any questions before we begin?
Storytelling Questions
These questions are generally asked during Exploratory Research.
Use these questions to explore these topics:
Who, What, Where, When, Why, How
- Tell me about what you do. What are your top responsibilities?
- Tell me about the last time…? And then what happened? Why (or how) did you do that?
- Tell me about a time that was difficult…?
- Your favorite moment…?
- What reasons helped you choose…?
- Anything else around phones?
- If you had a magic wand, what would you…?
- Scenarios/activities
- Real world context
- Pains and Joys
- First time, last time
- Best time, worst time
- Wish list
Reactionary Questions
These questions are asked generally during Validation and Usability Research.
- Can you describe the things you’re reading? What do you see?
- What do you think this page does? What can you do here?
- What questions are coming to your mind as you’re looking at this page?
- Before clicking on anything, what would you do next?
- What do you think would happen next?
- Why do you think that?
- When you mentioned ___, what did you mean?
- So what happened there?
- Was that what you expected? Why or why not?
- So what goes through your mind as you look at this?
- Which part of the page were you looking at?
- Did you find what you were looking for?
- What would you do next? Why?
- Is there anything else you would do at this point?
- Is there any other way to do that?
- What did you think of that?
- In what ways would you want this changed to make it better for you?
- What additional info would have helped?
Wrap Up Questions
- We’ve been talking about ___ for awhile, has this brought up anything you’ve been thinking lately?
- What were the top 2 things that worked well and didn’t work well?
- What was not clear? What would you change?
- What impact would this have given your situation?
- What does this remind you of?
- If we waved a magic wand to make this service better for you, what would that be?
- Is there anything else regarding … that you’ve been thinking of?
- What do you like/dislike about this?
- If you had 3 wishes to make this better for you, what would they be? Why?
- How would you describe this to a friend?
- Under what circumstances would you use this? Why?
- Can you describe to me what you see on this page?
- Which parts of this page are most/least important to you?
- What do you think this [point to a UI element] might do?
- What does this [point to a UI element] means?
- If you wanted to _______, how would you. . .?