How to Ask Questions

The Basics of Asking Questions

When sitting face-to-face with users, you really want to focus on open-ended questions instead of writing questions that can have a short, definitive answer.

Do:

Ensure all questions are “opened ended” and leave space for your participant to continue to expand. For example:

1) Walk me through how you share files between your group and outside clients?

2) Show me how do you inform your team about these changes?

Don’t:

Ask questions that can be answered in one word. This makes it more difficult for participants to feel like they can qualify or provide additional context. For example:

1) Do you use this feature often?

2) Have you shared a file like this before?

• Pro Tip: Indicate that you are actively listening by nodding along and making strong eye contact. Leave long, awkward silences looking at your participant to encourage them to continue talking. Use phrases like “Go on…” or “Tell me a bit more about that” to nudge participants to elaborate.

Open Ended:
Always ask open-ended, non-leading questions. Do not ask questions that can be answered with ‘yes’, ‘no’ or any one-word answer.

Keep participants talking:
Use phrases like ‘Tell me more about that,’ encouraging body language and long, awkward silences to nudge users to continue to provide more details.

Frequency:
The frequency of an edge case or common task will help inform your design. Get quantifiable information. For instance, ask how many times out of 10 or 100 times such a situation occurs.

Recency:
Asking users to quantify recency helps quantify how important a task or feature is. If it’s seemingly rare, ask when it happened the time before as well.

Impact:
When a user recounts a situation where things awry, it’s important to contextualize the impact of the problem. What was the impact of the situation or edge case? Ask extra “Why” questions to build additional context.

Handling Edge Cases

Research participants are only human: they are apt to focus on recent or impactful edge cases and might encourage you to focus on solving a litany of rare edge cases. As researchers and designers, we are apt to hear plenty of users say “It depends” as a response to our questions.

Example of an unclear response:

• Researcher: Walk me through how you document agreements for your team?
• User: After receiving an update from the vendor, I usually post a message, where the privacy is locked to our team only. That way, only we see messages. However, I have to repost the update via email because sometimes our clients don’t get updates I post in our app.
• Researcher: When was the last time users missed an update that we posted in the app?
• User: Hmm… 2 years ago? We had this email outage and emails weren’t being sent. It happened a year ago as well – but he had forgotten his password and was ignoring the notification emails since he couldn’t log in.

Do:

Ask for recent examples and definitive evidence of the regularity of edge cases / alternative solutions. For example:

1) When was the last time this happened? The time before that?

2) How many times (per month, per quarter, per year) does this happen?

3) What other ways could we prevent or handle this situation?

Don’t:

Assume edge cases they mention happen as regularly or with as much severity. Don’t rely sold on only on qualitative evidence for edge cases.

• Pro tip: Ask users to guide you through real-world tasks while you watch. For instance, ask them to delay completing certain non-critical tasks until a certain time where you can shadow them, or ask them to call you over the next time a certain situation comes up.

It can be really tempting to find out if a user likes our design. Our designs are, despite our best efforts, close to our hearts. As humans, we want to be liked – and just because we’re an UX professional – we can still want our work to be liked.

Asking users to determine if they would like or use a certain feature is a waste of both your, your team’s, and your participant’s time.

Law & Order: SVU’s own DDA Barba will give you this look if you even think of asking a leading question.

Leading questions are one of the most prevalent culprits when it comes to underlying biases. Biased questions don’t help us design better software, in fact, they pollute our data. Think to the last time you watched Law and Order: SVU, you’ll notice that when in court other lawyers will object when their opposition is leading the witness with questionable phrasing.

Once you prepare your set of questions for a given user interview, have another designer review the questions with you. You can make sure you aren’t biasing them with phrasing or leading them to provide the data you need rather than the data that matches reality.

When prepping for a user interview, we can use a specific technique to help us write non-biased questions. Take a piece of paper, horizontally, and then slice it into three columns.

Column 1: Hypothesis

• “This new design will decrease user confusion when in XYZ flow.”
• “Our new app will decrease the time from document complete to client invoiced, to under 3 days”
• “This feature will increase the rate of sign-ups on our website by 15%”
• “Users are struggling with communicating with their employees on the ground.”

If you can, include how you will be structuring this research session with:

Independent Variables: What we are monitoring for

Dependent Variables: What we are changing (design, feature, flow)

Measures: Then mention how you will measure yourself if it’s quantifiable

Column 2: Question Drafts

Let’s start by writing basic questions with leading phrasing. We can be as biased as we want to be here. For example:

• “How much easier is this design to use than the current system?”
• “How will this app decrease turnaround time?”
• “What are your pain points with your current employee communications?”

Just get the questions out – don’t worry about formatting. This step is designed to let you write questions that are knowingly somewhat leading.

Column 3: Sanitize Questions

Okay, in the last column, let’s take each question and do our best to format it without leading phrasing. Example:

• What changes to this design are most noticeable? Why?
• How would this app impact your workflow?
• Walk me through a situation where you communicate really well with your employee?
• Now consider a time that you couldn’t communicate easily, and walk me through that situation. How was it different?

Next, read Cognitive Biases