How To Build Your Hypothesis

In previous posts, we talked about the ROI of customer feedback and how to think about customer experiences in more productive ways.

You probably have strong ideas of who your customers are and what they need. The real challenge is to validate if your ideas are the right ones. A good way to start is to speak to customers or potential customers about the problems they’re trying to solve. But before you jump into interviewing anybody, start thinking about what hypothesis you want to validate.

We highly recommend checking out The 5 Components of a Good Hypothesis by Teresa Torres. Teresa suggests that the following elements need to be included in your hypothesis for it to be supported or refuted by an experiment:

  • The change that you are testing
  • What impact you expect the change to have
  • Who you expect it to impact
  • By how much
  • After how long

A good example of a hypothesis that contains these elements is:

Increasing the onboarding steps from 3 to 5 will increase new user engagement by 10% in the first week after signing up.

For your customer research project, your hypothesis may look more like this:

I believe that users who export data every week will share 15% more reports with colleagues if we automate reports for them.

Hypotheses are informed guesses and should include as much detail as possible. This will help you understand who to talk to and what kind of questions you should ask to prove or disprove your hypothesis.

Now that you have a clear hypothesis, we can start sourcing people for your interviews.

Getting Your Customers To Engage With You

There are over 95 ways to get people to talk to you. If you don’t believe it, check this incredibly detailed blog post by Jason Evanish: 95 Ways to find your first customers for customer development or your first sale.

Jason covers specific tactics based on a long list of channels you can use to source interviews. Many more channels have emerged since the post was published in 2013, so in theory, it has become increasingly easier to connect with strangers and ask them for 20 minutes of their time.

The challenge isn’t in identifying people you could talk to, but in convincing them to give you some of their precious time. The average Joe is much less likely than you to be passionate about giving feedback, especially if it means being taken away from something that seems more important.

Here are some principles that will help you increase the chances of getting people to talk to you:

Be 100% clear: If you’re asking people to go out of their way to spend time with you, they need to be clear on what the effort entails. Describe in few words the research you are working on, the type of people you are trying to talk to, and what’s required of them if they choose to participate.

Make it easy to arrange: The process of arranging the interview needs to be as frictionless as possible. For example, don’t ask for their availability, but share yours. To help people to easily book a time, use a public calendar like Calendly.com and set it with flexible time frames that’ll help them find a convenient time to talk.

Another important detail is to provide more than one way to communicate: share your Skype ID, Google hangout link and an alternative channel, because sometimes technology breaks. Dictate the plan for prioritizing communications channels. Tell them what they should do if one of those channels doesn’t work. And yes, the good old fashion phone works too. But ideally, you’ll get the chance to see the person face to face or on a video call.

Ongoing communication is key: Once you’ve arranged the call, send one or two reminders, as people will likely forget they booked some time with you. This is your project — it’s your responsibility to make sure they remember what they committed to do and when they committed to do it. In your reminder (which may be automated by your public calendar platform), include details about the primary channel of communication and the backup plan.

Recommendations are free: Don’t miss the chance to ask for recommendations of people who your current subject believes would be interested in participating. Chances are, the person you just interviewed also knows somebody that may be able to help. There’s nothing more useful than a warm intro from someone who has already experienced what you are asking for. Try to get as many as possible.

Here’s a template you can use to arrange your interviews:

Hi [insert name],

Thanks for replying to my [tweet, ad, intro etc]. As I mentioned before, I am doing research for [insert project] and I am trying to talk to [insert specific description of target audience]. It would be great if I could ask you a couple of questions about your [day, week, routine].

The interview will take a maximum of [Insert length] and we can arrange it at your most convenient time. Video interviews are most beneficial in these cases, so if it ok with you, we’d love to use [insert channels, Skype, Hangouts, etc] or any other channels that works best for you.

To make things easier, here’s my availability. You can book a time that works for you here: [insert calendar link]

After you’ve booked a slot, I will follow up with instructions for the call.

Thanks again for taking the time!

[Your name]

This approach would change slightly if you’re talking to current customers or people who already have a relationship with your product. In that case, you will have a more specific research goal. For example:

  • I need more insight into a particular feature
  • I’m seeing higher churn rates than I’d like
  • I want to understand how to align value with my pricing
  • I want to understand what is behind our promoters’ comments
  • I’m curious about how we measure up to our competition

In this case, you may want to segment your audience based on that research goal. For example:

  • People who requested a specific feature or are using a specific feature
  • Customer who canceled, especially those who left without any correspondence
  • Customers that signed up but never came back the product
  • Highlight engaged customers who have been actively using the product for more than six months
  • Any of the above + (location), (type of plan), (products purchased), etc.

The segmentation of your interviews will depend on what you want to learn based on your very detailed hypothesis.

Good Questions vs. Bad Questions

At this point, you have a clear hypothesis and you have sourced a few people from your target market or current customers to talk to. Now, it’s time for the interview. Here are some best practices for understanding the underlying motivations of your customers.

Bad Questions

Aspirational questions: Aspirational questions are very easy to answer with a yes or no. These questions normally relate to hypothetical scenarios. For example:

  • Would you be interested in a product that helps you do X?
  • Would you try a product that does XYZ?

Confirmation questions: These questions put the interviewee in a position where agreeing with you is the only option. For example:

  • Isn’t great how XYZ can be done faster thanks to products like X?
  • A lot of the people I talked to think X. Would you agree with that?

Good Questions

Get them to tell you stories: To get interviewees to enter a hypothetical world and focus on what they would actually do in real life, ask them about a time they were using your product and had to solve a problem similar to what you are investigating. Or, if they’re potential customers, ask about the problem only. For example:

  • Tell me about a time when you had to do X.
  • When was the last time you had to do X? Can you tell me how you got to that point?
  • If you are currently solving the problem, could you tell me how do you are doing that?

As they share their story, make sure you listen carefully and take notes. Dig deeper into the story by asking “why?” as many times as possible. Asking why will help you uncover the underlying pains or motivations behind the actions they took.

A remarkable book to help you nail your interviewing skills is Lean Customer Development: Building Products Your Customers Will Buy by Cindy Alvarez. We highly recommend taking a look if you’re still feeling hesitant about conducting your interviews.

The goal is to learn as much as possible about what your target audience is doing today, how they describe their problem, how they have tried to solve it in the past, and what products they’ve used before. You want to have a clear understanding of what people actually do and why, because you can’t possibly predict every individual method. It’s your responsibility to take these stories and apply them to the greater good of the product.

A common mistake many teams make (we’re guilty of it too) is trying to sell your product during customer development or user research interviews. Your job is to stay as neutral as possible. You don’t want to influence the interview in any way. This is easier said than done. 🙂

If you truly want to learn from current customers and potential customers as they evolve, you need to leave your ego at the door. Become a great listener and avoid asking questions that can lead your interviewees to say what you want to hear. Your goal for these interviews is to seek the truth, even if it is painful. The whole point of these exercises is to be proven wrong. You want to learn the truth so you can understand where you stand and how to improve.