Making Sense Of Your Customer Feedback

Humans are complicated. From biases to fallacies, humans are pretty bad at making rational decisions or predicting the future. That’s what makes feedback analysis so interesting. When making sense of customer feedback, you’ll often find that what most people say they want is often not what they need.

Behavioral psychologists have been studying all the reasons why people say things they don’t actually think or mean. Among the reasons are the “herd mentality,” where they feel pressure to follow fall inline with the majority. Another is wishful thinking, where people will convince themselves that something is true about them just because they want it to be or because of a preconceived notion. And of course, people have different mindsets in different contexts, making it nearly impossible to predict how we’ll feel about something in the future – in a different context.

But that doesn’t mean you should ignore what customers say. It only means you need to listen more carefully while paying attention to what they do as well.

Analyzing feedback requires putting your researcher hat on to see the data as evidence or signals, not as the ultimate truth. You’re looking for patterns — a way to understand what affects customers’ experience and what’s stopping them from being successful with your product.

Not All Feedback Is Created Equal

Just as important as the content of the feedback is the context in which it was created and the person who provided it.

At a high level, you can look at the feedback you’ve received from a specific period of time and identify themes by looking into repetitive comments. Once you have identified some commonalities, it’s time to segment the data based on who is providing the feedback.

For example: Is the feedback coming from paid or free tier plans? Is the feedback coming from a specific customer segment? Is it from customers in a specific location? How long have they been with the business? Are these customers more or less valuable than others? Meaning, are they sticking with your product longer? Are they upgrading and referring your product to other people? Who are the customers driving the most monthly recurring revenue?

By segmenting feedback based on customer type, behaviors, or specific properties (what they do with the product), you’ll see the data in more strategic ways.

The same applies to the sources of feedback. Is the feedback coming from social media or from your customer support system? As we discussed in lesson one, customers who share feedback with you privately have different motivations than those who share it publicly.

Borrowing From Useful Frameworks

David Cancel, CEO of Drift, developed a framework called The Spotlight Framework. David says, “People tend to focus on the wrong part of the feedback. Instead of focusing on the root cause or underlying issue behind the feedback, they focus on the subject of that feedback.”

This is a perfect example of why you shouldn’t look at what people say as the ultimate truth. As humans, we have different ways of communicating what we want and what we feel. And those nuances make a world of difference if we are willing to look deeper.

The Spotlight Framework suggests that we should separate feedback into three buckets based on the type of questions customers ask:

Bucket 1: User Experience Issues

How do I...?

What happens when...?

I tried to...

Bucket 2: Product Marketing Issues

Can you/I...?

How do you compare to...?

How are you different than...?

Why should I use you for/to...?

Bucket 3: Positioning Issues

I'm probably not your target customer...

I'm sure I'm wrong but I thought...

These classifications help you focus on what the customers are trying to achieve, rather than what they say they want to do. This framework works under the assumption that when a customer asks about ways of achieving something, they’re also telling you what is stopping them from doing it.

Example: A customer asks, “How do I export my data?”

There may be a couple of assumptions we can start building from this data point, including:

  • The user wants to export data but they can’t find where to do it in the app (UX issue)
  • The user can easily access the exporting feature but don’t understand how to use it ( UX issue)
  • The user is expecting to be able to export data but the product does not offer this functionality (potential feature)

Behind these assumptions lies a more important question: Why would users want to export data?

By unveiling the broader context behind their request, you’ll understand the jobs people are trying to get done with your product. Ultimately, this context will help you determine the best solution that can help your customer make progress.

Enter Jobs To Be Done (JTBD)

Jobs to be done (JTBD) is a different way of thinking about your product and what customers are trying to do with it. The main premise of this framework is to help you think about your product or service as a thing that gets hired to do a job. There’s a video by Clayton Christensen, a professor at Harvard Business School, that explains what JTBD is – often referred to as the Milkshake video. Check it out, it is only five minutes!

A lot has been written and researched in this area, so we have included a series of links in the useful resources section of this lesson.

In essence, JTBD focuses on the progress people want to make in their lives and the different jobs they need to do in order to achieve the process.

As Clayton Christensen explains,

“the circumstances are more important than customer characteristics, product attributes, new technologies, or trends… Jobs are never simply about function—they have powerful social and emotional dimensions.”

This approach is rooted in the idea that good innovations solve problems that formerly had inadequate solutions—or no solution at all—which represents a much bigger opportunity for businesses. That’s why it has become so popular.

Another premise is that using data, especially quantitative data to understand customers is never enough. More often than not, it can be detrimental. The challenge, instead, is to deeply understand the broader context of your customers’ problems.

Clayton explains that the fundamental problem with customer data is,

“most of the masses of customer data companies create is structured to show correlations: This customer looks like that one or 68% of customers say they prefer version A to version B. While it’s exciting to find patterns in the numbers, they don’t mean that one thing actually caused another. And though it’s no surprise that correlation isn’t causation, we suspect that most managers have grown comfortable basing decisions on correlations.”

As we mentioned at the beginning of this post, we humans are less rational than we think we are, and dealing with irrationality is not easy. This is why we find comfort in numbers. However, if you really want to understand what’s going on behind the numbers, you have to dig deeper. That takes time and is messy. The answers aren’t always as straightforward, but understanding who’s behind the numbers helps you build more empathy and can lead to making individual customers happy.

Jobs To Be Done Sounds Great, But How Do I Implement This?

Implementing jobs to be done has a lot to do with asking better questions and taking the time to study the context in which your product is being hired. The easiest first step is to learn as much as possible about what a good JTBD interview looks like.

Alan Klement wrote an incredibly helpful book called When Coffee and Kale Compete that we highly recommend. In the meantime, here are some of the questions Alan suggests to get you started.

Question for understanding the context around the point of purchase:

  • When did you purchase the product?
  • Where were you?
  • What time of day was it? (daytime/nighttime?)
  • What was the weather like?
  • Was anyone else with you at the time?How did you purchase the product?
  • Did you buy anything at the same time?

Understanding the initial triggers:

  • When did you first realize you [needed something to solve your problem]?
  • Where were you?
  • Were you with someone?
  • What were you doing or trying to do when this happened?

Understanding the emotional and social context:

  • Did you ask anyone else about what they thought about the purchase you were about to make?
  • What was the conversation like when you talked about purchasing the product with your <spouse/friend/parents>?
  • Before you purchased, did you imagine what using the product would be like?
  • Where were you when you were thinking this?
  • Did you have any anxiety about the purchase?
  • Did you hear something about the product that made you nervous?
  • What was it? Why did it make you nervous?

Remember, frameworks are just there to help us think about problems in different ways; they’re not guarantees of success. Sometimes the answers are straightforward. Adam Nash, CEO at Wealthfront explains:

“There is no mystery here. Listen to your customers, and know which features they want to see the most. You don’t necessarily want to implement every suggestion, but product professionals need to listen to direct requests carefully, with humility and deep consideration. Nothing irritates customers more than to see you roll out new features that exclude the ones that they have already identified and requested actively.”

Customer feedback is the start of the journey. Listen carefully to what customers say and always be prepared to dig deeper.