Share this content
Customer survey question

How your customer surveys could be harming your company

1st Mar 2018
Share this content

New research suggests that the quest for customer survey metrics could actually be causing more harm than good for the customer experience

One thing we CX-ers have in common: we love our metrics. Go to any CX conference, and the room that’s filled to overflowing is probably talking about metrics.

Metrics are comfortable for us. Whether we’re talking Net Promoter Score, Customer Effort Score, or good old customer satisfaction, survey metrics give us something to share with the business. Even better, in a role that is so focused on intangibles, we have one tangible thing we can point to.

Of course, to get these metrics we need surveys. Lots of surveys. Long relationship surveys, short (but frequent) transactional surveys, and medium-length touchpoint surveys. More data to analyse and report. We need to feed the beast.

I recently ran across some research that suggests that the continuing search for these metrics may actually be hurting your company.

In his article, Rethinking Market Research’s Place in the BusinessVikas Mittal, Ph.D. (a member of the faculty at the Jones Graduate School of Business at Rice University in Houston) shares some research results from “A 2017 study of 7,513 customers of a large North American automotive dealership.” He doesn’t cite the company. (Ironically, when reading the article online, the AMA website asked me to rate how likely I am to recommend this article!)

Are you over-surveying your customers?

His first point is that those who fill out a survey spend more than those who do not; he reports that, “sales per visit were $12.18 greater among those who filled out the satisfaction survey [versus those who did not].” He then argues that this shows a business benefit to surveys.

I’m not convinced. Mittal makes a causal assumption that I don’t share – that the survey caused the increase in spending. It seems much more plausible that the type of person who responds to surveys is different than the type who doesn’t. The person who responds to surveys is likely more engaged with the brand, and thus spends more (although I do acknowledge that there is academic research backing up the causal link between surveys and increased purchasing behaviour).

Regardless, his second point is much harder to dismiss: customers’ purchase amounts decreased the more surveys they responded to. 

This is a big deal. The continued linear nature does suggest causality, which also reflects what I have seen. The more you annoy a customer with repeated surveys, the less engaged they become with your brand, translating into reduced purchases.

Yes, by over-surveying customers, you may be costing your company revenue.

So, what’s a CX-er to do? I have three steps as a recommendation:

  1. Reduce the chance of survey overload. Ensure that no customer gets more than one survey per quarter; preferably no more than one every six months. If you don’t have the means to ensure this, then shut the survey down. We’re customer experience people. Our top priority should be engaging customers. Any disengaging activities need to stop.
  2. If necessary, reduce the frequency of your reporting. It may be that reducing the frequency of your surveys means you don’t have the sample size to report as frequently. So be it.
  3. Fill this void by moving your reporting away from survey metrics to business KPIs. This is what your business partners are more interested in, anyway. Tie your survey results to these business KPIs, showing the linkage between your metrics and the business results.

Now that you’re likely reporting less often, that will give you time to build this linkage. Show how improving your scores lead to real results, and include metrics such as order size, frequency of order, and customers lost and gained in your reporting.

And if you can’t link your surveys with this business data? Then reporting on your survey scores probably wasn’t that important, after all.

Replies (5)

Please login or register to join the discussion.

Lindsay Willott
By Lindsay Willott
05th Mar 2018 10:12

Great article Jim, thanks for sharing this research. One of the keys to solving this dilemma for companies desperate for customer feedback is attempting to get it in an appropriate way. Surveys have run amok in many companies, and the number of questions, and scale of the survey is often highly disproportionate to the interaction or product in question. Here's a recent example I experienced myself:

I don't think customers have a fundamental problem giving feedback to the companies they but from. But I do think the feedback process needs to be treated with a great deal more sensitivity and intelligence in many cases. Just because you can send out surveys at every opportunity doesn't mean it's okay.

Thanks (0)
By EJohn Morris
08th Mar 2018 12:41

Hi Jim, nice article which I believe captures one of the big challenges businesses face. There are a couple of points I would like add. Company's rush to survey customers but fail very often to survey those who serve them, the employees who often have strong views of customer feeling. The second point is that companies also fail to let their previous surveyors know what the findings where from previous surveys and what has been done as a result.

Thanks (0)
By PaulineAshenden
09th Mar 2018 17:02

The other point to note Jim, is whether these customer surveys are giving you the full picture of what customers are actually saying. They only capture part of the story - as we outline in this blog you need to expand your listening if you want to truly hear the Voice of the Customer.

Thanks (0)
Jeff Toister
By Jeff Toister
18th Mar 2018 17:04

Fascinating research! Can you share a link to the original study? I'm unable to find it online.

Thanks (0)
Replying to Jeff Toister:
By Neil Davey
19th Mar 2018 12:02

Hi Jeff, I believe this is the original article:

I've added the link to the above piece now as well.


Thanks (0)