Customer satisfaction measurement: Can't get no satisfaction?by
Customer satisfaction measurement is the most commonly used non-financial performance metric, but it is also a popular target for criticism. So is it an inherently flawed metric - or are poor measurement processes to blame?
By Neil Davey, editor
Customer satisfaction is the most commonly used non-financial performance metric in the business world, and its measurement is a billion dollar industry in its own right. There's an obvious logic behind its measurement: keeping customers satisfied is a critical component for success in the marketplace and in order to manage customer satisfaction we must, of course, measure it.
However, whilst the rise in customer relationship management has fuelled the popularity of its measurement, and it has subsequently evolved to form the backbone of several economic indicators, so it has also had to contend with increasingly vocal criticism. Certainly in the past the customer satisfaction research industry has struggled to unequivocally demonstrate its real value to businesses. And as such, its growing popularity has proven to be a red rag to its detractors.
Stephen Hampshire, The Leadership Factor
Speaking recently, Professor Robert Shaw delivered a damning verdict on the majority of customer satisfaction measurement. "Questions like 'are you satisfied?' neither tell you anything particularly useful from a consumer point of view, nor from the point of view of loyalty or repeat purchase," he suggests. "On reflection, satisfaction is a silly question to ask about. Yet there is a huge industry churning out this data."
Elsewhere, Joe España, MD of Performance Equations, has also waxed lyrical about the perceived shortcomings of the measurement system. "Customer satisfaction surveys only tend to provide a superficial measure of the behaviours that drive profitability and growth," he says. "We know this because detailed research has shown that between 60-80% of those customers who judged themselves to be 'satisfied' or 'very satisfied' on satisfactions surveys were saying so just before they defected to the competition."
Unfair criticism? Not necessarily so, admits Stephen Hampshire, business development manager of The Leadership Factor. "There are limitations," he concedes. "People are very difficult to predict. We are complex beasts. We don't always tell the truth. And sometimes we aren't entirely consistent. But it would also be true to say that, on average, companies with customers that say they are very loyal in surveys have customers that stay around longer in practice. You can always find exceptions, but it does give you a guide to what your customers' attitudes are, which is the best way you have of predicting how they are going to behave – albeit that it is not 100%."
Critically, however, Hampshire also emphasises that customer satisfaction has certain inherent advantages over other metrics deployed in business. "The majority of metrics are backwards looking," he continues. "However, today's satisfaction is tomorrow's loyalty which is next week's profits. So it gives you forward looking capability not provided by any of the standard financial metrics and also an ability to be in control of your own destiny to some extent. There are case studies where firms have managed to build quite detailed causal models showing that if they improve customer satisfaction then loyalty goes up by X% and sales by X% and so on. And that gives those companies really solid information on which to base their decisions."
Professor Claes Fornell is arguably the world's leading authority on customer satisfaction measurement, having founded the American Customer Satisfaction Index and many similar indices in other countries. He believes that whilst there are some shortcomings, there are solutions available to resolve many of these issues.
"All surveys have limitations – respondents sometimes are not truthful and answer things that are certainly not right - but we now have methods to deal with that, besides running samples where we can sort out a good deal of error variance," he suggests. "However, few companies seem to be aware of the fact that we have made these tremendous advances in measurement technology in the past 15 years or so, even though these kinds of issues can be dealt with."
Louise Vacher, ORC International
Indeed, many of the criticisms levelled at customer satisfaction can be traced to common flaws in the survey process. Selecting only the most satisfied customers for customer surveys, over-sampling to compensate for small sample sizes and dressing up marketing surveys as satisfaction surveys are just some of the worst habits that have served to undermine the value of customer satisfaction surveys.
Louise Vacher, head of consumer research at ORC International, believes that businesses need to take a step back and look at how they are measuring customer satisfaction in order to ensure they are doing it correctly.
"A lot of companies are still doing their own internal satisfaction studies, and they may not be asking the right questions, they may not be asking them in the right way, they may not be getting to enough customers, they may not be getting to the right customers," she explains. "You have to start from the point of designing the best programme that you can. You are never going to get all of your customers to give you feedback but you need to make it as easy as possible to do so and to monitor what you are getting to make sure that it is representative of your business."
So what good practice do our experts recommend?
- Ask the right people
Businesses need to include unhappy customers as well as happy ones, former customers as well as present ones. Don't simply rely on the customers that are most convenient to reach for your sample. "It is absolutely critical to make sure you are measuring it in the right way and are getting to the right people," says Vacher. "For instance, hotels often have paper questionnaires in bedrooms, but only a very small percentage of guests ever fill them in – mostly only the ones that feel very strongly, either positively or negatively. So these hotels are probably missing out the vast mass of customers that are reasonably neutral but are obviously a very important group to understand." Fornell adds: "In many cases I need to know what segment of my market it is that I'm really concerned with. But as soon as I know that and I know roughly the characteristics of that segment (even if it is my total customer population) then I know how to draw a random sample or a probability sample and control the various statistical properties. That way I know whatever result I get is representative of at least my customers."
- Ask the right questions
"There are correct ways of asking questions, and there are incorrect ways," says Vacher. "You may not be measuring the right aspects of the business – you may be measuring a host of areas that just aren't relevant or interesting to customers where as there may be a few crucial things that absolutely drive a customer's likelihood to rebuy/revisit and you simply haven't included it in your questionnaire." Hampshire agrees that the survey should reflect the things that are important to customers and the way customers see the relationship rather than just being a list of things that the company wants to measure. "We use exploratory research – probably focus groups - to help customers help us design the questionnaire," he explains. "It is absolutely fundamental to getting the right questions on the questionnaire, making sure that it is serving the customers' needs, not ours."
- Getting the frequency of data collection right
"Most of the clients we work with are measuring it on an ongoing basis," says Vacher. "For example, we are increasingly using online as a way of getting customer responses because most people have email addresses and we tend to get a much higher response rate from that methodology which means you are including a far greater proportion of your customers and so you are more likely to be getting a representative group. Often they are sending out emails every single day so that someone fills out the survey a day after they have experienced the company in question, whilst the data is very fresh in their mind and they are more likely to give an accurate recall of the experience they have had." Fornell adds: "It depends on the sector you are operating in. If you are dealing in major appliances, where too many things don't happen and the purchase cycle is not very frequent, you don't have to measure that often. If you have a fast-moving sector where purchase frequency is high, you need to measure more frequently."
Doing it for the right reasons
Ultimately, however, the most important thing to bear in mind is that customer satisfaction research must provide information that is actually usable. Arguably the greatest fodder for the customer satisfaction critics has been the vast number of firms that do customer satisfaction surveys because they feel they ought to rather than because they're actually using the data.
Tom Peters, Management guru
"It is absolutely critical," agrees Vacher. "If you get quarterly reports, generally it is quite often too late to do anything about any issues that are identified. We tend to use results websites that are updated every single day. Head office may look at it on a more strategic level, perhaps on a quarterly basis, but we would always say that it is critical to get feedback to those who actually work with it as quickly as possible, so we see clients logging in on a very regular basis."
Other firms use the data to draw up a satisfaction index which tells them how good they are in the present, and a list of priorities for improvement (PFIs) which are targets looking ahead.
"There are strategic levels and operational levels," Fornell emphasises. "The strategic level is necessary to be able to correlate financial results. The operational level is necessary to be able to take action. So you need both of those levels in your measurement system. And certainly the operational level has to be done more frequently so that you can follow it and know what you're doing."
Despite its vociferous critics, Fornell believes that customer satisfaction's value as an economic indicator will win out. "Balance sheets for companies essentially have information about supply factors… productivity-related things… what the company has in terms of assets that would help it in the market," he explains. "But these supply factors will no longer be very predictive of what is going to happen in the future in terms of the financial returns of a company. For the past 10-15 years we have seen the correlation between balance sheet and assets and future income dwindle towards zero. We can't look at the balance sheet and say very much about the future of a company anymore. Clearly we need other important types of information. And I look at the strength of a customer relationship as a true economic asset. It should be on the balance sheet of course, but it isn't! So companies need information about this, not only for predictive purposes but to be able to do something about it to improve their value further."
And he is in good company. Management guru Tom Peters has said: "In the customer arena, we believe that regular, quantitative measurement of customer satisfaction provides a much better lead indicator of future organisational health than profitability of market share change."
Nevertheless, a stricter measurement process could serve to silence some critics – and, most importantly, certainly provide more valuable actionable information for the firms involved.
"I am still astonished by the low quality of measurement that many companies still use even though there are many better methods available," concludes Fornell. "It is incredible to me that many firms are very concerned about the quality of their product and the quality of their service but then they mess up completely when it comes to the quality of their measurement."