The Tesla vs Amazon stats that highlight all of NPS’s flaws
Does an analysis of the varying fortunes of these two mega companies in 2019 prove that ranking an entire organisation by a single number is misleading?
Let me paint a picture: It’s January 2019, and I’m in the wake of a discussion/debate with some Net Promoter Score (NPS) experts about the rationale for companies investing in NPS.
I’m undertaking a little in-depth research to try and validate my arguments when I come across this tidbit of information that throws everything straight into perspective:
Tesla = NPS score of 98.
At the time, Tesla had never made an annual profit (to that point – apparently has now, with some contentious accounting measures enacted). It was making cars with a vast number of quality control issues and had no centralised CX/CS process to hammer out those issues with its customers (unless you count tweeting Elon to ask him for help and/or praise him for his efforts).
Amazon = NPS score of -26.
Again, at that point in January 2019, Amazon was the first/second most valuable company in the world, depending on the day. It was generally loved by a vast majority of its customers. It had CX woven through and baked into the entire organisation.
Yes, I appreciate, there’s some bias in my explanations there. But the crucial aspect is the numbers – how can the world’s most (or very close to most, at the time) valuable company have a Net Promoter Score of -26, whilst one of the world’s most financial vacuous companies be soaring above the clouds with a score of +98?
Building the bellwether
First, a bit of backstory. The root cause of my thoughts about NPS most likely emanate from having spent 15+ years bailing out large global consulting companies from their own messes with their clients – and knowing that the “processes” that consulting companies devise all have inherent and critical flaws – NPS clearly being no different.
It was created at one of the larger consulting firms in the world – Bain Consulting – back in 2003, by an executive named Fred Reichheld, when he introduced it in a Harvard Business Review article. It was initially introduced as “Net Promoter Score” – it is now billed “Net Promoter System” as it has evolved to become, some might argue, more bloated and considerably more expensive.
Amusingly, if rumours are to be held true, Fred deeply regrets what NPS has now developed into – a costly, unnecessary bellwether for the current declining status of customer experience in many large corporations – at a time they need good CX the most.
One need only look at the on-going efforts of large companies who have “tied their wagon” to NPS being a saviour for their business – and it not working out that well (take a look at a certain two large businesses in the UK telecoms and banking sectors, for examples).
But still, the thought that “NPS is something we need for our business and our customers” persists and there are many, many more big businesses wedded to it.
And then we get to the Tesla vs Amazon revelation. How can these scores be representative of the businesses and their (at that time) vastly differing fortunes?
Flaws in the system
One of the main issues with most large, multi-national companies is their complete over-reliance on Net Promoter Score/System (NPS) as a primary measurement tool in determining the company’s perceived “success”.
A recent University of Cambridge research paper contends that “NPS measurement does not necessarily correspond to actual behaviour”. The researchers advocate for a multi-dimensional model, rooted in big data, that taps behavioural and attitudinal data.
The researchers built a model based on several points of data that represented customer activity (purchases, purchase size, feedback, etc.). This allowed them to compare NPS with customers’ future purchases.
What they found was that many promoters were not promoters (in terms of being the most loyal and best customers), and many detractors were not detractors.
With their model, they eliminated the need for the passives category and split customers into either detractors or passives based on their behaviour.
Another set of researchers, led by Timothy L. Keiningham in 2007, published a paper that argued other metrics were better measures of customer loyalty than NPS.
In their research, they found that NPS was no better than ACSI.
One of the main issues with most large, multi-national companies is their complete over-reliance on NPS.
Other research out of Stanford University also found that other indicators are equal or better measures of customer loyalty and referrals than NPS. The most effective indicator their research found was how much a customer “liked” a company (take that for what you will, given that it is a very wide set of interpretations from customers of what the term “like” might be).
NPS, in its best form, is not necessarily “bad” when seen in its best light, but it is clearly not good enough to even be considered the “best”, nor worth all the effort expended for it -- despite being so widely used. It clearly does not make up for the overhyped, and overly expensive implementation and upkeep costs for something that reveals so little in true value.
Keiningham’s paper goes on to state that managers have adopted NPS because they believe it is superior and that this superiority is based on “solid science”.
The paper notes: “[O]ur research suggests that such presumptions are erroneous.”
The researchers posit a related risk of “potential misallocation of resources as a function of erroneous strategies guided by Net Promoter on firm performance, company value, and shareholder wealth”.
Both Keiningham and other researchers like Morgan and Rego have attempted to replicate the initial claims that NPS outperforms other metrics for customer loyalty.
Improving the customer experience?
So, as we’ve seen in our Tesla vs Amazon example, the main issues remain:
Mobilising an entire organisation to a single number, such is the benefit of the Net Promoter Score/System, can be misleading and can lead to behaviours overly focused on the score and moreover and ironically, completely miss the intended spirit of NPS: Improving the customer experience.
Arguably, banks and telecommunications businesses probably suffer most from an over-reliance on the stated “NPS number” to help model the past attempts at developing a customer experience strategy; and resulting required processes to drive potential success in customer satisfaction, across the scope of their entire retail and business banking divisions.
It is a common issue for several global companies though – and it is one of the reasons that companies should be evolving away from using NPS as a primary indicator of the success of customer experience (as well as other factors as determining employee bonuses and other compensation formats – which has happened increasingly in recent years) and 0% detractors. Both result in an NPS of +40 which does not seem very logical.
NPS alone does not represent any reality of a business’ success at that given point.
The main thrust of the principal problem with NPS is that although it is a “simple number” that can be easily referenced to show a type of measurement within an organisation – it doesn’t show you how to fix the problems that are at the core of the problems, or even give you a rudimentary idea of when those base complications might arise. It therefore transforms itself into a vanity metric.
The overall NPS score is unlikely to drive any actionable insights, but individual scores, matched with the verbatim feedback from your customers (and other available customer data) is a key to making transformative decisions and progress within your organisation.
If we continue to look at the stated example given – the wildly varying scores of Tesla vs Amazon in January 2019, it is clear that NPS alone does not represent any reality of a business’ success at that given point.
Businesses must develop a set of relevant customer-focused KPIs that meet all their needs across the range of their customers and effectively measure how customers are interacting with the various parts of the institution and how feedback is handled and processed.
Each business should have a unique blend of KPIs that matter to both them and their customers. But that takes time, money, and the right set of people to figure out the best path. The thing is that if you invest wisely, the payoff is exponential to both the business and the development of the relationship with the customers. But instead, they continue to invest in lazy, useless things like NPS and wonder why it does not work.
The cost vs. actual value does not add up when the rubber hits the road. And as a final point, it’s worth asking this question:
When was the last time you heard someone say, "I want to buy/work/deal with the company that has the best NPS score..."
Dan brings a wealth of knowledge to CCO Global as the Chief Executive Officer with broad Customer Experience (CX), Employee Engagement, and Customer Strategy capability. With extensive experience in the fields of corporate strategy, operations and marketing, digital innovation (including mobile & social media) & Customer Relationship...