To be honest, the question for me is why do people continue to believe in magic pills?
In my opinion, the experience the customer has/ or could have (how customers think-feel and what they do) is not something that can fit within a linear metric framework that assumes governing constraints, that is supported because its simple to understand (that's office politics not proper research).
If you want to know how to measure the experience the customer I would argue: tell me what the customer experience is, and I'll tell you what to measure.
Or if you must have something simple, use satisfaction against CTQ (critical to quality measures) of relevance to your business.
IMHO you can learn a lot from Agile frameworks such as Cynefin and narrative metrics rooted in consumer psychology such as "more stories like this, fewer stories like that". Thank you.
Thanks for the article.
The challenge I would pose is the use of the term 'drivers'; does this not require justification? For instance, why should experiences assume a causative path rather than being mere correlation; and what is the interrelationship between the experiences measured i.e., can we assume they are so discrete and linear in path?
I would be interested in how you would respond to that.
My personal experience of working with statistics, is that most CX models that engage multiple variables on likert scales fall over, are multicollinear, and fail to consider consumer psychology well. The last being very important.
Don't get me wrong I am not saying you fall into this trap, what I am saying is I would like to understand how you are avoiding them? Especially when NPS itself is riven with gifting and gaming (it's a cultural metric not a science) - not your fault I know.
My personal opinion is that you should start by understanding the experiences in question before getting to test the stats - perhaps you do that?
As an example for me many experiences fall into the complex domain (see Cynefin) and it is partially false to assume a direct relationship between likert scales and consequent consumer behaviour - we are not mathematical creatures after all. IMHO.
The problem is political and cultural. To land an experiential metric, we work with a C-level that has trouble thinking beyond an engineering paradigm of linear metrics. This is unfortunate since, the experience the customer has is broader than any one metric can ever define. So while NPS is a good starting point, value is left on the table and it is not an end state. For me, the best that can be done is to show a maturity curve where businesses start with NPS, then move up the maturity level to engage with other forms of data that uncovers customer value. My personal advice would be to enrich NPS with 'deep data' and experience designs and seek to include more open ended metrics such as the sensemaker metric 'more stories like this. fewer stories like that' focused more on setting parameters and volume improvements i.e., we reduced negative stories by 10% or increased positive ones by 5%. IMHO.
I have no doubt that such frameworks have value, however I take issue with a few things.
Firstly, you measure CX based on sentiment. Presumably from a machine that assumes the customer evaluates everything in a positive -negative way based on text. While that has value it is only a partial truth. My point: such an evaluative view over indexes on seamlessness and underindexes on value (I will tell you about the pain, less so where value might lie from an improved UX design or UX writing).
Secondly, your sentiment line seems very unusual.
There is no baseline to the norm (i.e., I would expect use to flatline to a norm on the simple basis that 'emotions (and likewise sentiments) are for learning' and once learnt we fall into a pattern of regularity. Hence use, would and should be lower in terms of 'sentiment' spikes. This is homeostasis.
My point?
The real issue is understanding the customer is not in the centre with any of these frameworks. I have found this many time from tech people who think tech first and never the psychology of the customer. When CX is about the psychology of the customer.
A better framework is simply to embed Qualitative research more fundamentally into design.
The lack of VoC within design is the problem.
The lack of feedback from the customer on the success of the design is the problem
The fact that vendor SLAs do not include customer feedback, the final part of the problem.
NPS is an off-the shelf measure rooted in statistics, so show me the evidence.
However, be clear, this is a minefield, stats is as much an art as a science and I find it a shame that commentators do not give us the hard statistical facts.
For instance, Tim Keiningham's replicate work showed it was no different than CSAT in terms of predictive power (AVE).
I concur with this having reviewed 10s of thousands of data points put through structural equation modelling and seeing low scores on standardised AVE against spend and other hard variables.
(NPS for me shows all the characteristics of poor predictability you would expect from an off-the-shelf aggregate measure).
Likewise it is very highly correlated to CSAT.
Finally, it confuses correlation with causation.
In terms of its ontology, it is also flawed. To reference Dave Snowden and Cynefin, NPS only works where prior root cause is apparent - yet not all CX is predictable and mechanistic like this. The past does not necessarily predict the future.
Does that make it a bad measure - actually No.
It has cultural cache and an interesting dynamic, it encourages action where no prior relationship exists. However, we should be careful to treat this as anything more than a starting point. And avoid de-emphasising the value of qualitative research and ideation.
There is also one other angle, what people say and what they do (I recommend but have never actually done so) and less considered the difference between what people might say when they recommend and what they say in a survey about why they give a score (there is only a 50% relationship, so I might give a score of 0 out of 10 but with a friend I would say something positive).. this was shown in Ericsson research with Bharti Airtel I have previously highlighted.
In short, its a culture metric. And while there are aspects of root cause in data (especially transactional and negative scores) that do work well and demonstrate ROI, companies in their use of blunt instruments to help scale should not treat it as the one number to grow.
Personally I have no great issue with NPS, but would always apply a mixed approach i.e., verbatims, qualitative research. And always understand NPS as being grounded in an engineering approach to human psychology i.e., its only partially correct and no better than CSAT or CES.
At the end of the day human data is not mechanical data and should not be treated as such. Which is why, in my opinion, applying approaches such as Cynefin and SenseMaker, rooted in anthropology and psychology are the better starting point than ones rooted in engineering and computer science.
Which comes to my real issue. NPS etc.. just reflects the bias of the C-level executive used to engineering and financial concepts. Until we change this understanding we are doomed to repeat the same mistakes.
There is big data, deep data and rich data. Covering all three is important in CX. The term Unified only refers to skating over the surface of big data.
Dean - I worked with Strativity and the Zen team on a starting piece of CX work. Good to see you are keeping to the great customer service theme and not being diverted to price competition.
To be honest, I think the problem lies with IT controlling CX budgets. A colleague of mine once explained that people took jobs in IT to get away from people. So don't be surprised with a mindset that focuses on scale without empathy; root-cause thinking without consideration.
Rip budgets for CX away from IT and you might have more success. IMHO.
My answers
To be honest, the question for me is why do people continue to believe in magic pills?
In my opinion, the experience the customer has/ or could have (how customers think-feel and what they do) is not something that can fit within a linear metric framework that assumes governing constraints, that is supported because its simple to understand (that's office politics not proper research).
If you want to know how to measure the experience the customer I would argue: tell me what the customer experience is, and I'll tell you what to measure.
Or if you must have something simple, use satisfaction against CTQ (critical to quality measures) of relevance to your business.
IMHO you can learn a lot from Agile frameworks such as Cynefin and narrative metrics rooted in consumer psychology such as "more stories like this, fewer stories like that". Thank you.
Rolls Royce Power by the Hour springs to mind.
Thanks for the article.
The challenge I would pose is the use of the term 'drivers'; does this not require justification? For instance, why should experiences assume a causative path rather than being mere correlation; and what is the interrelationship between the experiences measured i.e., can we assume they are so discrete and linear in path?
I would be interested in how you would respond to that.
My personal experience of working with statistics, is that most CX models that engage multiple variables on likert scales fall over, are multicollinear, and fail to consider consumer psychology well. The last being very important.
Don't get me wrong I am not saying you fall into this trap, what I am saying is I would like to understand how you are avoiding them? Especially when NPS itself is riven with gifting and gaming (it's a cultural metric not a science) - not your fault I know.
My personal opinion is that you should start by understanding the experiences in question before getting to test the stats - perhaps you do that?
As an example for me many experiences fall into the complex domain (see Cynefin) and it is partially false to assume a direct relationship between likert scales and consequent consumer behaviour - we are not mathematical creatures after all. IMHO.
The problem is political and cultural. To land an experiential metric, we work with a C-level that has trouble thinking beyond an engineering paradigm of linear metrics. This is unfortunate since, the experience the customer has is broader than any one metric can ever define. So while NPS is a good starting point, value is left on the table and it is not an end state. For me, the best that can be done is to show a maturity curve where businesses start with NPS, then move up the maturity level to engage with other forms of data that uncovers customer value. My personal advice would be to enrich NPS with 'deep data' and experience designs and seek to include more open ended metrics such as the sensemaker metric 'more stories like this. fewer stories like that' focused more on setting parameters and volume improvements i.e., we reduced negative stories by 10% or increased positive ones by 5%. IMHO.
I have no doubt that such frameworks have value, however I take issue with a few things.
Firstly, you measure CX based on sentiment. Presumably from a machine that assumes the customer evaluates everything in a positive -negative way based on text. While that has value it is only a partial truth. My point: such an evaluative view over indexes on seamlessness and underindexes on value (I will tell you about the pain, less so where value might lie from an improved UX design or UX writing).
Secondly, your sentiment line seems very unusual.
There is no baseline to the norm (i.e., I would expect use to flatline to a norm on the simple basis that 'emotions (and likewise sentiments) are for learning' and once learnt we fall into a pattern of regularity. Hence use, would and should be lower in terms of 'sentiment' spikes. This is homeostasis.
My point?
The real issue is understanding the customer is not in the centre with any of these frameworks. I have found this many time from tech people who think tech first and never the psychology of the customer. When CX is about the psychology of the customer.
A better framework is simply to embed Qualitative research more fundamentally into design.
The lack of VoC within design is the problem.
The lack of feedback from the customer on the success of the design is the problem
The fact that vendor SLAs do not include customer feedback, the final part of the problem.
NPS is an off-the shelf measure rooted in statistics, so show me the evidence.
However, be clear, this is a minefield, stats is as much an art as a science and I find it a shame that commentators do not give us the hard statistical facts.
For instance, Tim Keiningham's replicate work showed it was no different than CSAT in terms of predictive power (AVE).
I concur with this having reviewed 10s of thousands of data points put through structural equation modelling and seeing low scores on standardised AVE against spend and other hard variables.
(NPS for me shows all the characteristics of poor predictability you would expect from an off-the-shelf aggregate measure).
Likewise it is very highly correlated to CSAT.
Finally, it confuses correlation with causation.
In terms of its ontology, it is also flawed. To reference Dave Snowden and Cynefin, NPS only works where prior root cause is apparent - yet not all CX is predictable and mechanistic like this. The past does not necessarily predict the future.
Does that make it a bad measure - actually No.
It has cultural cache and an interesting dynamic, it encourages action where no prior relationship exists. However, we should be careful to treat this as anything more than a starting point. And avoid de-emphasising the value of qualitative research and ideation.
There is also one other angle, what people say and what they do (I recommend but have never actually done so) and less considered the difference between what people might say when they recommend and what they say in a survey about why they give a score (there is only a 50% relationship, so I might give a score of 0 out of 10 but with a friend I would say something positive).. this was shown in Ericsson research with Bharti Airtel I have previously highlighted.
In short, its a culture metric. And while there are aspects of root cause in data (especially transactional and negative scores) that do work well and demonstrate ROI, companies in their use of blunt instruments to help scale should not treat it as the one number to grow.
Personally I have no great issue with NPS, but would always apply a mixed approach i.e., verbatims, qualitative research. And always understand NPS as being grounded in an engineering approach to human psychology i.e., its only partially correct and no better than CSAT or CES.
At the end of the day human data is not mechanical data and should not be treated as such. Which is why, in my opinion, applying approaches such as Cynefin and SenseMaker, rooted in anthropology and psychology are the better starting point than ones rooted in engineering and computer science.
Which comes to my real issue. NPS etc.. just reflects the bias of the C-level executive used to engineering and financial concepts. Until we change this understanding we are doomed to repeat the same mistakes.
There is big data, deep data and rich data. Covering all three is important in CX. The term Unified only refers to skating over the surface of big data.
Dean - I worked with Strativity and the Zen team on a starting piece of CX work. Good to see you are keeping to the great customer service theme and not being diverted to price competition.
To be honest, I think the problem lies with IT controlling CX budgets. A colleague of mine once explained that people took jobs in IT to get away from people. So don't be surprised with a mindset that focuses on scale without empathy; root-cause thinking without consideration.
Rip budgets for CX away from IT and you might have more success. IMHO.