Share this content
Painting
istock

CX driver analysis: How to turn statistics into a visual CX roadmap

by

Key driver analysis research is not new. But typical driver analysis reporting approaches fail to bring the data to life, making it difficult to tell an actionable story. However, there is an underused approach that not only visualises the statistics and tells a compelling story, but also better surfaces actionable insights. 

15th Aug 2022
Share this content

Key driver analysis research is not new. The goal is to identify the different customer touchpoint experiences, which are drivers of an organisation's overall performance measure (e.g. NPS, overall satisfaction, customer effort/ease, etc.). Essentially, a driver analysis computes importance scores, which then prioritise the touchpoint experiences that have the greatest impact on performance measures.

There are various multivariate approaches to determine key drivers; these methods include regression, canonical correlation, factor analysis, and correspondence analysis, among others. Regardless of the approach, driver analyses are typically reported as quadrant maps or as a data table. While somewhat easy to read, many find them perplexing to understand and interpret.

Typical driver analysis reporting approaches fail to bring the data to life, making it difficult to tell an actionable story. Historically, driver analysis only provides a basic and very limited understanding of those elements that are most important in influencing performance measures. What they fail to provide is the insight that makes the analysis actionable and impactful.

[Click to enlarge]

cx driver analysis

 

A quick interpretation of the above slide:

For the first grey box under the NPS is "Customers Service is Available When Needed," this touchpoint experience descriptor was identified as having the single greatest impact if ALL customers believe the statement is true (rated Top 3 Box on an 11-point scale). If ALL customers believed the descriptor was true, the percentage of Promoters has the potential to increase to 74%, as indicated in the green box. 

Conversely, the red box for the same descriptor indicates the potential maximum decrease if ALL customers believed the statement is not true (not rated Top 3 Box). The percent of Promoters would decrease from 67% to 44%.

Following the path to the right, you see the 4 statements that have the greatest potential to increase NPS Promoters from 67% to 92%. 

By following the red box on the first descriptor, "Customers Service is Available When Needed," you see the path to follow if you cannot deliver on "Customers Service is Available When Needed."

[Click to enlarge]

experience impact analysis

 

Forgoing the actual statistics for now (that will come later), we are able to calculate the impact each key driver has on a performance measure. By providing the actual impact on performance measures, the organisation can align on a common goal and give CX leaders the insights required for increased resourcing and funding.

With the prioritised drivers and their impact on performance measures, the approach then allows for the creation of a graphical roadmap that is easily interpreted and communicates the required action that must be taken by management. We call this graphical roadmap the Experience Impact Analysis. Several CX leaders have referred to it as the "CX money slide" because of its ability to align the organisation and drive action. 

With only a quick explanation, we often see presentation participants' eyes widen as they understand within seconds exactly how to impact and improve the performance measure. In the example above, the purple box is the performance measure; in this case, NPS Promoters, and the percentage is 67%.

The grey boxes underneath being displayed as a hierarchical tree are the key drivers. Each driver has a green or red box associated with it. The percentages shown in the green boxes are the maximum potential increase in the performance measure if all customers believe the company delivers on the driver. The percentages in the red boxes are the maximum decrease in the performance measure if all customer believes the company does not deliver on the driver.

The statistical approach

Experience Impact Analysis is similar to regression analysis. However, it takes key driver studies beyond a relative importance score. It provides not only the drivers but also the numerical positive and negative impact on performance measures.

Both regression and Experience Impact Analysis are statistical methods that use data to construct models to predict the value for one variable (the performance measure in Experience Impact Analysis) based on the values reported for a set of other variables (experience variables in Experience Impact Analysis).

The main differences between the Experience Impact Analysis and regression are:

  • EIA reports the value of the key performance measure for each sample split to identify both the touchpoint experience causing the change as well as numerically reporting the impact of achieving a required score for that experience.
  • Regression analysis provides a relationship model that indicates how each statistically identified experience measure impacts the performance measure. The approach does NOT tell you which combination of measures yields the highest performance score. This identification of the maximum possible performance measure, and which combination of experience measures are required to yield that score, is the key benefit of Experience Impact Analysis over regression.

Regression models provide the identification of statistically significant variables and a numerical score for each of those variables. The end result is to identify the variables that have the greatest impact on the performance measure, not necessarily the combination of variables that generate the HIGHEST performance measure.

EIA works differently in that an iterative approach is used to identify the combination of variables that, when considered together, generate the highest possible score for the performance measure. At the first iteration (Step 0), EIA computes either an F, T, or Chi-square value for each of the experience variables versus the performance measure. All experience variables with a statistically significant measure of 95% or higher are selected. The one which generates the highest performance measure value is selected and becomes the first node on the tree, thus generating two branches. In iteration 2, all remaining experience variables are again considered for EACH branch. The selection process is performed by looking at the statistical significance and maximisation of the performance measure. This process again   generates two additional branches for each of the Iteration 2 branches. The iteration process continues until there are no longer any significant experience variables remaining, or if the sample size becomes too small for any branch to become meaningful and actionable.

The EIA approach has proven itself over the years. The technique was conceived in the early 1950s and became more mainstream in the 70s and 80s. It has been used in a variety of applications outside of marketing research. Today, it is often used as a replacement for Regression techniques and Multidimensional Scaling approaches because of its pictorial outputs and easily understood metric reporting.

In conclusion, selecting the right multivariate approach is foundational to having valid insights. The Experience Impact Analysis approach, while under-used, provides actionable insights that does not require statistical expertise to interpret and implement. 

The Experience Impact Analysis answers the question: what will the empirical impact be to the performance measures?

The benefits of Experience Impact Analysis:

  • Visualises the statistics.
  • Tells a compelling story.
  • Identify the combination of touchpoint experiences that produce the HIGHEST performance measure.
  • Aligns organisation to a common goal.
  • Builds the business case for resourcing and funding.

About the authors:

Ed Murphy, President, Imprint CX
Co-founder and President of Imprint CX, a modern marketing and customer experience services company. Ed has achieved a very successful career by building relationships and providing innovative solutions to meet clients’ needs. With over thirty years as a global researcher, management consultant, and business leader, Ed brings his experience, expertise, and passion to every consulting assignment. To learn more about ImprintCX contact Ed at [email protected]

Mike DeVita, Statistical Guru and Founder of EDP Services
Mike has worked on an academic team at the University of Pennsylvania that did early research on the development and application of many of the advanced multivariate techniques in general use today. Mike has been involved in every facet of complex marketing research projects, both qualitative and quantitative. Included in these offerings are conjoint measurement, perceptual mapping, and cluster analysis. He played the role of not only using, but writing much of this software and therefore has the unique perspective of both a developer and practitioner of these techniques.

Rick Heller, Market Research and Strategy Executive and Consultant
Rick has successfully been an independent Insights and Strategic Planning consultant since 2006. Prior to being a consultant, he had been the Market Research Director for Volkswagen of America and Sr. VP Research Director at Doner Advertising. Rick uses consumer insights and measurements to derive a clear, results-oriented strategic platform that is easy to understand and highly actionable. He strives to create a synergy between business objectives, insights, and analytic tools to develop fully integrated marketing solutions.

 

 

 

Related content

Replies (3)

Please login or register to join the discussion.

Steven Walden
By Steven Walden
19th Aug 2022 08:39

Thanks for the article.
The challenge I would pose is the use of the term 'drivers'; does this not require justification? For instance, why should experiences assume a causative path rather than being mere correlation; and what is the interrelationship between the experiences measured i.e., can we assume they are so discrete and linear in path?
I would be interested in how you would respond to that.
My personal experience of working with statistics, is that most CX models that engage multiple variables on likert scales fall over, are multicollinear, and fail to consider consumer psychology well. The last being very important.
Don't get me wrong I am not saying you fall into this trap, what I am saying is I would like to understand how you are avoiding them? Especially when NPS itself is riven with gifting and gaming (it's a cultural metric not a science) - not your fault I know.
My personal opinion is that you should start by understanding the experiences in question before getting to test the stats - perhaps you do that?
As an example for me many experiences fall into the complex domain (see Cynefin) and it is partially false to assume a direct relationship between likert scales and consequent consumer behaviour - we are not mathematical creatures after all. IMHO.

Thanks (0)
Replying to Steven Walden:
avatar
By EdMurphy
06th Sep 2022 16:52

One of the first things learned in statistics is that “correlation is not necessarily causation.” Understanding the relationship between variables requires both a knowledge of the topic being studied as well as experience in using statistical techniques that measure these relationships.

Proving a true causal, and not just correlated, relationship between a series of variables is exceptionally difficult. However, it is generally accepted that when you are using an end measure such as Purchase Interest, Satisfaction, NPS, Customer Ease, etc. and using a number of alternative measures that reflect attitudes and perceptions that might lead one to increase or decrease the scoring of these performance measures, the measures that are statistically related are most often referred to as drivers. The simple reasoning is that if I rated a service highly on treating me well and I had a high NPS or Satisfaction score, it is unlikely that the NPS/Satisfaction attitude happened before being treated well. Therefore, if I can show a statistical relationship between being treated well and NPS/Satisfaction. Therefore, being treated well is a driver of the more broad, end result measure. Whether or not the word ‘driver’ is justified is another discussion, but currently widely accepted and used.

The multivariate statistical technique most often used to measure a driver scoring system takes into account both correlation with the performance variable and multicollinearity among the predictor variable set. If the attributes/experiences chosen to be measured are based on other research and insight, and that information is properly collected and vetted, I would argue that it does not fail to consider consumer psychology as the research will include those factors that the consumer uses to make decisions, evaluations and general mindset.

The process that we use does not start with writing a survey and then statistically measuring the responses. We spend as much time and effort in understanding the experiences in question, consumer and employee attitudes, wants and desires so that the survey tool has been maximized to measure the most important issues facing both the consumer and the service/product provider.

While we are not mathematical or statistical creatures, we do measure those factors that we have determine to be most important and imperative to all parties. We then use statistics to help understand and highlight the relationships. The end result of our program is to highlight and provide insight into complex issues. It is also not meant to be the one and only source of information, but rather support and enhance learning from multiple sources.

Thanks (0)
avatar
By dailyak
28th Aug 2022 09:10

I have an NPS survey results from employees with up to 30+ variables. I have done a bit of research online on the methods to get meaningful coefficients from the survey data. I was wondering what were the best methods on approaching the issues I may face when handling a large number of variables especially with factors like:

Thanks (0)