Under-funded, under-resourced, over-worked - so why has the past year been a big win for customer insight?by
Market research and customer insight teams have been at the very heart of the pandemic over the past year, reporting on the impact of COVID-19 on consumers. It has been a dramatic struggle. But it could also have been a pivotal moment for the insight function.
It’s a cruel twist of fate that at a time when organisations had a more urgent requirement for up-to-the-minute customer insight arguably than ever before, insight teams and agencies were in a state of flux. But then the pandemic has left no organisation, and no business function, free from disruption.
But while the customer insight function certainly hasn’t emerged unscathed by COVID-19, evidence suggests that the discipline has evolved to become more agile and innovative to deal with the pressures and demands it has found itself under.
And there is optimism that the customer insight function will actually emerge stronger in several ways.
Customer insight ‘reset’ by COVID-19?
Market Research Society thinktank The MRS Delphi Group conducted a study at the end of last year to examine the issues that COVID-19 has created for the research sector, talking to a wide range of experts from companies including eBay, BT and Vodafone. It concluded that the impact of the pandemic was so profound that not only were current plans no longer relevant, but that a fundamental reappraisal of the function was required. The pandemic had, it suggested, ‘reset’ the customer insight function.
While this may seem hyperbolic, it is worth emphasising that market research and customer insights have been at the very heart of the pandemic over the past year, reporting on the impact of COVID-19 on consumers. The soaring demand for insights into changing consumer behaviours and preferences has forced the field to become more agile and innovative to cater for new requirements at a time when its own standard methods have been forced to fundamentally change. And this adaptation has been crucial during a time when customer insights are more valuable than ever.
“Customer behaviour has changed and the experiences organisations offer customers need to evolve, or they will get left behind,” says Marius Bartsch, head of customer engagement at Digitas UK.
“We have seen the adoption of some channels skyrocketing (e.g. live chat, selling and servicing by video-call, social media), and brands are interested in how customers are using those channels. The customer insight function is key to understanding those trends, and we are seeing their voice being listened to more and more by brands trying to find their way (and determine which channels to invest in) in the face of change.”
David Lloyd, Wunderman Thompson's head of data & insights believes that the ‘ground-zero empiricism’ effect of the pandemic was huge. “The comparatively slow-moving consumer trends we’d been analysing not only turned on their heads overnight, they shifted erratically on a continual basis as the public first came to terms with the pandemic and then lived through the ups and downs,” he says.
“Interestingly, demand for insights increased from both COVID winners and losers. Businesses that suffered most (e.g. fashion, out-of-home entertainment) sought consumer understanding to inform rapid re-thinking of their business models to avert crisis and steal a march on similarly challenged competitors. Some businesses benefited (e.g. tech, digital retailers) and these sought to invest further in their customer insight capabilities, also leveraging the surge in their 1st party data as consumers flocked online. The speed of demand mirrored both the urgency of businesses to pivot, but also a thirst for high frequency low latency insights to stay on the pulse of the ‘corona-coaster’ effect.”
The need for speed
Anda Ziemele, data scientist and data storyteller at Capgemini, believes that the combination of the volatility created by COVID-19 and government policy and decisions impacting consumer behaviour has made it very difficult to forecast trends and patterns of behavivour - which has led to a requirement for more rapid-fire shorter-term insights.
"During the pandemic, looking at behaviours on a more typical seasonal or quarterly timeline would not have provided the understanding required," she emphasises. "Insights were needed constantly to understand the public’s reaction to different levels of lockdown and different stages of the pandemic.”
Unsurprisingly then, when The MRS Delphi Group study surveyed the customer insight profession, one of the key trends that emerged was the need for speed.
There is a lot less time to prepare and the insights need to surface really quickly because the environment is changing so rapidly. Data collected last month may already no longer be valid!
“Companies have ripped up annual (or even quarterly!) plans and are planning from month to month, making the speed at which insights are delivered really critical,” says Dr Christine Bailey, author of Customer Insight Strategies, CMO, speaker, & founder of Sophia Marketing. “There is a lot less time to prepare and the insights need to surface really quickly because the environment is changing so rapidly. Data collected last month may already no longer be valid!”
Customer insight speaker, mentor & trainer Paul Laughlin adds: “Although the short-term demand was focused on pandemic-related questions, since then the demand is being driven by a desire to be more insight/data led in understanding future buying patterns and the customer relationships needed for the future. All this has driven a demand to deliver more insight more quickly, more frequently and a number of leaders cite a shift in balance of proactive versus reactive. With so much changing in their businesses, they have needed to be more proactive to influence business decisions on changes that matter to customers.”
At the same time, Capgemini notes that the volume of data available to insight teams has also dramatically increased.
Jo Peplow, Capgemini’s AI & analytics team lead, notes: “One of the major by-products of both business and consumers having to move to digital services is the creation of customer data. Businesses that previously had very little customer/consumer data points will now be able to generate a much deeper understanding of their customers which they can use to generate better insights into behaviours and preferences. Businesses have had to, often rapidly, innovate digitally-led experiences to keep consumers engaged with their brands through this time. For example, restaurants creating online menus with increased integration for Deliveroo and Uber Eats.
“However, it remains to be seen how ready businesses are to implement and action these insights. Those unfamiliar with digital consumer insight platforms have had to rapidly up-skill to understand their customers and how their needs have evolved over what has become a significant time.”
The challenge of skills and resources
With the volume of data growing rapidly, the need for new skills and resources has become increasingly urgent.
“The volume of data has grown immensely as we’re ingesting in and crunching potentially all conversations mentioning or relating to COVID,” explains Matt Tubb, senior solutions architect at Capgemini.
“More people talk or present their views on the internet now through forums, social media, news comments, blogs, reviews on ecommerce sites and review platforms. With the continual adoption and use of these services, more data is generated leading to more resources being consumed and more skills required from analysts to interrogate data at scale and to provide actionable insight.”
More data is generated leading to more resources being consumed and more skills required from analysts to interrogate data at scale and to provide actionable insight.
In normal circumstances, urgent requirements for improvements in speed, scale and skills would be addressed through budget increases. But this is a time like no other. And the likes of Capgemini and the MRS Delphi Group have reported that in many cases budgets last year were frozen or cut.
Tubb continues: “At the high of the pandemic in Q2 2020, we were seeing clients’ slashing what were deemed unessential budgets - for example marketing budgets - to try and manage the impact of the pandemic on the business as a whole. For businesses whose insights budgets sat in these areas, this led to a huge increase in demand for insight and answers being dealt with by a team running at max capacity.”
Fortunately, there are signs that budgets may be increasing once more - Capgemini notes that there has been a surge in recruitment for consumer insight professionals since Q3, while Censuswide research on behalf of Stravito indicates that 74% of businesses are looking to boost their market and consumer research budgets this year.
Nonetheless, for a lengthy period last year, many insight teams were drastically under-resourced. And with regulations surrounding social distancing meaning that standard research methods such as face-to-face interviewing have been halted in many countries, insight teams have had to innovate and explore how new tools and methods can enable greater efficiency and effectiveness.
Agility and innovation
Faced with a requirement for much more agile, short-term insights, teams have had to become more adaptable.
“With face-to-face research out of the window, coupled with existing fatigue with completing online surveys, insight teams are having to rely more on digital behaviour and quantitative data,” notes Dr Bailey. “There has been a shift towards quantitative data, with a lot more emphasis on analysing purchase behaviour, then using that for modelling and scenario planning. For example, what have people purchased, when have they purchased it, how have they purchased it, what digital footprint did they leave behind, what content did they consume? Which sectors have increased/decreased consumption? Have certain sectors or companies found new business models?”
Paul Laughlin agrees that there has been a much greater use of online research methodologies. “For many organisations this has meant far more focus on quantitative surveys, however others have found that increased customer familiarity with Zoom has meant that depth interviews and online communities have worked well for qualitative research," he says. "One experienced customer research leader told me they were having great success with self-filmed ethnographic research, so there are research innovation successes amidst the disruption.”
With face-to-face research out of the window, coupled with existing fatigue with completing online surveys, insight teams are having to rely more on digital behaviour and quantitative data.
Anda Ziemele believes that insight teams are doing their best to connect the different data points that are available to them. “This often involves thinking creatively and utilising more unusual data sources that might be analysed in a silo,” she explains. “For example, location-based reports from Google, people’s listening habits, searches on ecommerce websites, community/group analyses on Facebook, health data on fitness apps, Clubhouse and TikTok. People still need to socialise, engage with others and continue with their lives so the key is finding those meaningful data points and staying on top of where this is happening in the digital world, and then being able to gain actionable insights from this.”
But, as Laughlin notes, businesses will have to bear in mind that this bigger emphasis on digital sources could exclude insights from some customer segments. He explains: “Inclusion is a concern with all these changes, there is a risk that the voice of those who are not online is being excluded. Some are seeking to avoid this with more phone usage, but the risk that an older and less affluent portion of the population are excluded continues.”
A massive win for customer insight?
So what of the long-term implications of the past year for the customer insight function? Has the ‘reset’ that the MRS Delphi Group report identified changed the insight landscape forever? Have the innovations and exertions of the past year enabled insight teams to become more skilled and agile going forwards? Or will things snap back to the way things were pre-pandemic?
For Capgemini’s Jo Peplow, the past year has accelerated change that was already in motion.
“Customer behaviour and perceptions will change permanently, and insight functions reflect this,” she predicts. “We will need a wider range of people with a varying skillset, especially in data. The key will be the application of domain knowledge, empathy, insight, curiosity, agility, and emotional intelligence when working with data.
“It will be for the better as I think we will see more diversity in research, data and technology in general, as a wide range of perspectives and viewpoints will be absolutely crucial to stay on top.
“Insight functions have been on an evolutionary journey for a while now. The pandemic has forced an increase in the pace and focus of this change. Post-pandemic insight functions will not remain still but will continually change. Reinvention and adaptation will likely be required to succeed and provide actionable insight. This will be driven by new and emerging technologies, modelling approaches and the continual consumer adoption of digital services.”
Laughlin, however, is more circumspect about what the future may hold, and believes that some challenges may still lie ahead, particularly as budgets are still tight for some.
“Financial pressures on their organisations mean many insight leaders are facing the prospect of job cuts or at least losing potential headcount. Those who have delivered well are also becoming ‘victims of their own success’ as faster delivery of research is now expected as the norm,” he explains.
“But teams also report that they feel they have taken a step change in being more proactive and educating their stakeholders on the broader trusted advisor role they can offer. Some leaders honestly express that they don’t know yet - it may be too soon to tell how research teams will need to adapt to a new normal.”
But what is certain is that the pandemic has demonstrated the value of customer insights like never before. And the hope is that this epiphany will encourage more organisations to become insight-led, and indeed invest more, in the function as we move through 2021.
“The pandemic has made it harder to collect data, but it has also put a spotlight on the necessity for insights. So overall it's a massive win for the customer insight function,” says Dr Bailey. “Budgets are so tight, companies can't afford to waste a penny, so every decision they make needs to be informed by insights.”
David Lloyd concludes: “Assuming that we can free ourselves of COVID restrictions later this year and experience some of the ‘old normal’, we can expect some level of reset back to pre-COVID days. But, we can’t ignore that many of the changes in consumer behaviour we’ve seen will have become more conditioned now and it’s doubtful they will fully shift back (e.g. online buying, working from home).
“Similarly, insight functions have needed to adapt and innovate during this period with new methods, ways of working and services, which I believe will have a beneficial permanent impact. Circumstances have demanded and delivered improvement. As Peter Drucker said “The enterprise that does not innovate, ages and declines”. And in a period of rapid change such as the present, the decline will be fast.”
Neil Davey is the managing editor of MyCustomer. An experienced business journalist and editor, Neil has worked on a variety of newspapers, magazines and websites over the past 20 years, including Internet Works, CXO magazine and Business Management. He joined MyCustomer in 2007.