Using AI and social data to boost CX

AI-enabled consumer intelligence (AICI) is the next big thing when it comes to data-driven CX, marketing, and product development (discussed in my article from last year). And among AICI applications, perhaps the most exciting and least understood use case is using AI and social data to spot unmet consumer needs. Not just to improve product or service experience, but also to identify opportunities for innovation. Anyone in client-facing roles should be paying attention. But also thinking: “isn’t this what social listening was supposed to address?” Yes! And no.

To understand why, a quick bit of history.

Social listening is necessary, but not sufficient to predict what consumers want 

Back in the day, social monitoring (social intelligence v1) was mainly about tracking keywords and mentions about your company, products, campaigns, competitors, or other topics of interest. It was mostly owned by social marketing teams, who were usually detached from other VOC or customer analytics efforts. It’s fair to say these efforts were reactive at best (remember the “United breaks guitars” saga and response…) and good for tracking share of conversations on individual social channels, but poor for really finding actionable insights. Or at least finding them before it was too late!

Enter social listening (social intelligence v2).  When teams started getting together, and gathering and analyzing data from a wider variety of social media sources (and a few others), it became possible to see bigger trends, and power decisions beyond core social teams. More comprehensive brand tracking and brand protection programs became possible. And marketers started gaining insights to create more effective social campaigns, and started to identify and engage with communities of interest on platforms like Twitter or Reddit.

We also started to see the potential of using social to be more proactive. Could you spot pockets of discontent in your user base before it became a full-on product boycott? Or spot service and support frustration by tracking hash tags or key influencers? Likely. Could you see less obvious, emerging consumer trends or other “unmet needs” like the next food trend, or fashion movement. Possibly. But not without looking at signals beyond just social. And certainly not without AI and data science to process the millions of conversations, clean out noise, spot patterns, and help teams explore what is happening - and human helpers to add cultural context, behavioral frameworks, and see if trends you spot will increase or peak and fade. 

So, that brings us back to AICI (social intelligence v3). And how this model is enabling both CX and insights teams to get their online and offline data together, and deliver insights that support CX, customer success, and yes, even sales and marketing efforts more broadly across the enterprise. To make this leap we continue to need social signals. But also inputs from customer surveys, and search patterns, and other owned data from websites and marketing tools and beyond.

But the real enabler is AI. And more specifically machine learning. 

This is why the most interesting developments in CX are where AI meets consumer insights (see our free Ipsos Views report on the growing role of AI for consumer insights). The cutting edge of this application is using semantic AI and machine learning to process and visualize what consumers are saying and doing. At Synthesio we call this Topic Modeling, and it’s a direction we see most brands heading as they look to master the next generation of social intelligence.

Here’s a view into how this approach works and what CX teams can expect as more vendors embrace this flavor of data science in their insights (and engagement) platforms.

New innovations in machine learning let CX teams spot emerging issues across data sets – and even create better satisfaction surveys

CX teams know that customer surveys – delivered online or via email or even phone, provide critical clues to satisfaction, product performance, etc. But anyone who has designed a survey knows that you typically start with… the questions you asked the last time! But what about the questions you didn’t yet know to ask? Or related to novel use cases, or surprising experiences (good or bad) that your client had while engaging with a rep or fellow user.

Topic Modeling uses a bottom-up analysis of conversations (from any source, but here we’ll focus on social channels), to look at what users are saying, clustering conversations, labeling the clusters, and providing a visual workbench to explore and compare insights. At Synthesio, our tool was co-built with the smart folks at Ipsos to productize more than 5 years of research methodology related to trend detection. Vendors often leverage relationships with universities as well.

In our new Topic Modeling solution, we apply a flavor of “semantic AI” in the form of unsupervised machine learning to find patterns in a social data set that may indicate emerging trends – and visualization tools to explore and validate new topics – what has been described as “unknown unknowns.” For everyday practitioners, this means the tool can scan millions of consumer data points, automatically clustering and visualizing conversations into themes. Then analysts can explore and test these themes, and use them to inform a follow-on consumer survey, training for the support team, campaign planning, or even next year’s product roadmap.

As my colleague Sandro Kaulartz has noted, discovering unknown unknowns from people data is really the pinnacle of consumer intelligence. Starting with user-generated data – like social posts or consumer reviews, this approach provides a big sandbox to find the unexpected and explore openly shared consumer experiences. 

Under the covers there are 3 flavors of AI at work:

• Semantic AI aka Natural Language Processing is applied to understand consumer language and what they are saying,

• A clustering algorithm (for data science nerds, we use an approach called HDBSCAN) categorizes conversations, producing a topic landscape, and 

• A Natural Language Generation (NLG) algorithm generates label for clusters to help teams identify and focus on drivers or unmet needs. 

An example in the consumer beauty sector

A great example is in the consumer beauty sector, where customer satisfaction and loyalty is increasingly tied to how the brand is embracing a sustainability imperative, in everything from sourcing and business practices, to packaging. 

In this case, a goal was examining consumer experiences with sustainable packaging and refillables, a topic important to both brands as well as their retail and hospitality partners (think of all those bottles of shampoo and lotion in every hotel you’ve every visited!). An initial Topic Modeling study exposed expected topics like the demand for “zero waste refillables,” but also found unmet needs related to experiences where packaging broke or wasn’t as durable as expected.

Digging deeper into specific social mentions and online communities, Topic Modeling also helped the team discover new innovation opportunities, such as bamboo as an alternative packaging material due to its uncommon strength and biodegradability. 

With these new insights around the CX of current options, the team could then launch a focused follow-on survey to further see what consumers are thinking and identify their purchase intent. Specifically, the survey validated the importance of sustainability in the product consideration process (applicable also to merchants like a retailer or hotel which featured these products). And also showed that a third of consumers would purchase a bamboo refillable product if one was offered.

The best consumer insights create change. In this example, better CX starting by listening to what consumers really wanted. And then being more responsive by letting AI do a lot of the heavy lifting – to spot needs, suggest options, and allow teams to find solutions faster, before one bad experience becomes a trend.