Why unstructured data holds the key to understanding the customer

As counterintuitive as it sounds, the majority of opportunities to understand your prospects and customers currently lie outside of your customer relationship management (CRM) system.

Conventional CRM does a very good job of recording a customer’s interactions with an organisation in a highly structured and regimented way. The data recorded about each customer in such systems has to be ‘structured’ because a CRM tool contains predefined fields which necessarily impose structure. Typical examples of this might include prospect/customer name, account number, company, purchase history or time of purchase.

To be sure, structured data like this is hugely important for understanding customers, however, consider all of the insight relating to your customers and prospects that is not recorded in your CRM database.

Buried away across your emails, reports, spreadsheets, contracts, warranties, telephone/member listing books, advertisements, marketing materials, annual reports, customer call logs, employee evaluations, ordering information, surveys, social media, and blog posts is a whole raft of customer insight that is ‘unstructured’ data.

At the DreamForce conference last year, Salesforce CEO, Marc Benioff, put the unstructured data situation in context by explaining that a focus on typical transaction and account information meant unstructured data now outweighs structured data five-to-one within the marketing, sales and service environments.

Benioff went on to identify that the biggest imperative for customer-centric companies is to make sense of their unstructured data so as to be in a position to leverage it to better understand and influence the customer journey.

Adding structure to the unstructured

The problem with all this unstructured data being produced from connected products, customer-facing apps and social networks is that it it is often completely disconnected from the business because legacy analytics software is unable to process it.

Organisations need to look to next generation analytics tools that are able to make unstructured data useful by doing these three things:

• Aggregation
• Analysis
• Standardisation

Aggregation refers to the act of pulling together multiple sources into one central repository. All of these sources of unstructured data - all of this content - need to be freed from their organisational constraints so that analysis and standardization can take place. This aggregation can often be done through methods such as RSS.

The next part is applying some kind of analysis to this mass of unstructured data to understand what’s actually there.

This can’t be done efficiently or at scale with a human workforce. Content volumes are ballooning according to Benioff:

“90% of the world’s data was created in the last two years...There’s going to be 10 times more mobile data by 2020, 19 times more unstructured data, and 50 times more product data by 2020."

Semantic analytics technologies which use NLP (natural language processing) can be applied to unstructured content to ‘read’ it and identify the ‘topics’ that are contained with each piece of content. Examples of such topics could be people, places, products, model numbers, etc. Extracting these topics from content makes it easier to understand at a glance what the key themes are of the content without having to pore over it yourself.

Finally, comes standardisation: the act of creating some semblance of order (structure) to the unstructured data that you have aggregated and analyzed. This can be done through adding descriptive metadata (or ‘data about data’). The metadata could include basic descriptors such as author name, length and published date, but it could extend to the very topics extracted from the analysis of your content. Again, automating this process is much more desirable to doing it manually.

The result after these three steps is a corpus of previously unstructured data that has now been analysed and standardised. It is from here that customer-centric organizations are then in a position to use the data to derive actionable insights their customers.

Making unstructured data useful for customer insight

There are many things you could learn from using the insight derived from unstructured customer data.

You could analyse customer feedback in call centre transcripts and identify trends in product and service reputation and faults very quickly. Or, you can analyse your social community on Twitter to find the topics, sentiments, locations of their tweets, and of the links they have shared, to build up a much more detailed picture of your followers.

Perhaps the most interesting leap is consolidating unstructured data with the structured data already inside of the CRM environment to gain a more complete view of the customer. As Karine Del Moro intimated here, unstructured data poses a real problem if it’s treated differently from the rest of your information.

For a sales rep using a marketing automation or CRM tool for prospecting, it’s helpful to have a structured record of which pages a customer has viewed on my site but it’s even more powerful if topics being discussed in those pages are made clear to me so that I can understand what in particular was of interest to my prospect. Using that insight means I can immediately determine the interests and intent of my prospect, and engage in a more relevant sales dialogue.

When this kind of insight from unstructured data is made available in CRM tools, it gives the competitive advance that Benioff envisioned at DreamForce, where sales, marketing and customer service reps are able optimise their conversations with prospects and customers using interest and intent data.

This is something being explored by asset management company, Climate Change Capital who use idio to integrate their content marketing with their CRM system. Carlos Moran, who heads the marketing and communications activities, explains why:

“[We] aggregate our website's content to show us which of our topics are getting more interactions from visitors. By organising the information we have about it and feeding it into our CRM system - it allows us to draw insight from it and act upon it.”

If organisations add structure to the unstructured behavioural data they own, and make order out of chaos, they will be in the enviable position of being able to understand and influence their customers from the beginning to the end of the purchase journey.

Andrew Davies is CMO & co-founder of idio.