Social media monitoring: The tips and tools to get it right

Jennifer Major examines what current social media analysis tools are available and how these can benefit organisations looking to maximise this rich vein of market intelligence.

Social media sites make up at least half of the top 20 websites in most regions of the world. This has left companies, that primarily judge the health of their brands in the online world by web statistics and click-through rates, in real need of rethinking their social media strategies.

Today, organisations are doing little more than measuring volume and reach of social media engagement. In other words, "how much are people talking about us?", (or engaging in our social media channels), and "how many people are talking about us?" Unfortunately, these basic measures provide little insight into the value of social media to the organisation.

A recent survey by Opinion Matters on behalf of the Internet Advertising Bureau’s Social Media Council, confirmed that the majority of organisations are not sure how to go about analysing social media. It stated that almost three quarters of respondents found the main challenge in the social media context was in proving that it could generate ROI.

However, high profile brands, such as Pepsi, are moving away from television advertising to social media for brand awareness campaigns. This implies that there is a strong belief amongst brand marketers that social media analysis does work, and also reflects the gradual migration of many advertising budgets from television to online.

So what should they be measuring? Before considering how to analyse social media, an organisation needs to be clear on its objectives and strategy for engaging in social media. The type of analysis conducted will differ depending on the goals of using social media. The goals typically fall into three main categories:

1. Raising brand awareness.
2. Conducting market research.
3. Providing customer service – using social media as an alternative channel.

The few organisations that are measuring the impact of social media campaigns today on their brand awareness and customer services goals are reliant on traditional methods of measurement: market research, measurements of declines in call centre traffic and increased customer satisfaction.

However, hardly any organisations are making use of the content of these captured conversations to really understand what their customers are talking about.

Social media monitoring tools

From Nielsen’s Buzzmetrics, to the funkier named Tweetfeel, Twendz and Twitrratr, a plethora of social media analysis tools have hit the market. These tools all claim to provide measurement of the effectiveness of social media. However, the bulk of the social media analysis tools out there are using existing techniques to ‘measure’ this new mode of brand interaction.

The big-name companies who are now providing social media analysis fall into two main camps; the web analytics vendors and the market research companies.

Many of the web analytics vendors are simply taking their traditional means of measuring traffic volumes, and using these to measure the amount of referrals from social media sites. They will also measure the amount of discussion taking place in forums on an organisation's own web site. This indicates how much your brand or products are being talked about, but does not give much insight into the value of these conversations.

At the other end of the spectrum are the market research companies. They are better placed to provide insight into the opinions being expressed within social media, but their approach is typically around finding the answers to specific questions around brand awareness and public perception about products and services. The techniques used to assess public opinion around pre-defined topics are likely to miss out on other unanticipated topics that arise in social media conversations.

Then, of course, there are the hundreds of new suppliers who are popping up in this space, claiming to provide measurement of both influence and sentiment. While social media clearly demands new methods of analysis, it is still very difficult for organisations to know who they can trust to provide accurate and useful measurements of this type. This is partly because there are no agreed standards yet on how to measure these sorts of things.

At this early stage in the development of social media analysis, organisations should look to vendors with a strong track record in a number of disciplines. This includes text mining to identify new themes and topics, natural language processing to classify the themes in conversations and identify sentiment, and a strong basis in statistical analysis to measure the degree to which social media engagement has contributed to meeting the organisation's goals.

Social media snapshot

Organisations who are looking to use social media to achieve goals of brand awareness, customer service or market research need to recognise that social media, like any marketing programme, is not going to achieve instant results. Engagement in social media must be part of a long term strategy, and the impact can only be measured by continually monitoring it over time.

For a social media analysis tool to be effective, it must therefore be able to monitor the same  criteria on an ongoing basis, and those criteria must be focused on areas that the organisation is able to influence. To truly understand the impact that social media is having for an organisation, they must also be able to identify and monitor other factors that will have an impact on their social media performance indicators.

Many of the tools out there for social media analysis are only partly formed, and only provide a snapshot of some of the aspects of social media. Organisations must recognise that social media analysis, like the use of social media itself, requires a long term strategy.

The value of social media can only be measured if you are able to use it to understand not only how many people are talking about you, but also what they are saying, and how those conversations are changing over time.

There is also another important issue to be considered, which is around the availability of data to enable social media analysis. Ideally, an organisation would want to understand what consumers are saying, regardless of where they are saying it. However, data ownership laws may prohibit automated collection of conversations that are taking place on social media websites such as Facebook. This needs to be considered when organisations determine how and where they intend to engage in social media.

Companies that have created a community on their website are in a privileged position of owning the data that is posted to the site. This allows direct analysis of what people are saying, and supports the goals of conducting market research and providing customer service in terms of understanding what people like and dislike about the products. Owning the data also allows more generic web analysis, for example, counting the amount of traffic to the ‘community’ website, the number of comments received, commonly taken paths through the site and areas with high volumes of interest.

Companies using existing social networks such as Twitter or Facebook, are not necessarily entitled to use data posted on that site; it usually belongs to the site provider, and others are not permitted to use it for commercial gain. This means the company is unable to analyse either the content, or the activity within their accounts on those social media sites. It may be the case that negotiation with the site owner is required for the right to analyse what happens within these pages, or feeds.

Sentiment analysis

Sentiment analysis is a vital part of social media measurement. It’s no good knowing that lots of people are talking about your brand or products if you don’t know what they are saying, and whether their opinions are generally positive or negative.

However, unless you are simply interested in brand tracking, it’s not enough to know that people are saying good things or bad things about your products; you also need to know what products they are talking about, and what aspects of those products they liked or disliked.

The pre-requisite to sentiment analysis, therefore, is knowledge about what you want to measure sentiment on – what aspects of your products or brand are people discussing? This requires another discipline – text mining. This technique is already employed by a number of leading companies to identify the causes of customer complaints, and for early identification of product faults. For example, American Honda uses text mining on warranty and call centre notes to identify potential faults on their cars before they become a big issue, saving millions in recall costs and unnecessary part replacements. I bet Toyota wishes that it had implemented that sort of early warning system!

Only once you have identified what themes are being discussed, can you start to measure sentiment in a meaningful way.

You also have to be very careful about which vendor you choose to perform sentiment analysis. There are hundreds of new start-ups out there, who are providing sentiment analysis for social media – many of them for free! A lot of these will simply be looking for the occurrence of ‘positive’ and ‘negative’ terms in conjunction with mentions of your brand. This is liable to be somewhat inaccurate. True sentiment analysis requires sophisticated natural language processing. Not only do you want to be able to measure what aspect of your brand, product or service the sentiment is being expressed about, but you also need software that can understand the vagaries of whichever language it is analysing.

Take English for example; 'Not bad at all’ is often used as an understatement for ‘fantastic!’ – however, the statement ‘not bad at all’ contains the word ‘bad’ which might be crudely classified as being a negative sentiment. And what if something is described as ‘cheap’? This is bad if it’s referring to quality, but good if it’s referring to price. So before you engage a vendor to perform sentiment analysis for you, make sure that you know how sophisticated their natural language processing really is.

Accurate measurement

Sentiment analysis is an immature art at the moment. The companies to watch in this space are those that already have a track record in text mining and content classification – sentiment analysis is a natural progression from these two disciplines.

As an enabler in measuring social media ROI, sentiment analysis is only a piece in the jigsaw puzzle. Companies need to go through a number of steps, and employ a number of techniques to achieve an accurate measurement of the value of social media to their organisation:

1. Identify the goals of their social media engagement – is it for customer service, brand recognition or market research?
2. Identify the scope of social media impact that you are going to measure – are you going to try to capture all mentions in blogs, newsfeeds etc, or are you going to focus on your own discussion forums, twitter feeds and other social media presences such as Facebook pages.
3. Identify and agree the criteria that you will capture information across over time.
4. Identify other external factors that will also have an impact on these measures – and see if you can also measure these.
5. Set up a collection strategy to collect the content of social media discussions.
6. Consider other data sources that should be analysed alongside social media conversations – for example, customer contacts through other channels, such as the call centre or email.
7. Use text mining to determine the main themes being discussed – for example, what attributes of the brand/company/product are being talked about.
8. Use content categorisation to identify themes and topics being discussed in any post/comment/blog, to get a measure of how much the different topics/themes are being discussed.
9. Use sentiment analysis to identify whether comments regarding the brand/company/products are positive or negative.
10. Set up strategies (marketing plans/customer engagement plans) to influence the KPIs that are being measured.
11. Review and refine your social media measures over time.
12. Be patient! Social media campaigns will take 3-12 months to show a tangible result.
     

Jennifer Major is business development manager, at the communications, media and entertainment practice at SAS UK.