By Peter Beard, WhiteLight Systems' VP of worldwide sales.
A few years ago, people used to talk about milk or butter ‘mountains’, sitting in warehouses because they couldn’t be put to good use. In more recent times, almost all big companies have come to build up data ‘mountains’; vast accumulations of data sitting around, because it couldn’t be adequately examined for the extremely valuable information contained within it. These mountains are not only huge, but also complex. As every event and every little detail about customers has come to be electronically logged somewhere in each business, the data mountains have come to encompass almost every aspect of a businesses’ operations.
Fighting a losing battle
As always, solutions have sprung up to try and make inroads into these ‘mountains’, the most well known of which, OLAP (online analytical processing), has gained a large following. However, as the information mountains grow larger and larger, the traditional OLAP tools are falling further behind in the constant struggle to cope.
They are fighting a losing battle. OLAP works by taking dimensions of a company’s relationship with customers (profits, volume and frequency of purchasing, range of services ordered etc.) and turning these figures into a giant cube of data, which can then be examined using query tools. Unfortunately, as soon as more than a handful of dimensions or large amounts of data are involved, the cube becomes so large and unwieldy that it can grind to a halt.
Aim of analytics software
In datawarehouses today there are terabytes upon terabytes of data, divided into tens of dimensions, each of which represents a crucial part of a business’ operation, and each of which only has any meaning when related to the other dimensions.
For example, it is no use knowing that average profit margins on a particular special offer are 45%, without knowing how this high margin is keeping down total uptake of the offer, or how many customers are not taking up the offer at all. Only by referencing all dimensions against each other can you fully understand any one of them. This is the aim of analytics software.
What’s the difference?
The term ‘analytics’ has become increasingly common in the last year, especially in America where it has become a ‘buzz’ word. Late in 2000, Merrill Lynch published a report suggesting that the analytics market will grow to $25 billion by 2004. But what does analytics software do, and how is it different from the now commonplace OLAP tools?
The key factors are:
• speed of analysis
When evaluating customer relationships, analysts have to look at an enormous and complex range of factors. These multiple factors are not just the internal ones, such as customer profiles, buying patterns, churn factors, and so on, but also the external ones – statistics about market history and external direct marketing results. This complexity makes it very difficult to collate all the data in one place.
Then there is the volume of data, all in multiple dimensions, with large companies generating and then needing to cross-reference millions of data elements every day.
Finally there is the question of speed. As e-business propels the commercial world to ever faster change and adaptation, the need for products which are both speedily launched as well as based upon sound analysis of customers’ needs becomes paramount.
Virtual cube of data
The first point about analytics software, then, is that it removes the limitations on complexity and volume, and reduces the time required for analysis to a matter of seconds. It is designed to allow everything from the broadest general view, rather like an aggregated Excel spreadsheet (but in many more dimensions), to the most detailed close-up, ‘drilling down’ to examine a particular short time period, for example.
Instead of attempting the impossible and building a colossal, static and cumbersome data cube with everything inside it, analytics software only takes data from back end sources as needed – a part-filled ‘virtual cube’ that sits between these back end sources and the front end analyst, providing CRM analysts with a real ability to see how customers react to different prices and offers.
These analysts are then able to notice an immediate difference. If a number of customers similar in profile change buying habits, the company will instantly be able to spot this andexamine fully the possible reasons for the change. This full picture will be very telling, revealing as it does the precise nature of the change, as well as every other factor surrounding that change.
An example of this would be, if all factors remain the same except the largest customers in one particular region drastically reducing purchases, it might be inferred that a competitor has undercut the company’s rate, and that the customers switched to the cheaper offer. The company can then react to this market change by re-examining the profitability and success of the company’s offerings in that area.
This is where the second distinguishing feature of analytics software is key, the element of‘predictive analysis’. It is all very well being able to look at past data, asking ‘what happened?’ or perhaps even ‘why did it happen?’ However, these are not the questions that CRM analysts need to answer. The only important information they need relates to the future, the ‘what if..?’ questions about future implications of the decisions they make now.
Using a spreadsheet it is possible to do basic ‘predictive analysis’. It simply involves changing a cell value, then recalculating to come up with possible effects on the rest of the chart. Unfortunately, this is not the most efficient method.
• First, it uses only a small amount of top-level data, without going deeper into the figures behind these aggregates.
• Second, it cannot compare more than two dimensions against each other – not much use if you need to change one value then see the effect of this on a further twenty dimensions.
• Third, doing spreadsheet-style predictive analysis means that data must first be exported from an OLAP system, cut down into a small subset of top-level figures and then imported into the spreadsheet. This is hardly an efficient process for carrying out data studies.
Direct from the source
Using analytic software, data is accessed directly from its source instead of switched into a spreadsheet.Because the software imposes almost no limits on complexity and volume of data, it is possible to zoom in from the widest view of the data to the most detailed view, changing values and recalculating the entirety of the data.
This is true ‘predictive analysis’, in which any element of the data can be modified by an analyst, who then sees the effects of the modification on all the remaining data – basically, on-the-fly generation of future business scenarios based on all the facts, and answering the most important questions about the potential impact of business decisions.
The third key facet of analytics is the ‘closed loop’. The idea of a loop refers to the circular process, which is involved in data analysis.
First, data is examined by an analyst to gain a summary understanding – how many customers there were, how much profit was generated, and so on.
In the next stage, the analyst looks at the data in more detail, ‘drilling down’ to analyse less profitable areas and look for poor performance in need of improvement.
After this is the ‘predictive’ stage, in which the analyst alters certain values to test out possible scenarios – for example, queries like ‘if prices in our premium range were cut by 5% and direct marketing increased by 10%, what are the implications for profits from our biggest-spending customers?’
Then comes the ‘prescriptive stage’, in which the analyst decides on the best course of action, following the results of the scenario testing. This is a complete process.
When the different parts of the process are not performed using the same program, nor one immediately after another, the process becomes tortuous and time-consuming. Working with GE, WhiteLight found that this gap-filled ‘analysis loop’ took six to nine months to go through a single cycle – a cycle which analytics software can shorten literally to a matter of minutes. This is because the entire ‘loop’ is a closed cycle, with each stage taking only seconds to minutes.
An end to guesswork
The advanced analytics approach has not been possible until recently, and companies stand to gain a great deal by mastering these data mountains, understanding customers and controlling costs, trends and opportunities. Analytics offer an end to guesswork, rule of thumb modifications, and lengthy analysis cycles. Analytics software offers the ability to reassess and re-evaluate in real time, giving – for the first time – an instant and single view of a customer base.