With the Midata initiative promising a shake-up of the UK's data protection laws, Andy Richardson believes that businesses will need a vision of how data should be made available to consumers if they are to get to grips with the task of standardising data.
Following an article on Midata I wrote last week for MyCustomer.com, I was asked to appear on BBC Radio 4’s You and Yours with Professor Nigel Shadbolt of Southampton University, to discuss the subject further.
Midata represents a major shake-up of Britain’s data protection laws, which proposes to make it possible for consumers to request electronic copies of the “historic transaction data” that retailers or banks hold on them.
Professor Shadbolt wants supermarkets to routinely release information transaction information as a means of increasing price comparison and fairness for the consumer. One thing which struck me during our conversation was the professor’s assumption that companies will be able to provide individuals with their data in an ‘open standard’ format.
It seems to me that the task of standardising and formatting data sharing processes between companies in order to aggregate content on an individual’s transactions and business dealings is, frankly, mammoth. Even within one organisation, such as a supermarket, customer details may appear on numerous databases – they might have a credit card, insurance and even a mobile phone contract all with the same organisation. A bank may easily be storing information on a single customer across four or five channels. This information is often stored on old systems designed to support a transaction rather than report it. Without single data warehouses, companies will require massive business intelligence efforts in order to collate their own information on consumers uniformly.
And then there is the question of how organisations will adequately collate this data. For example, the information that retailers might hold on customers (that is, their purchasing habits, typical spending power, and extrapolations on their lifestyle) will be highly dissimilar to the database containing an insurance provider’s dealings within an individual, determining whether he or she is suitable to insure, and what level of risk they pose. To agree a common format across multiple sectors (let alone make that data consistent and useful to consumers) will be an extensive process in itself.
Without a sturdy vision of how this data will be available to consumers, how can business come together and even begin to get to grips with the task of standardising data? As part of its Consumer Data Principles the UK Department of Business, Innovation and Skills (BIS) states that consumers should be able to “access, retrieve and store their data securely” and that consumers should have the ability to “analyse, manipulate, integrate and share their data as they see fit” which includes participation in collaborative or group purchasing. As such, BIS is surely advocating a common portal, functioning in the cloud with the capability to manipulate and integrate data so that individuals can access their own and others information, comparing themselves with their peer group and accessing information on trends.
A site compiling this amount of data will surely require business intelligence specialist involvement from a design stage in order to utilise the content effectively, and safely with regard to data protection and portal security. Given that most Business intelligence projects fail, there must be a trusted consultancy involved throughout the project, for it to stand any chance of success.
It seems clear to me that businesses will require a massive amount of support in order to provide this service to consumers. And, should the provision become a workable reality, organisations will also require extensive support networks in order to efficiently provide customers with clear explanations of how data was collected, and what it represents (a further BIS requirement). The level of data capture and multiple channels of capture will require technically eloquent customer service representatives on a vast scale.
Ultimately, I support the call to provide customers with their own data, and I feel it is certainly a service that can provide substantial benefits to the consumer. I think, however, that this might be an instance of a grand premise which centres around too long term a solution. Might a shorter term, phased approach that can deliver key benefits to the consumer over months rather than years not be better? Even the shift to looking for standard formats within an industry rather than across industries, and several databases rather than one, might go some way to making this idea more attainable in the near future.
Andy Richardson is CEO of independent software consultancy Influential Software.
Comments
midata pain?
Andy ,
You have introduced a major misunderstanding in this article - exactly the opposite of what the midata programme is trying to do.
You say that BIS's idea that "consumers should be able to access, retrieve and store their data securely and that consumers should have the ability to analyse, manipulate, integrate and share their data as they see fit" means that it is "surely advocating a common portal, functioning in the cloud with the capability to manipulate and integrate data so that individuals can access their own and others information, comparing themselves with their peer group and accessing information on trends".
Quite the opposite. BIS is advocating releasing data back to individuals. This is absolutely NOT 'a common portal' - though it may point to the rise of new services such as personal data stores, which help individuals securely store, manage and share their data under their own control.
Also, I think you may be over-egging the "mammoth" data standards challenge. A CSV file is not the cutting edge of data integration. Also, the midata consultation focuses on the release of transaction/usage data which most companies already have in their billing/CRM systems. This does not require a mammoth new data integration project.
In other words, we are doing exactly what you advocate: "a shorter term phased approach that delivers key benefits".
Alan Mitchell
Strategy director, Ctrl-Shift and midata Project Board