Humans are driving a data deluge - it's time to optimise your data value chain

For many the Data Age started with the Google IPO in August 2004. It was clear then that the industry would thrive, even though in those days very few people had figured out that it would become a multitrillion industry.

Now, over a decade later data has become a first class commodity, whereas the cost of data collection and retention has fallen prodigiously. In fact, one gigabyte of data decreased from $10,000 in 1990 to $0.10 in 2010.

Technical innovations have removed the barriers to mass data collection and we have begun to witness the surge of Human Data with a constructive maturation of wearable technologies.

The data industry is shifting towards Open Source and data science

It might not seem obvious, but Open Source is central in the upcoming development of the data industry. Android OS and Mozilla Firefox are just a few examples of everyday life applications used by general consumers. The truth is Open Source’s constant evolution allowed it to significantly dominate high-end technologies such as supercomputers, Big Data, Cloud platforms and web servers. A study published by the Ponemon Institute states that “more than 70% of US IT professionals prefer Open Source to proprietary software for continuity & control”. This became one of the key learnings of 2014 – the use of Open Source not just primarily for its ability to reduce costs.

Anecdotally, Open Source can be seen as a spaceship – amazingly powerful but very complex to fly – where data scientists have proved to be outstanding pilots. Data industry pure players understood the importance of the human factor, thus their appreciation of data scientists who master all of the statistical methodologies, developer’s capabilities and machine learning skills.

LinkedIn recently published its “25 Hottest Skills of 2014” based on 330 million user profiles, and guess what? The most requested skills were data mining and statistical analysis. It’s not surprising that more and more companies are hiring data scientists in order to structure the data deluge and maintain ethical standards.

The hype around wearables, IoT (Internet of Things), Smart Homes and Smart Cities is definitely more justified than what it was two years ago. For instance, Samsung CEO, BK Yoon designated the IoT as a core pillar in his company’s strategy, during his keynote at the recent Consumer Electronics Show in Las Vegas. The influential CEO stated that every single Samsung product will be connected to the Internet of Things by 2020, and even more interestingly, he announced that all of its Internet of Things devices will be open!

Even though Human generated data is a small subset of the “data deluge”, it is surely the most valuable for marketers, since most of the overall generated data is machine to machine and artificial intelligence oriented. Human Data is only streamed through wearables and certain smartphones applications, which is why, large technology corporations such as Apple and Samsung are heavily investing into advancing these market segments.

Human data has proved its profitability within contemporary business models

Some very successful companies have based their business models on the justification of human generated data (geolocation, personal preferences, etc) via dedicated applications. Great examples of new market entrants that are creating disruption as a result are Uber and Airbnb; both of which challenged the leaders of their respective industries through innovative applications. The existing market leaders remained static, strangled by a conservative approach and unable to leverage new and innovative opportunities – instead remaining unfilled by the Data Age.

Indeed, business executives and senior managers are increasingly experiencing a transition and maturation in understanding and optimisation of the data value chain. In particular, a couple of key trends have become readily apparent.

Businesses that succeed usually avoid never-ending data crunching to focus on what matters, instead digging where the value is for both the company and the consumers. They adopt agile and cost-effective approaches through POCs – proof of concept. But, if that was general business philosophy, what is most interesting is the human factor, because put simply, there is no perfect solution to manage all of your data. It is only possible to optimise the mix between the human brain and machine automation. Data scientists are key to understanding this in controlling algorithms as well maintaining ethical standards.

What’s in store?

2015 will definitely be a year to watch from a Data perspective. Open Source will continue to dominate and Data Scientists will have a key role to sustain its development, particularly in analysing the large Data generated by the broader adoption of wearables and the Internet of Things. It will be interesting to observe the strong growth of digital health products and services.

There is a real excitement about the use of this technology to lower health care costs, promote behavior change and improve patient compliance. The only factor that can seriously slowdown the Data Industry is a possible correction of the technology industry, in which there were more IPOs in 2014 than in any year since 2000. But the Data Industry is currently thriving, so instead it is exciting times as human data collection accelerates and as a result Data Scientists become increasingly expert at delivering insights and value to businesses and people.

Othmane Zrikem is digital strategist at Ekimetrics.