Breaking down the great global data challenge

By Mark Benson, Chief Technology Officer, Logicalis UK&I

There are plenty of statistics around the staggering amount of data being created and consumed globally every day. 2.5 quintillion (million million million) bytes created daily by internet users, a 5000% increase in data interactions between 2010 and 2020, and so on.

For a casual observer, struggling to grasp the difference between an exabyte and a zettabyte, it is tempting to give up, consigning data to the same category as grains of sand on the beach or stars in the sky – just an unimaginably big number.

Those who use data as a business asset can’t afford that luxury. They need to understand the sources of this data, what it means in business terms, and how to deliver maximum value from it.

This blog considers where all this data is coming from, the drivers for future growth, and what this means for maximising business value from data. Other blogs in this series cover how to manage all this data effectively using Information Lifecycle Management (ILM) disciplines <link>, and explain some key data management terms <link>.

What is driving data growth?

To help understand what is driving the explosion in data, it is worth considering the different types and uses of data, their current and likely future contribution to growth, and the challenges they can present for organisations looking to manage and maximise data value.  

Until the 1990s, the bulk of data processed by IT consisted of the structured databases and files used by business systems and held on centralised or distributed corporate infrastructure. This data has seen largely linear growth as business system use has expanded, creating challenges around what to do with data that is no longer ‘live’ – retention policies, archiving and retrieval. 

The advent of the PC, then of an increasing range of mobile and consumer IT and storage devices, has driven an expansion in data stored locally, often duplicating that held on corporate resources. This has been a major contributor to the volume of dark data, defined by Gartner as “the information assets organizations collect, process and store during regular business activities, but generally fail to use for other purposes.” The existence of dark data raises challenges for organisations around investment in data storage, data security and compliance.   

While expanding corporate structured data use and IT device consumerisation have driven some growth, the explosive increase in data has come largely from the Internet and social media, and particularly from the massive increase in unstructured media. Digital video and sound media are generally orders of magnitude larger than data files, so they have a disproportionate effect on growth measured in bytes. Organisations face the challenge of how best to manage and exploit this ever-growing ocean of unstructured data.

The Internet of Things (IoT), automation, machine learning and Artificial Intelligence (AI) create a new level and volume of complex digital transactions, which in turn create a new level of data management requirements. Increasingly intelligent technology should grow the capability to automate management and exploitation of all this data, but there are likely still to be challenges in assuring compliance and security and maintaining human oversight of how data is used. 

How should organisations respond?

The first steps for any organisation looking to maximise value from data are to have a data strategy that is fully aligned with business strategy, and to make sure that all the data it owns is under control through robust Information Lifecycle Management (ILM).

This should focus data activity on the right business priorities, eliminate waste and business risk around data retention and dark data, and provide a solid foundation for deploying the ever-growing range of technology capabilities for exploiting both structured and unstructured data to the full.     

What next?

Logicalis UK&I have helped many clients deliver success in managing and maximising value from data.

Download our eBook to find out more about the critical role of data management in digital business, or visit www.uk.logicalis.com to find out more.