Don’t Drown In The Data Lake

Getting enough data isn’t the problem. Capturing and storing the data isn’t the problem.  Building valuable insights isn’t the problem. The true challenge of analytics is inserting insight into tactical systems and using insight to shape long-term business strategy.

Traditionally data has originated from customers or high level organisational activities. Micro-events and detailed data streamed from operational events, sensors, or devices are now rapidly filling the data lake. We understand large dumps of data and streaming data to supply data lakes, but being able to fill a data lake is less than half of the battle.

The execution of this insight impacts, for example, decisions such as price changes, customer retention initiatives, marketing investments, asset maintenance, production (demand) forecasting, and healthcare planning.  The work by Analytics8 (and the best analytics professionals everywhere) becomes pervasive in every decision made every day across the organisation.  Consistently delivering this type of capability, and at low operational risk, also demands rigorous information and data management processes to be prepared from the start – from the depths of the data lake to the lofty cloud-enabled customer-facing reporting solutions. 

The I.T. systems of an organisation are rarely simple, and change is the only constant.  Making analytics operational requires integration and data governance to ensure analytical algorithms and processes continue to operate successfully whilst data sources are continually added and changed within the data lake.

Failure to adopt a strategy of data governance and integration will lead to diminished performance from analytics initiatives, regardless of how advanced or innovative they are.  Surveys of organisations that have undertaken analytics initiatives support this viewpoint that implementation and support structure of analytics is key, for example the Harvard Business Review “Minding the Analytics Gap” states:

“…the number one barrier mentioned by survey respondents involved translating analytics into business actions – in other words, making business decisions based on the results, not producing the results themselves.”

An effective communication and reporting strategy for embedding analytics throughout the organisation remains a key obstacle to achieve successful outcomes from analytics initiatives, as highlighted in this HBR article “Embedding Analytics for Growth: Creating a Data-Driven Culture”, specifically: 

“…Using big data and analytics to solve problems and make better decisions requires new behaviors across all the business functions in an organization. People may need to share and collaborate more…”

Data is often described as the life-blood of an organisation, but it is nothing without an effective support-system to use it. We often see innovation drown without the essential data plumbing that integrates analytics experts, and a feedback loop that allows business subject matter experts and operations to report upon and refine data analytics in an on-going basis.