Will Data Historians Die in a Wave of IIoT Disruption? A transformation in data historian thinking will happen!

A group of us were asked to comment on this article by Matthew Littlefield , President and Principal Analyst, LNS Research, on October 22, 2015. It certainly is an integrating questions, and valid question in the current industrial , operational transformation that is happening around us. As we answered it on email, I thought it is a valid topic for blog discussion.

http://www.automationworld.com/databases-historians/will-data-historians-die-wave-iiot-disruption

My immediate first response is “that the traditional thinking of industrial data historians will transform”. Actually it is already transforming, due to type , volume, and required access to the data. It is important to not look at the situation as a problem, but as a real opportunity to transform your operational effectiveness through increased embedded “knowledge and wisdom”:

The article raises the question of how or is this a disruptive point in the industrial data landscape, I would argue that is a “transformation point”.

Mathew states in the article:

“Even so, one area of the industrial software landscape that many believe is ripe for disruption is the data historian. The data historian emerged out of the process industries in the early 1980s as an efficient way to collect and store time-series data from production. Traditionally, values like temperature, pressure and flow were associated with physical assets, time stamped, compressed, and stored as tags. This data was then available for analysis, reporting and regulatory purposes.
Given the amount of data generated, a modest 5,000-tag installation that captures data on a per-second basis can generate 1 TB per year. Proprietary systems have proven superior to open relational databases, and the data historian market has grown continually over the past 35+ years.
The future may seem very bright for the data historian market, but there is disruption coming in the form of IIoT and industrial Big Data analytics.
As these systems have been rolled up from asset or plant-specific applications to enterprise applications, the main use cases have slightly expanded, but generally remained the same. Although there is undisputed incremental value associated with enterprise-level data historians, it is well short of the promise of IIoT.

In our recent post on Big Data analytics in manufacturing, I argued that Big Data is just one component of the IIoT Platform, and that volume and velocity are just two components of Big Data. The other (and most important) component of Big Data is variety, making the three types structured, unstructured and semi-structured. In this view of the world, data historians provide volume and velocity, but not variety.

If data historian vendors want to avoid disruption, expand the user base, and deliver on the promise of IIoT use cases, solutions must bring together all three types of data into a single environment that can drive next-generation applications that span the value chain.

It is unlikely that the data historian will die any time soon. It is, however, highly likely that disruption is coming, making the real question twofold: Will the data historian be a central component of the IIoT and Big Data story? Which type of vendor is best positioned to capture future growth—traditional pure-play data historian provider, traditional automation provider with data historian offerings, or disruptive IIoT provider?

If the data historian is going to take a leadership role in the IIoT platform and meet the needs of end users, providers in the space will have to develop next-generation solutions that address the following:

·         How to provide a Big Data solution that goes beyond semi-structured time-series data and includes structured transactional system data and unstructured web and machine data.

·         How to transition to a business/pricing model that is viable in a cheap sensor, ubiquitous connectivity, and cheap storage world.

·         How to enable next-generation enterprise applications that expand the user base from process engineers.”

The comments are very valid, that the data we now capturing is increased in both volume and variety, but I would argue that it needs to transformed into contextualized information, to knowledge so that  proportional wisdom growth can occur. The diagram below shows the potential direction many companies can go, of blowing out on data and not gaining the significant advantage of wisdom for operational efficiency from the increased data in the Industrial “sea”.

The way in which people will access and use data is transforming, they not using it just for analysis on traditional trends etc. They are applying big data tools, and modeling environments to understand situations early in assets condition, operational practices, and process behavior.

They are expecting to leverage this past history to predict the future through models that “what ifs” can applied. They are expecting access to their answers from people who with limited experience, in role or location (site/ plant awareness). They will not use traditional tools, they will expect “natural langue search” to transverse the information, and knowledge “ no matter where the location.

The article took me back to a body of work I collaborated on with one of the leading Oil and Gas companies around “Smart Fields” and in those conversations we talked about the end of the historian as we know it, due to the distributed nature of data capture, and the availability of memory, why would historise to disk vs leave the history in the device in memory.

I think this really drives the thought pattern around how the data is used, and the key 3 are:

• Operational “actionable decisions”
• Operational/ process improvements, through analysis and understanding to build models that transform situations in history to knowledge about the future.
• Operational, process records archiving.

The future is federated history that partitions the “load” between most-recent transient fast history in the device itself (introducing a concept of  “aggregators”) with periodic as-available uploads to more permanent storage. These local devices will have their own memory storage and can “aggregate” the data to central long term storage.

But when you are access information in the now you will not go to historian, you will go to the information model, that will navigate across this “industrial sea” of data and information, delivering it fast, and in a knowledge form.

So is the end of historian here, I would say no, but certainly as the article points out the transformation of the enterprise information system is happening, so are the models you will buy, manage, access the data.