Representative End User Clients
Representative Automation Clients
Representative Software Clients
Industrial data is the key to digital transformation and operational excellence for many organizations. However, managing and utilizing industrial data poses many challenges, such as data volume, variety, velocity, quality, security, speed (e.g., split seconds to minutes/real time), connectivity and integration. To overcome these challenges, organizations need operational historian solutions that can acquire, store, analyze, and visualize data from various industrial sources, systems, machines, sensors, and locations.
Operational and enterprise historians are among the most widely used data management solutions in the industrial sector, as they provide historical time-series data storage and analysis capabilities for process optimization, production efficiency, troubleshooting, and reporting. However, as the demands and expectations of industrial data grow, the traditional role and function of historians are evolving, adapting and expanding. Partly because of newer technologies like cloud, middleware, data lakes, data, hubs, artificial intelligence (AI), digital twin, digital thread and other data platform technologies. In this report, we will discuss the current state and future trends of operational and enterprise historians, and how manufacturers and operations can leverage new data technologies and platforms to enhance value and performance.
The operational historian, often referred to as an industrial data platform today, is not going away and will continue to expand with new capabilities that enable modern streamlined processes and technologies and the ability to manage and store more data in cloud historians. Historians are more than connected databases, because not only do they collect and aggregate all types of plant data, but they also model and contextualize it and organize plant data into trends, graphs, reports, sequence of events (SOE) and digital dashboards. While there are some - mostly smaller plants and operations, that rely on older standards, traditional on-premise historians, and Excel spreadsheets and PowerBI for storing and managing real-time data, many mid and larger plants either have or are planning on transitioning to more modern operational historians and enterprise operational historians because of the industrial capabilities that they offer. Newer historians can store operational data more efficiently and securely and make it available for all types and roles of workers internally and externally with partners.
The scope and complexity of operational historians has expanded beyond the traditional historians that were primarily deployed on site and managed only real-time data for a particular plant. ARC has witnessed a proliferation of hybrid operational historian/operational data platform installations that couple the traditional strengths and quick response times of on-premise installations with newer cloud-based installations. Historians will continue to evolve and will remain a critical part of the overall operational data architecture along with its integration and interoperability and expansion with enterprise data systems.
Ease of use and visualization are an essential capability for enhancing worker experience in industrial operations as it pertains to data. Modern operational historian platforms provide workers in different roles and positions access to the data and intelligence they need in time to perform their tasks and often to prevent abnormal behavior. They can quickly visualize data on digital dashboards with trends, images, videos, 3D AR/VR, digital twin, simulation, digital thread data, weather and operations costing data that can be easily customized to their roles and needs.
Newer historians are beginning to use AI powered contextualization that allows subject matter experts to gather relevant information quickly using a natural language search, produce meaningful models, dashboards and reports and work collaboratively across teams and organizations to help identify and take action to prevent potential abnormal behavior. The next-gen historian enhances the user experience for the new generation of workers who do not have the same skillsets or process knowledge as the generation that is retiring, and it helps them gain an increased understanding of complex operations and processes and assets quickly.
Key findings from ARC research include:
The operational data historian/data platform market continues to grow and is fueled by increasing demand for industrial automation data, new IIoT data, images, video, weather, energy and newer types of industrial data.
Most industrial companies would like to collect even more data, from existing and new sensors, machines, automation systems and other data sources.
Operational and enterprise historians are evolving and adapting to the growing demands and expectations of industrial data (e.g., real time and other types of data) with new capabilities, including better automated data management, contextualization, models, data-based analysis and visualization (e.g., digital dashboards, reports, etc.).
Newer data technologies and platforms, such as cloud historians and data lakes, are enhancing the value and performance of on-premise operational and enterprise historians by enabling scalability, flexibility, interoperability, comparisons across the enterprise and better data-based insights.
The future operational historian/operational data platform is being influenced by newer kinds of data, lower memory costs, higher computing power, and the use of cloud, edge and other new technologies.
Newer historians can store operational and other types of data e.g., unstructured, semi-structured data including weather, energy, AR/VR, etc.) more efficiently and make it available for all types and roles of connected workers and partners internally and externally.
New historians will have better tools for the connected worker to be able to access and visualize data easier, and when combined with applications and/or advanced analytics/ including AI, will result in more precise actionable data decisions.
AI will have a profound impact on the operational and enterprise historian market with more AI tools being developed and embedded as a tool for users that will include having more intuitive digital dashboards, auto connect functionality and other auto functionality, and better data-based predictions.
Traditional Operational Historian to Next-Gen Industrial Enterprise Historian
The past few years have been a time of transformation for operational historians resulting from digitization, the digital transformation, and the need to store and manage more and different types of industrial data to run a plant. Although more changes are expected in the near future that include automated integration of newer types of data, better connectivity, naming and other standards, newer data models, newer databases including open-source databases and historians available on the edge or in the cloud.
Traditional operational historians became a central repository for spreadsheets, sensor data, automation system data, machinery data and the data and metadata in other data sources. Traditional operational historians have always integrated data in silos from disparate databases and other data sources in operations and plants, contextualized, managed and stored data on-premise. Most of the data in traditional operational historians' data was 97 to 98 percent tag type real-time data (tags, report, quality, work order, recipe, etc.) along with its metadata. Today only about 90 percent of the data is real time tag data because other types of data are growing faster.
Newer modern historians have expanded and can accommodate traditional types of real-time data and other types of data needed for today’s industrial processes. The next-gen historian includes and supports the data infrastructure, data collection, integration, trending of data, reporting of data, analytics, visualization (including AR/VR), digital twin simulation, dashboards, or digital displays that operators use for intelligence at a glance. In some processes it is also necessary to integrate costing information so that workers can understand how the manufacturing process and products impact the bottom line. The next-gen operational and enterprise historian empowers teams to make operational improvements tied to productivity, sustainability and other business objectives.
The operational historian for the enterprise extends from plants, sites and across the enterprise in the cloud to enable comparative data analysis across sites, plants and the enterprise, with capabilities that lead to process optimization and reduced costs. The enterprise historian often includes data hubs for data management, data integration, data traffic and data lakes for data storage of all types of data in raw format in the cloud. Enterprise historians typically connect multiple on-premise and cloud historians from different locations. Some architectures include a middle layer or cloud historian layer that connects multiple historians. The integration among these elements will be seamless and scalable, enabling effective processing of real-time tag data along with other types of operational data. Enterprise historians are valuable for data comparisons and are automating some of the capabilities with AI, ML and other advanced analytics either built into the historian or working with a third-party supplier often included with the next-gen historian. Enterprise historians enable process optimization from better data-based insights and actionable decision making.
Operational Historian/Data Platform Trends
As the industrial sector continues its digital transformation and embraces new technologies such as the Industrial Internet of Things (IIoT), cloud computing, machine learning (ML), and artificial intelligence (AI), the role and function of operational and industrial enterprise historians will continue to change and expand. The modern operational historian will not only acquire, store, and analyze data from multiple industrial historians and data sources, databases, systems, and locations, but it will also integrate, and process industrial data combined with newer sources of data using advanced technologies such as AI. Data will continue to be stored on-premise especially for mission critical and hazardous operations, but data in the cloud, data lakes, etc. will continue to grow faster than on-premise. By effectively managing, governing, and orchestrating data in operational historians and enterprise historians, organizations will further unlock the value of real-time, historical and newer types of industrial data. This will result in enhanced operational performance, increased efficiency, and more informed and better decision making across the organization.
Traditional operational historians became a central repository for spreadsheets, sensor data and the data and metadata in data sources. and modern enterprise historians can accommodate traditional types of real-time tag data and other types of data that are needed in industrial applications today.
Emerging Trends and Auto Capabilities
Future historians will provide more efficient data operations that enable seamless connections, better interoperability between systems, auto-updates, auto-deployment, auto-dashboards, and other automated data operations such as automatically contextualizing and modeling the data to make it meaningful. By doing so, operational historians and the people who analyze the data will help industrial organizations gain insights, improve performance, and optimize business outcomes. Some of the other emerging trends and capabilities of the future historians include:
Automated Data Collection, Integration and Standards
Future historians will be able to ingest and integrate data from a variety of industrial sources, such as sensors, controllers, machines, systems, databases, datahubs, data lakes, and cloud services. They will also be able to handle different types of data, such as structured, unstructured, streaming, and batch data. They will use standard communication and integration protocols like MQTT and Kafka, APIs, and other connectors to facilitate data exchange, integration, and interoperability. Newer standards will be developed for integration of asset models and templates, and these will be automatically implemented. Although being done today, next-gen historians that utilize AI and other newer technologies will be able to auto connect and integrate easily.
Better Data Storage, Compression, Governance and Orchestration
Future historians will be able to store and compress large volumes of data with high speed and efficiency. They will use hybrid storage architectures and operations that combine on-premise and cloud storage and balance performance, cost, and scalability. They will also use adaptive compression algorithms (in AI) that automate compression algorithms according to the data type, quality, process, and importance. Newer historians have focused on improving data modeling, data context, data governance and data operations.
Table of Contents
Executive Overview
Traditional Operational Historian to Next-Gen Industrial Enterprise Historian
Operational Historian/Data Platform Trends
New Architectures
Historian Data Challenges
Summary and Recommendations
ARC Advisory Group clients can view the complete report at the ARC Client Portal.
Please Contact Us if you would like to speak with the author.
Obtain more ARC In-depth Research at Market Analysis