How Data Historian Stores Time-Series Data

Data Historian uses a highly optimized and efficient approach to storing time-series data, balancing performance, scalability, and long-term data retention.

The following table describes how the system typically stores data files.

Feature	Description
Time-series data storage	Proprietary binary file format: Stores data in a proprietary, compressed binary file format optimized for time-series data. This format allows high-speed data writes and reads, with a focus on minimizing storage space without sacrificing retrieval performance. High-resolution data: Stores data points with high precision, including time stamps and values. These data points can come from real-time sources, such as PLCs, SCADA systems, sensors, and IoT devices. Tag-based storage: Stores data in a tag-based structure, where each tag represents a data point or variable from the industrial process. The tags are organized into logical groups for efficient storage and retrieval.
Compression mechanisms	Lossless compression: Uses advanced lossless compression algorithms to reduce the size of data files without losing any information. Lossless compression ensures that every data point can be accurately retrieved later. Dynamic compression: Applies different compression techniques based on the type of data and its frequency, allowing it to optimize storage for different use cases, such as high-frequency data from sensors vs. lower-frequency data from other devices.
Tiered storage architecture	Hot storage: Stores recent or high-priority data in high-speed hot storage, typically located on fast disks like solid-state drives (SSDs). This storage tier is optimized for rapid data retrieval and for writes, ensuring low-latency access to the most current data. Cold storage (historical data): Moves older data, which is accessed less frequently, to cold storage. Cold storage typically resides on slower, less expensive disks (such as hard disk drives (HDDs)) or long-term storage solutions. This approach reduces the cost of storing large volumes of historical data while maintaining access to it. Archival storage: Archives very old, rarely accessed data that needs to be retained to long-term storage, such as tape drives or cloud storage like Microsoft Azure Blob Storage.