Abstract:Columnar storage is now an industry standard design in most open-source or commercial time series database products, making them HTAP systems. The time column of a time series serves as the key for identifying the other value column, namely single-column storage scheme. When multiple time series share a similar set of timestamps, very likely in a module of multiple sensors, it is natural to group them together, i.e., one time column identifies multiple value columns in a single-group storage scheme. While multiple value columns sharing the same time column reduce the space cost of repeating timestamps, it may introduce extra space cost for recording null values. The reason is that time series may not be exactly aligned on each timestamp, owing to missing values, distinct data collection frequencies, unsynchronized clocks and so on. The columngroups storage scheme is thus to divide columns into multiple groups, within which the value columns share the same time column. Unfortunately, the problem of finding the optimal column groups for the minimum space cost is highly challenging, NP-hard according to our analysis. Thereby, we propose a heuristic algorithm for automatically grouping time series for efficient columnar storage. The column groups storage has been deployed in Apache IoTDB, an open-source time series database. The extensive performance analysis, over real-world data from our industrial partners, demonstrates that the proposed column groups achieve near optimal storage, more concise than the storage of single-column or single-group schemes. Interestingly, both the flushing and querying time costs of column groups are comparable to those of single-column or singlegroup, i.e., without incurring extra time cost.

TVStore: Automatically Bounding Time Series Storage via Time-Varying Compression

TSCache

MOST: Model-Based Compression with Outlier Storage for Time Series Data

Time Series Data Encoding for Efficient Storage

A Dedicated Storage Solution of Network Video Surveillance System

VStore: A Data Store for Analytics on Large Videos

Spatiotemporally adaptive compression for scientific dataset with feature preservation -- a case study on simulation data with extreme climate events analysis

Time-tired compaction: An elastic compaction scheme for LSM-tree based time-series database

Impression Store: Compressive Sensing-based Storage for Big Data Analytics.

Time-universal data compression and prediction

Mtsc: An Effective Multiple Time Series Compressing Approach

Grouping Time Series for Efficient Columnar Storage.

Time series data encoding in Apache IoTDB: comparative analysis and recommendation

Dynamic Compression Strategy for Time Series Database Using GPU

TgStore: An Efficient Storage System for Large Time-Evolving Graphs

A Fast Lightweight Time-Series Store for IoT Data

Study of Seamless Organization and Storage Structure for Massive Spatio-Temporal Data

A High Performance Compression Method For Climate

Segmentation of Time Series Based on Kinetic Characteristics for Storage Consumption Prediction

Change a Bit to save Bytes: Compression for Floating Point Time-Series Data

Multi-Dimensional Regression Analysis of Time-Series Data Streams.