Lindorm TSDB: A Cloud-native Time-series Database for Large-scale Monitoring Systems
shen chunhui,Qianyu Ouyang,feibo li,liu zhipeng,Longcheng Zhu,Yujie Zou,Qing Su,Tianhuan Yu,Yi,jianhong hu,Cen Zheng,Bo Wen,Hanbang Zheng,Lunfan Xu,Sicheng Pan,Bin Wu,Xiao He,Ye Li,Jian Tan,Sheng Wang,Dan Pei,Wei Zhang,Feifei Li
DOI: https://doi.org/10.14778/3611540.3611559
2023-01-01
Abstract:Internet services supported by large-scale distributed systems have become essential for our daily life. To ensure the stability and high quality of services, diverse metric data are constantly collected and managed in a time-series database to monitor the service status. However, when the number of metrics becomes massive, existing time-series databases are inefficient in handling high-rate data ingestion and queries hitting multiple metrics. Besides, they all lack the support of machine learning functions, which are crucial for sophisticated analysis of large-scale time series. In this paper, we present Lindorm TSDB, a distributed time-series database designed for handling monitoring metrics at scale. It sustains high write throughput and low query latency with massive active metrics. It also allows users to analyze data with anomaly detection and time series forecasting algorithms directly through SQL. Furthermore, Lindorm TSDB retains stable performance even during node scaling. We evaluate Lindorm TSDB under different data scales, and the results show that it outperforms two popular open-source time-series databases on both writing and query, while executing time-series machine learning tasks efficiently.