Abstract:This paper proposes a hierarchical approximate-factor approach to analyzing high-dimensional, large-scale heterogeneous time series data using distributed computing. The new method employs a multiple-fold dimension reduction procedure using Principal Component Analysis (PCA) and shows great promises for modeling large-scale data that cannot be stored nor analyzed by a single machine. Each computer at the basic level performs a PCA to extract common factors among the time series assigned to it and transfers those factors to one and only one node of the second level. Each 2nd-level computer collects the common factors from its subordinates and performs another PCA to select the 2nd-level common factors. This process is repeated until the central server is reached, which collects common factors from its direct subordinates and performs a final PCA to select the global common factors. The noise terms of the 2nd-level approximate factor model are the unique common factors of the 1st-level clusters. We focus on the case of 2 levels in our theoretical derivations, but the idea can easily be generalized to any finite number of hierarchies. We discuss some clustering methods when the group memberships are unknown and introduce a new diffusion index approach to forecasting. We further extend the analysis to unit-root nonstationary time series. Asymptotic properties of the proposed method are derived for the diverging dimension of the data in each computing unit and the sample size $T$. We use both simulated data and real examples to assess the performance of the proposed method in finite samples, and compare our method with the commonly used ones in the literature concerning the forecastability of extracted factors.

Clustering Time Series Utilizing A Dimension Hierarchical Decomposition Approach

A clustering algorithm for distributed time-series data

Efficient Forecasting of Large Scale Hierarchical Time Series via Multilevel Clustering

Divide-and-Conquer: A Distributed Hierarchical Factor Approach to Modeling Large-Scale Time Series Data

Semi-supervised Hierarchical Clustering Analysis for High Dimensional Data

Efficient hierarchical clustering of large high dimensional datasets.

Hashing-Based Distributed Clustering for Massive High-Dimensional Data

New Time Series Clustering Algorithm Based on DTW Metric and Local-Close-Link Graph

Clustering Time Series Based on Dependence Structure

Varying Density Spatial Clustering Based on a Hierarchical Tree

Constructing hierarchical time series through clustering: Is there an optimal way for forecasting?

Streaming Hierarchical Clustering Based on Point-Set Kernel

A Time Series Clustering Method Based on Hypergraph Partitioning

Hierarchical Clustering using Auto-encoded Compact Representation for Time-series Analysis

Tree-based Methods for Clustering Time Series Using Domain-Relevant Attributes

Paralinear Distance and Its Algorithm for Hierarchical Clustering of High-dimensional Discrete Variables

Time Series Clustering with Topological and Geometric Mixed Distance

An Hierarchical Clustering Method Based on Data Fields

DTW-C++: Fast dynamic time warping and clustering of time series data

Mining Arbitrary Shaped Clusters and Outputting a High Quality Dendrogram.

A New Efficient Medoid Based Divisive Hierarchical Clustering Method for Large-Scale Data