Abstract:Time series modeling is a well-established problem, which often requires that methods (1) expressively represent complicated dependencies, (2) forecast long horizons, and (3) efficiently train over long sequences. State-space models (SSMs) are classical models for time series, and prior works combine SSMs with deep learning layers for efficient sequence modeling. However, we find fundamental limitations with these prior approaches, proving their SSM representations cannot express autoregressive time series processes. We thus introduce SpaceTime, a new state-space time series architecture that improves all three criteria. For expressivity, we propose a new SSM parameterization based on the companion matrix -- a canonical representation for discrete-time processes -- which enables SpaceTime's SSM layers to learn desirable autoregressive processes. For long horizon forecasting, we introduce a "closed-loop" variation of the companion SSM, which enables SpaceTime to predict many future time-steps by generating its own layer-wise inputs. For efficient training and inference, we introduce an algorithm that reduces the memory and compute of a forward pass with the companion matrix. With sequence length $\ell$ and state-space size $d$, we go from $\tilde{O}(d \ell)$ na\"ively to $\tilde{O}(d + \ell)$. In experiments, our contributions lead to state-of-the-art results on extensive and diverse benchmarks, with best or second-best AUROC on 6 / 7 ECG and speech time series classification, and best MSE on 14 / 16 Informer forecasting tasks. Furthermore, we find SpaceTime (1) fits AR($p$) processes that prior deep SSMs fail on, (2) forecasts notably more accurately on longer horizons than prior state-of-the-art, and (3) speeds up training on real-world ETTh1 data by 73% and 80% relative wall-clock time over Transformers and LSTMs.

Mechanism-Assisted Deep State Space Model for Dynamic System Identification

A Multi-Task Learning-Based Automatic Blind Identification Procedure for Operational Modal Analysis

Deep State Space Models for Nonlinear System Identification

A Confidence-Aware Machine Learning Framework for Dynamic Security Assessment

Relational State-Space Model for Stochastic Multi-Object Systems

Subspace Model Identification for Industrial Processes

Deep Switching State Space Model (DS$^3$M) for Nonlinear Time Series Forecasting with Regime Switching

Towards a theory of learning dynamics in deep state space models

Deep Direct Discriminative Decoders for High-dimensional Time-series Data Analysis

Life-cycle modeling driven by coupling competition degradation for remaining useful life prediction

Time-SSM: Simplifying and Unifying State Space Models for Time Series Forecasting

Regularization-Based Efficient Continual Learning in Deep State-Space Models

EfficientState Space Model viaFast Tensor Convolutionand Block Diagonalization

DyG-Mamba: Continuous State Space Modeling on Dynamic Graphs

Data-driven models of nonautonomous systems

[Case of thorotrast-induced canceer of the liver with epidural metastasis to the spinal cord].

Unifying Theorems for Subspace Identification and Dynamic Mode Decomposition

Effectively Modeling Time Series with Simple Discrete State Spaces

DynaNet: Neural Kalman Dynamical Model for Motion Estimation and Prediction

A scaled derivative-based DMDc method for modelling multiple-input multiple-output mechanical systems

State-space model with deep learning for functional dynamics estimation in resting-state fMRI