Abstract:For the advancements of time series classification, scrutinizing previous studies, most existing methods adopt a common learning-to-classify paradigm - a time series classifier model tries to learn the relation between sequence inputs and target label encoded by one-hot distribution. Although effective, this paradigm conceals two inherent limitations: (1) encoding target categories with one-hot distribution fails to reflect the comparability and similarity between labels, and (2) it is very difficult to learn transferable model across domains, which greatly hinder the development of universal serving paradigm. In this work, we propose InstructTime, a novel attempt to reshape time series classification as a learning-to-generate paradigm. Relying on the powerful generative capacity of the pre-trained language model, the core idea is to formulate the classification of time series as a multimodal understanding task, in which both task-specific instructions and raw time series are treated as multimodal inputs while the label information is represented by texts. To accomplish this goal, three distinct designs are developed in the InstructTime. Firstly, a time series discretization module is designed to convert continuous time series into a sequence of hard tokens to solve the inconsistency issue across modal inputs. To solve the modality representation gap issue, for one thing, we introduce an alignment projected layer before feeding the transformed token of time series into language models. For another, we highlight the necessity of auto-regressive pre-training across domains, which can facilitate the transferability of the language model and boost the generalization performance. Extensive experiments are conducted over benchmark datasets, whose results uncover the superior performance of InstructTime and the potential for a universal foundation model in time series classification.

What problem does this paper attempt to address?

The paper aims to address several key issues in the task of Time Series Classification (TSC): 1. **Difficulty in Cross-Domain Knowledge Transfer**: Existing time series classification methods struggle to achieve effective transfer learning across datasets from different domains. The main reason is the differences in aspects such as the number of channels between datasets from different domains, which hinders the direct application of models in various scenarios. 2. **Limitations of Label Representation**: Most current methods use one-hot encoding to represent target labels. This representation ignores the similarity and comparability between categories, resulting in the model's inability to fully utilize this information to generalize and improve accuracy. 3. **Effective Utilization of Auxiliary Information**: Existing models find it challenging to effectively integrate rich side information (such as the age and gender of patients), which is crucial in certain application scenarios (e.g., medical diagnosis). To address the above challenges, the paper proposes a new method called InstructTime, which redefines the time series classification task as a multimodal understanding problem by combining time series data and language descriptions to generate label text. In this way, the paper attempts to solve the issues present in traditional methods and improve the model's performance in various application scenarios.

Advancing Time Series Classification with Multimodal Language Modeling

Modality-invariant Temporal Representation Learning for Multimodal Sentiment Classification

Hierarchical Multimodal LLMs with Semantic Space Alignment for Enhanced Time Series Classification

Multi-view Self-Supervised Contrastive Learning for Multivariate Time Series

DualTime: A Dual-Adapter Multimodal Language Model for Time Series Representation

Integrating Data-Driven Segmentation, Local Feature Extraction and Fisher Kernel Encoding to Improve Time Series Classification

ChatTime: A Unified Multimodal Time Series Foundation Model Bridging Numerical and Textual Data

A Deep Multi-Task Representation Learning Method for Time Series Classification and Retrieval.

TimeCMA: Towards LLM-Empowered Multivariate Time Series Forecasting via Cross-Modality Alignment

TableTime: Reformulating Time Series Classification as Zero-Shot Table Understanding via Large Language Models

Towards the Generalization of Time Series Classification: A Feature-Level Style Transfer and Multi-Source Transfer Learning Perspective

ChatTS: Aligning Time Series with LLMs via Synthetic Data for Enhanced Understanding and Reasoning

Exploiting Multi-Channels Deep Convolutional Neural Networks for Multivariate Time Series Classification

Time-frequency deep metric learning for multivariate time series classification

Multiobjective Learning in the Model Space for Time Series Classification

Towards Effective Time-Aware Language Representation: Exploring Enhanced Temporal Understanding in Language Models

Towards the Generalization of Time Series Classification

Unsupervised Multimodal Domain Adversarial Network for Time Series Classification.

TV-Net: Temporal-Variable feature harmonizing Network for multivariate time series classification and interpretation

Time-MMD: Multi-Domain Multimodal Dataset for Time Series Analysis

TimeChat: A Time-sensitive Multimodal Large Language Model for Long Video Understanding