Abstract:For the advancements of time series classification, scrutinizing previous studies, most existing methods adopt a common learning-to-classify paradigm - a time series classifier model tries to learn the relation between sequence inputs and target label encoded by one-hot distribution. Although effective, this paradigm conceals two inherent limitations: (1) encoding target categories with one-hot distribution fails to reflect the comparability and similarity between labels, and (2) it is very difficult to learn transferable model across domains, which greatly hinder the development of universal serving paradigm. In this work, we propose InstructTime, a novel attempt to reshape time series classification as a learning-to-generate paradigm. Relying on the powerful generative capacity of the pre-trained language model, the core idea is to formulate the classification of time series as a multimodal understanding task, in which both task-specific instructions and raw time series are treated as multimodal inputs while the label information is represented by texts. To accomplish this goal, three distinct designs are developed in the InstructTime. Firstly, a time series discretization module is designed to convert continuous time series into a sequence of hard tokens to solve the inconsistency issue across modal inputs. To solve the modality representation gap issue, for one thing, we introduce an alignment projected layer before feeding the transformed token of time series into language models. For another, we highlight the necessity of auto-regressive pre-training across domains, which can facilitate the transferability of the language model and boost the generalization performance. Extensive experiments are conducted over benchmark datasets, whose results uncover the superior performance of InstructTime and the potential for a universal foundation model in time series classification.

BiTimeBERT: Extending Pre-Trained Language Representations with Bi-Temporal Information

Towards Effective Time-Aware Language Representation: Exploring Enhanced Temporal Understanding in Language Models

Extraction of temporal information from social media messages using the BERT model

Efficient Continue Training of Temporal Language Model with Structural Information

Incorporating Structured Sentences with Time-enhanced BERT for Fully-inductive Temporal Relation Prediction

Time Matters: Examine Temporal Effects on Biomedical Language Models

Pre-Trained Language Model for Temporal Knowledge Graph Completion

Text2Time: Transformer-based Article Time Period Prediction

Temporal Effects on Pre-trained Models for Language Processing Tasks

Improving Event Duration Prediction via Time-aware Pre-training

Bidirectional Generative Pre-training for Improving Healthcare Time-series Representation Learning

XLTime: A Cross-Lingual Knowledge Transfer Framework for Temporal Expression Extraction

TML: A Temporal-aware Multitask Learning Framework for Time-sensitive Question Answering.

On Using Temporal Features to Create More Accurate Human-Activity Classifiers

Contrastive Bidirectional Transformer for Temporal Representation Learning

Time-Aware Language Models as Temporal Knowledge Bases

Temporal word embedding with predictive capability

Improving Temporal Generalization of Pre-trained Language Models with Lexical Semantic Change

Classifying Temporal Relations Between Events by Deep BiLSTM

CEHR-BERT: Incorporating temporal information from structured EHR data to improve prediction tasks

Advancing Time Series Classification with Multimodal Language Modeling