Multitask Learning for Time Series Data with 2D Convolution

Chin-Chia Michael Yeh,Xin Dai,Yan Zheng,Junpeng Wang,Huiyuan Chen,Yujie Fan,Audrey Der,Zhongfang Zhuang,Liang Wang,Wei Zhang

2023-10-10

Abstract:Multitask learning (MTL) aims to develop a unified model that can handle a set of closely related tasks simultaneously. By optimizing the model across multiple tasks, MTL generally surpasses its non-MTL counterparts in terms of generalizability. Although MTL has been extensively researched in various domains such as computer vision, natural language processing, and recommendation systems, its application to time series data has received limited attention. In this paper, we investigate the application of MTL to the time series classification (TSC) problem. However, when we integrate the state-of-the-art 1D convolution-based TSC model with MTL, the performance of the TSC model actually deteriorates. By comparing the 1D convolution-based models with the Dynamic Time Warping (DTW) distance function, it appears that the underwhelming results stem from the limited expressive power of the 1D convolutional layers. To overcome this challenge, we propose a novel design for a 2D convolution-based model that enhances the model's expressiveness. Leveraging this advantage, our proposed method outperforms competing approaches on both the UCR Archive and an industrial transaction TSC dataset.

Machine Learning,Artificial Intelligence

What problem does this paper attempt to address?

The paper mainly aims to address the following issues:

1. **Application of Multi-Task Learning (MTL) in Time Series Classification (TSC)**: The paper explores how to apply MTL to the problem of time series classification of time series data. Although MTL has achieved significant success in fields such as computer vision and natural language processing, its application to time series data is relatively rare.

2. **Poor performance of existing models in MTL**: When combining 1D convolution-based TSC models (such as ResNet1D) with MTL, it was found that their performance actually decreased. This indicates that the expressive capability of 1D convolution layers is limited and insufficient to capture certain useful features.

3. **Proposing a new 2D convolution model**: To overcome the above challenges, the paper proposes a new 2D convolution-based time series classification model (ResNet2D) to enhance the model's expressive capability. Experimental results show that this model outperforms other methods on both the UCR archive and industrial transaction TSC datasets.

In summary, the paper aims to improve the application of multi-task learning in time series classification and enhance overall performance by introducing a new 2D convolution model.