Abstract:We propose a simple but strong baseline for time series classification from scratch with deep neural networks. Our proposed baseline models are pure end-to-end without any heavy preprocessing on the raw data or feature crafting. The proposed Fully Convolutional Network (FCN) achieves premium performance to other state-of-the-art approaches and our exploration of the very deep neural networks with the ResNet structure is also competitive. The global average pooling in our convolutional model enables the exploitation of the Class Activation Map (CAM) to find out the contributing region in the raw data for the specific labels. Our models provides a simple choice for the real world application and a good starting point for the future research. An overall analysis is provided to discuss the generalization capability of our models, learned features, network structures and the classification semantics.
What problem does this paper attempt to address?
### Problems the Paper Aims to Solve
This paper aims to address the problem of time series classification using deep neural networks from scratch. Specifically, the authors propose a simple yet powerful baseline model for end-to-end classification of raw time series data without the need for complex data preprocessing or feature engineering.
### Background and Motivation
Time series data is prevalent in many fields, such as weather records, financial data, physiological signals, and industrial observations. The study of representation learning and classification of time series has broad application potential in finance, industry, and healthcare. However, learning representations and classification of time series remains a research hotspot.
Traditional distance-based methods (such as Euclidean distance or dynamic time warping) and feature-based methods (such as extracting features of global/local time series patterns) require extensive data preprocessing and feature engineering. In recent years, some studies have attempted to use deep neural networks, particularly convolutional neural networks (CNNs), for end-to-end time series classification, but these methods often require complex preprocessing steps and extensive hyperparameter tuning.
### Proposed Methods
The authors propose three deep neural network architectures as baseline models:
1. **Multilayer Perceptron (MLP)**: A basic MLP model is constructed by stacking three fully connected layers, each containing 500 neurons, and using Dropout and ReLU activation functions to improve generalization.
2. **Fully Convolutional Network (FCN)**: An FCN model is constructed by stacking three convolutional blocks, each containing a convolutional layer, batch normalization layer, and ReLU activation layer. The final output is obtained through a global average pooling layer and a softmax layer.
3. **Residual Network (ResNet)**: A ResNet model is constructed by stacking three residual blocks, each containing multiple convolutional layers, and using skip connections to facilitate gradient flow. The final output is also obtained through a global average pooling layer and a softmax layer.
### Experimental Results
The authors tested these three models on the UCR time series dataset and compared them with existing best benchmark methods. The experimental results show that FCN and ResNet achieve comparable or better performance than the existing best methods on most datasets. In particular, FCN performs best on multiple metrics, while ResNet also excels in mean per-class error (MPCE) and ranking.
### Main Contributions
1. **End-to-End Baseline Model**: Proposed an end-to-end time series classification model that does not require complex data preprocessing and feature engineering.
2. **Superior Performance**: FCN and ResNet achieve comparable or better performance than existing best methods on multiple datasets.
3. **Interpretability**: Through global average pooling layer and class activation maps (CAM), the model can identify regions contributing to specific labels, enhancing model interpretability.
4. **Evaluation Metric**: Proposed a new evaluation metric—mean per-class error (MPCE)—to more fairly assess the performance of multiple models across multiple datasets.
### Conclusion
This paper provides a simple yet powerful baseline model for time series classification using deep neural networks from scratch. Through end-to-end training, FCN and ResNet achieve excellent performance on multiple datasets, and the interpretability of the models is enhanced. These results provide a good starting point for further research.