Abstract:In recent years, two competitive time series classification models, namely, ROCKET and MINIROCKET, have garnered considerable attention due to their low training cost and high accuracy. However, they rely on a large number of random 1-D convolutional kernels to comprehensively capture features, which is incompatible with resource-constrained devices. Despite the development of heuristic algorithms designed to recognize and prune redundant kernels, the inherent time-consuming nature of evolutionary algorithms hinders efficient evaluation. To efficiently prune models, this paper eliminates feature groups contributing minimally to the classifier, thereby discarding the associated random kernels without direct evaluation. To this end, we incorporate both group-level ($l_{2,1}$-norm) and element-level ($l_2$-norm) regularizations to the classifier, formulating the pruning challenge as a group elastic net classification problem. An ADMM-based algorithm is initially introduced to solve the problem, but it is computationally intensive. Building on the ADMM-based algorithm, we then propose our core algorithm, POCKET, which significantly speeds up the process by dividing the task into two sequential stages. In Stage 1, POCKET utilizes dynamically varying penalties to efficiently achieve group sparsity within the classifier, removing features associated with zero weights and their corresponding kernels. In Stage 2, the remaining kernels and features are used to refit a $l_2$-regularized classifier for enhanced performance. Experimental results on diverse time series datasets show that POCKET prunes up to 60% of kernels without a significant reduction in accuracy and performs 11$\times$ faster than its counterparts. Our code is publicly available at <a class="link-external link-https" href="https://github.com/ShaowuChen/POCKET" rel="external noopener nofollow">this https URL</a>.

What problem does this paper attempt to address?

The paper primarily addresses the issues present in time series classification (TSC) models based on random convolutional kernels and proposes solutions. Specifically, the paper focuses on the following problems: ### Background and Challenges - **ROCKET and MINIROCKET Models**: These two models have garnered widespread attention due to their low training costs and high accuracy. They extract features using a large number of 1-D random convolutional kernels, which are then used to train a linear classifier. - **Resource Limitation Issue**: Since these models require a large number of random convolutional kernels to comprehensively capture features, this leads to significant time and memory overhead when deploying on resource-constrained devices. - **Feature Redundancy**: There is significant redundancy among the randomly generated convolutional kernels, meaning many kernels have minimal impact on the classification results. ### Paper's Solutions - **Objective**: The paper aims to identify and eliminate these redundant random convolutional kernels to improve the efficiency of ROCKET and MINIROCKET models while maintaining classification performance. - **Method**: The paper proposes a new method called POCKET, which efficiently prunes random convolutional kernels from the perspective of feature selection. Specifically, this method indirectly eliminates the corresponding random convolutional kernels by pruning the feature groups that contribute the least to the classification in the classification layer. ### Main Contributions 1. **Innovative Perspective**: Unlike existing methods that directly evaluate or modify random convolutional kernels, POCKET proposes a new approach to efficiently prune ROCKET-based models through feature selection. 2. **Algorithm Design**: The pruning task is modeled as a group elastic net classification problem, and two algorithms are proposed to solve this problem. The POCKET algorithm, in particular, excels in efficiency, being 11 times faster than other methods, and allows users to adjust the pruning rate based on computational budget. 3. **Experimental Validation**: Extensive experiments on multiple datasets validate the effectiveness and performance of the proposed algorithm. The results show that even when more than 60% of the convolutional kernels are pruned, the accuracy of the ROCKET model does not significantly decrease and even slightly improves. In summary, the paper addresses the issues of resource consumption and redundancy in time series classification models based on random convolutional kernels and proposes an efficient pruning method, POCKET, which not only improves model efficiency but also maintains classification performance.

POCKET: Pruning Random Convolution Kernels for Time Series Classification from a Feature Selection Perspective

Detach-ROCKET: Sequential feature selection for time series classification with random convolutional kernels

ROCKET: exceptionally fast and accurate time series classification using random convolutional kernels

MiniRocket: A Very Fast (Almost) Deterministic Transform for Time Series Classification

MultiRocket: multiple pooling operators and transformations for fast and effective time series classification

Accelerated multi-kernel sparse stochastic optimization classifier algorithm for explainable prediction

Prognostic classification based on random convolutional kernel

LLM-based Knowledge Pruning for Time Series Data Analytics on Edge-computing Devices

Time series classification with random convolution kernels based transforms: pooling operators and input representations matter

Adacket: ADAptive Convolutional KErnel Transform for Multivariate Time Series Classification

Random Convolution Kernels with Multi-Scale Decomposition for Preterm EEG Inter-burst Detection

Intermediate-grained kernel elements pruning with structured sparsity

DRoP: Distributionally Robust Pruning

CRISP: Hybrid Structured Sparsity for Class-aware Model Pruning

Dynamic Probabilistic Pruning: A General Framework for Hardware-Constrained Pruning at Different Granularities

Two-stage sparse multi-kernel optimization classifier method for more accurate and explainable prediction

CUP: Cluster Pruning for Compressing Deep Neural Networks

Discrimination-aware Network Pruning for Deep Model Compression

HESSO: Towards Automatic Efficient and User Friendly Any Neural Network Training and Pruning

CATRO: Channel Pruning via Class-Aware Trace Ratio Optimization