Abstract:Dataset condensation is a newborn technique that generates a small dataset that can be used in training deep neural networks to lower training costs. The objective of dataset condensation is to ensure that the model trained with the synthetic dataset can perform comparably to the model trained with full datasets. However, existing methods predominantly concentrate on classification tasks, posing challenges in their adaptation to time series forecasting (TS-forecasting). This challenge arises from disparities in the evaluation of synthetic data. In classification, the synthetic data is considered well-distilled if the model trained with the full dataset and the model trained with the synthetic dataset yield identical labels for the same input, regardless of variations in output logits distribution. Conversely, in TS-forecasting, the effectiveness of synthetic data distillation is determined by the distance between predictions of the two models. The synthetic data is deemed well-distilled only when all data points within the predictions are similar. Consequently, TS-forecasting has a more rigorous evaluation methodology compared to classification. To mitigate this gap, we theoretically analyze the optimization objective of dataset condensation for TS-forecasting and propose a new one-line plugin of dataset condensation designated as Dataset Condensation for Time Series Forecasting (CondTSF) based on our analysis. Plugging CondTSF into previous dataset condensation methods facilitates a reduction in the distance between the predictions of the model trained with the full dataset and the model trained with the synthetic dataset, thereby enhancing performance. We conduct extensive experiments on eight commonly used time series datasets. CondTSF consistently improves the performance of all previous dataset condensation methods across all datasets, particularly at low condensing ratios.

Advantages and Pitfalls of Dataset Condensation: An Approach to Keyword Spotting with Time-Frequency Representations

Dataset Condensation via Efficient Synthetic-Data Parameterization

Condense: A Framework for Device and Frequency Adaptive Neural Network Models on the Edge.

Elucidating the Design Space of Dataset Condensation

Dataset Condensation for Time Series Classification via Dual Domain Matching

Dataset Condensation with Distribution Matching

Less is More: Efficient Time Series Dataset Condensation via Two-fold Modal Matching--Extended Version

Dataset Condensation with Gradient Matching

Koopcon: A new approach towards smarter and less complex learning

Leveraging Hierarchical Feature Sharing for Efficient Dataset Condensation

Multisize Dataset Condensation

CAFE: Learning to Condense Dataset by Aligning Features

Calibrated Dataset Condensation for Faster Hyperparameter Search

CondTSF: One-line Plugin of Dataset Condensation for Time Series Forecasting

You Only Condense Once: Two Rules for Pruning Condensed Datasets

Decomposed Distribution Matching in Dataset Condensation

Dataset Condensation with Differentiable Siamese Augmentation

Towards Model-Agnostic Dataset Condensation by Heterogeneous Models

Subdomain contraction in deep networks for robust representation learning

An Improved Condensing Algorithm