Abstract:Data preprocessing is a crucial part of any machine learning pipeline, and it can have a significant impact on both performance and training efficiency. This is especially evident when using deep neural networks for time series prediction and classification: real-world time series data often exhibit irregularities such as multi-modality, skewness and outliers, and the model performance can degrade rapidly if these characteristics are not adequately addressed. In this work, we propose the EDAIN (Extended Deep Adaptive Input Normalization) layer, a novel adaptive neural layer that learns how to appropriately normalize irregular time series data for a given task in an end-to-end fashion, instead of using a fixed normalization scheme. This is achieved by optimizing its unknown parameters simultaneously with the deep neural network using back-propagation. Our experiments, conducted using synthetic data, a credit default prediction dataset, and a large-scale limit order book benchmark dataset, demonstrate the superior performance of the EDAIN layer when compared to conventional normalization methods and existing adaptive time series preprocessing layers.

What problem does this paper attempt to address?

### What problems does this paper attempt to solve? This paper aims to solve the key problems in the data pre - processing stage when using deep neural networks for time - series prediction and classification. Specifically: 1. **Irregularities of real - world time - series data**: - Real - world time - series data usually has characteristics such as multimodality, skewness, and outliers. - If these irregularities are not properly handled, the model performance will decline rapidly. 2. **Limitations of traditional normalization methods**: - Traditional static normalization methods (such as Z - score normalization and Min - Max scaling) only change the position and scale of data and cannot effectively handle the above - mentioned complex characteristics. - Using these fixed normalization schemes may lead to sub - optimal results, especially in the presence of skewed distributions, heavy - tailed distributions or outliers. 3. **The need for automated and adaptive data pre - processing**: - Determining the most suitable pre - processing method usually requires a great deal of time and repeated training and performance testing. - Therefore, researchers hope to propose a new type of highly efficient automated data pre - processing method to optimize the prediction performance of neural networks, especially in the normalization of multivariate time - series data. To solve these problems, the author proposes the **EDAIN (Extended Deep Adaptive Input Normalization) layer**, which is a new adaptive neural layer that can learn how to properly normalize irregular time - series data in an end - to - end manner instead of using a fixed normalization scheme. By optimizing unknown parameters and training with back - propagation together with deep neural networks, EDAIN can more effectively handle multimodality, skewness, outliers, etc., thereby improving the performance and training efficiency of the model. ### Summary The main objective of this paper is to propose a novel, efficient automated data pre - processing method, especially for the normalization of multivariate time - series data, to optimize the prediction performance of deep neural networks and address the shortcomings of existing methods in handling complex data characteristics.

Extended Deep Adaptive Input Normalization for Preprocessing Time Series Data for Neural Networks

Deep Adaptive Input Normalization for Time Series Forecasting

GAS-Norm: Score-Driven Adaptive Normalization for Non-Stationary Time Series Forecasting in Deep Learning

Adaptive Normalization for Non-stationary Time Series Forecasting: A Temporal Slice Perspective

Time series forecasting and anomaly detection using deep learning

Time series prediction using deep echo state networks

CAAP: Class-Dependent Automatic Data Augmentation Based On Adaptive Policies For Time Series

Breaking Time Invariance: Assorted-Time Normalization for RNNs

DANA: Dimension-Adaptive Neural Architecture for Multivariate Sensor Data

Deep Recurrent Neural Networks for ECG Signal Denoising

Deep Temporal Convolution Network for Time Series Classification

Channel-Selective Normalization for Label-Shift Robust Test-Time Adaptation

Adaptive deep fusion neural network based soft sensor for industrial process

Deep Learning for Functional Data Analysis with Adaptive Basis Layers

DAAIN: Detection of Anomalous and Adversarial Input using Normalizing Flows

Encoding temporal information in deep convolution neural network

Earliness-Aware Deep Convolutional Networks for Early Time Series Classification

SIN: Selective and Interpretable Normalization for Long-Term Time Series Forecasting

Frequency Adaptive Normalization For Non-stationary Time Series Forecasting

Exponential Moving Average Normalization for Self-supervised and Semi-supervised Learning

On Mini-Batch Training with Varying Length Time Series