Neural Fourier Modelling: A Highly Compact Approach to Time-Series Analysis

Minjung Kim,Yusuke Hioka,Michael Witbrock
2024-10-07
Abstract:Neural time-series analysis has traditionally focused on modeling data in the time domain, often with some approaches incorporating equivalent Fourier domain representations as auxiliary spectral features. In this work, we shift the main focus to frequency representations, modeling time-series data fully and directly in the Fourier domain. We introduce Neural Fourier Modelling (NFM), a compact yet powerful solution for time-series analysis. NFM is grounded in two key properties of the Fourier transform (FT): (i) the ability to model finite-length time series as functions in the Fourier domain, treating them as continuous-time elements in function space, and (ii) the capacity for data manipulation (such as resampling and timespan extension) within the Fourier domain. We reinterpret Fourier-domain data manipulation as frequency extrapolation and interpolation, incorporating this as a core learning mechanism in NFM, applicable across various tasks. To support flexible frequency extension with spectral priors and effective modulation of frequency representations, we propose two learning modules: Learnable Frequency Tokens (LFT) and Implicit Neural Fourier Filters (INFF). These modules enable compact and expressive modeling in the Fourier domain. Extensive experiments demonstrate that NFM achieves state-of-the-art performance on a wide range of tasks (forecasting, anomaly detection, and classification), including challenging time-series scenarios with previously unseen sampling rates at test time. Moreover, NFM is highly compact, requiring fewer than 40K parameters in each task, with time-series lengths ranging from 100 to 16K.
Machine Learning
What problem does this paper attempt to address?
### What problem does this paper attempt to solve? This paper aims to solve the key problems in time - series analysis, especially how to directly model and process time - series data in the Fourier domain. Traditional time - series analysis methods mainly focus on time - domain modeling, although some methods also combine the frequency - domain representation as an auxiliary feature. However, these methods do not fully utilize the advantages of the frequency - domain to capture the intrinsic dynamics of the time - series. Specifically, this paper proposes **Neural Fourier Modelling (NFM)**, which is a brand - new, compact and powerful time - series analysis method. The main contributions of NFM include: 1. **Model time - series completely in the frequency - domain**: By regarding the time - series as a function in the frequency - domain, NFM can process time - series data more effectively and can perform various operations (such as resampling and time - span expansion) in the frequency - domain without the need to convert back to the time - domain. 2. **Frequency interpolation and extrapolation**: NFM reinterprets the frequency - domain data operations as frequency interpolation and extrapolation and makes them the core learning mechanism. This enables the model to flexibly handle time - series data in different tasks. 3. **Introduce two learning modules**: - **Learnable Frequency Tokens (LFT)**: Used to capture effective spectral prior information and achieve flexible frequency expansion. - **Implicit Neural Fourier Filters (INFF)**: As the main processing operator, it realizes highly - expressive global convolution to learn interpolation or extrapolation in the frequency - domain. 4. **High - efficiency and compactness**: NFM performs excellently in multiple tasks (such as prediction, anomaly detection and classification), while having a very small number of parameters (less than 40,000 parameters for each task), and is suitable for resource - constrained environments. ### Summary The core problem of this paper is to explore how to use the frequency - domain representation to improve time - series analysis, especially to directly perform data modeling and operations in the frequency - domain. By proposing NFM, the author provides a novel and efficient solution, which not only improves the performance but also maintains the compactness of the model.