Maximum Mean Discrepancy on Exponential Windows for Online Change Detection

Florian Kalinke,Marco Heyden,Georg Gntuni,Edouard Fouché,Klemens Böhm

2024-09-16

Abstract:Detecting changes is of fundamental importance when analyzing data streams and has many applications, e.g., in predictive maintenance, fraud detection, or medicine. A principled approach to detect changes is to compare the distributions of observations within the stream to each other via hypothesis testing. Maximum mean discrepancy (MMD), a (semi-)metric on the space of probability distributions, provides powerful non-parametric two-sample tests on kernel-enriched domains. In particular, MMD is able to detect any disparity between distributions under mild conditions. However, classical MMD estimators suffer from a quadratic runtime complexity, which renders their direct use for change detection in data streams impractical. In this article, we propose a new change detection algorithm, called Maximum Mean Discrepancy on Exponential Windows (MMDEW), that combines the benefits of MMD with an efficient computation based on exponential windows. We prove that MMDEW enjoys polylogarithmic runtime and logarithmic memory complexity and show empirically that it outperforms the state of the art on benchmark data streams.

Machine Learning

What problem does this paper attempt to address?

### Problems the Paper Attempts to Solve

The paper aims to address the problem of detecting change points in data streams. Specifically, the authors focus on efficiently detecting distribution changes in data streams, which has significant applications in predictive maintenance, fraud detection, medicine, and other fields.

### Background and Challenges

Traditional Maximum Mean Discrepancy (MMD) methods can effectively detect distribution changes, but their computational complexity is high, especially for large data streams. The time complexity of the classic MMD estimator is \(O(m^2 + n^2)\), where \(m\) and \(n\) are the sizes of the two datasets, making it impractical to apply MMD directly in data streams.

### Solution

To address the above issues, the authors propose a new change detection algorithm called Maximum Mean Discrepancy on Exponential Windows (MMDEW). This algorithm combines the non-parametric detection capability of MMD with an efficient computation method based on exponential windows, achieving polynomial logarithmic time complexity and logarithmic memory complexity.

### Main Contributions

1. **Efficient Online Approximation**: MMDEW reduces the time complexity of each new observation to \(O(\log^2 t)\) and the memory complexity to \(O(\log t)\) through online approximation of MMD.
2. **New Data Structure**: A new data structure is introduced to approximate the quadratic time MMD in an online setting. This data structure accelerates the process by storing observation summaries and logarithmic-sized samples for each window.
3. **Experimental Validation**: Experiments on standard benchmark datasets show that MMDEW outperforms existing state-of-the-art change detection algorithms on 4 test datasets. In more challenging settings with short detection delays, MMDEW performs better on 3 out of 6 datasets.

### Method Overview

1. **Threshold Extension**: Extends the threshold of the MMD two-sample test to accommodate different sample sizes.
2. **Data Structure Design**: Designs a new data structure that enables efficient MMD computation through an exponential window slicing method.
3. **Observation Insertion**: Recursively processes each new observation, creating new windows and merging windows when necessary.
4. **MMD Calculation and Change Detection**: Computes the MMD statistic between windows and compares it to the threshold to detect change points.

### Experimental Results

Experimental results show that MMDEW performs excellently in terms of Average Run Length (ARL) and Mean Time to Detection (MTD), especially in high-dimensional data and large-scale data streams. These results indicate that MMDEW not only has superior complexity in theory but also performs well in practical applications.

### Conclusion

By combining the non-parametric detection capability of MMD with efficient online computation methods, MMDEW successfully addresses the computational bottleneck of change detection in data streams, providing a new solution for real-time change detection.