Unsupervised Detection of Behavioural Drifts with Dynamic Clustering and Trajectory Analysis

Bardh Prenkaj,Paola Velardi
DOI: https://doi.org/10.1109/TKDE.2023.3320184
2023-12-14
Abstract:Real-time monitoring of human behaviours, especially in e-Health applications, has been an active area of research in the past decades. On top of IoT-based sensing environments, anomaly detection algorithms have been proposed for the early detection of abnormalities. Gradual change procedures, commonly referred to as drift anomalies, have received much less attention in the literature because they represent a much more challenging scenario than sudden temporary changes (point anomalies). In this paper, we propose, for the first time, a fully unsupervised real-time drift detection algorithm named DynAmo, which can identify drift periods as they are happening. DynAmo comprises a dynamic clustering component to capture the overall trends of monitored behaviours and a trajectory generation component, which extracts features from the densest cluster centroids. Finally, we apply an ensemble of divergence tests on sliding reference and detection windows to detect drift periods in the behavioural sequence.
Machine Learning
What problem does this paper attempt to address?
### What problem does this paper attempt to solve? This paper aims to solve the problem of **real - time unsupervised detection of behavioral drifts**, especially in e - Health applications. Specifically, the authors propose a brand - new algorithm named DynAmo to identify gradual changes (i.e., drift anomalies) in behavioral data. Such gradual changes are more challenging than sudden temporary changes (point anomalies) because they usually occur slowly over a long period and are difficult to detect in a timely manner. #### Main problems and challenges: 1. **Gradual change vs. Sudden change**: - Most existing anomaly detection models mainly focus on sudden, temporary changes (point anomalies) and pay less attention to gradual changes (drift anomalies). 2. **The need for unsupervised learning**: - Behaviors and their anomalies are highly dependent on specific contexts and individual situations, so a method for anomaly detection without pre - training behavioral data is required. 3. **Discovery of potential drift periods**: - When the reference set (training set) contains abnormal behaviors, existing models may not be able to discover potential drift periods, which is inevitable in practical application scenarios. 4. **Cold - start problem**: - Many existing methods have cold - start problems when dealing with real - time data, especially performing poorly in critical scenarios such as continuous remote monitoring. 5. **Adapting to different types of drifts**: - Existing methods can usually only detect specific types of drifts (such as gradual or periodic drifts) and have difficulty dealing with sudden distribution changes. 6. **Reproducibility and replicability of experiments**: - Experiments in the original paper often lack detailed code and data - processing instructions, making it difficult to reproduce experimental results. ### Contributions of DynAmo: DynAmo solves the above problems in the following ways: 1. **Fully unsupervised drift - detection technique**: - Based on dynamic clustering and trajectory detection, independent of the knowledge of input data distribution and anomaly types. 2. **Avoiding the cold - start problem**: - There is no need to reserve part of the input data to fine - tune the model for drift detection. 3. **Adapting to multiple types of drift anomalies**: - Robust to gradual and periodic drifts. 4. **Integrating backtracking parameters**: - Use the parameter λ to consider past events in the behavior trajectory and detect potential changes within the same window. 5. **Visualization tools**: - Provide trajectory maps of dense cluster centroids to help non - professionals identify drift trends in two - dimensional space. 6. **Supporting open science**: - Publish the code and provide steps for easy reproduction of experiments. Through these improvements, DynAmo can detect behavioral drifts more accurately in real - time environments, especially in fields such as e - Health and continuous patient care.