Abstract:Safety is a top priority for civil aviation. Data mining in digital Flight Data Recorder (FDR) or Quick Access Recorder (QAR) data, commonly referred to as black box data on aircraft, has gained interest for proactive safety management. New anomaly detection methods, primarily clustering methods, have been developed to monitor pilot operations and detect any risks from such flight data. However, all existing anomaly detection methods are offline learning — the models are trained once using historical data and used for all future predictions. In practice, new flight data are accumulated continuously and analyzed every month at airlines. Clustering such dynamically growing data is challenging for an offline method because it is memory and time intensive to re-train the model every time new data come in. If the model is not re-trained, false alarms or missed detections may increase since the model cannot reflect changes in data patterns. To address this problem, we propose a novel incremental anomaly detection method based on Gaussian Mixture Model (GMM) to identify common patterns and detect outliers in flight operations from digital flight data. It is a probabilistic clustering model of flight operations that can incrementally update its clusters based on new data rather than to re-cluster all data from scratch. It trains an initial GMM model based on historical offline data. Then, it continuously adapts to new incoming data points via an expectation–maximization (EM) algorithm. To track changes in flight operation patterns, only model parameters need to be saved, not the raw flight data. The proposed method was tested on three sets of simulation data and two sets of real-world flight data. Compared with the traditional offline GMM method, the proposed method can generate similar clustering results with significantly reduced processing time (57 %–99 % time reduction in testing sets) and memory usage (91 %–95 % memory usage reduction in testing sets). Preliminary results indicate that the incremental learning scheme is effective in dealing with dynamically growing data in flight data analytics.

Railway Fault Text Clustering Method Using an Improved Dirichlet Multinomial Mixture Model

A Robust Infinite Gaussian Mixture Model and Its Application in Fault Detection of Nonlinear Multimode Processes

An Adaptive Dirichlet Multinomial Mixture Model for Short Text Streaming Clustering

X-DMM: Fast and Scalable Model Based Text Clustering

Fault Diagnosis Method for Railway Signal Equipment Based on Data Enhancement and an Improved Attention Mechanism

A Novel Sequence Discriminative Feature Extraction Network and Its Application in Offline Industrial Fault Pattern Clustering

Analyzing Risk of Service Failures in Heavy Haul Rail Lines: A Hybrid Approach for Imbalanced Data

Fault Diagnosis of Signal Equipment on the Lanzhou-Xinjiang High-Speed Railway Using Machine Learning for Natural Language Processing

Wear diagnosis for rail profile data using a novel multidimensional scaling clustering method

An Enhanced Fault Detection Method for Railway Turnouts Incorporating Prior Faulty Information

Semi-supervised fault diagnosis of wheelset bearings in high-speed trains using autocorrelation and improved flow Gaussian mixture model

M-CLUSTER: multistage clustering for unsupervised train wheel condition monitoring

A Clustering-based Framework for Highly Imbalanced Fault Detection with the Applications on High-Speed Trains

Rail Steel Health Analysis Based on a Novel Genetic Density-based Clustering Technique and Manifold Representation of Acoustic Emission Signals.

A New Method for Handling Unstructured Data in the High-Speed Railway Passenger Service System

Railway dangerous goods transportation system risk analysis: An Interpretive Structural Modeling and Bayesian Network combining approach

An incremental clustering method for anomaly detection in flight data

Bi-TLLDA and CSSVM based fault diagnosis of vehicle on-board equipment for high speed railway

An Online Classification Method for Fault Diagnosis of Railway Turnouts

Model-based clustering and segmentation of time series with changes in regime

Tensor Dirichlet Process Multinomial Mixture Model for Passenger Trajectory Clustering