Abstract:The development of structural health monitoring (SHM) on civil infrastructures has resulted in enormous amount of acquired data along with the pressure of data processing and data mining. Abnormalities in data can lead to serious analytical error in later assessment. Such anomalous data patterns generally account for a relatively small portion of the overall dataset, which can be easily misclassified as normal data by regular classifiers. In this paper, a novel and robust data anomaly detection framework was proposed. The core novelty in this framework is the utilization of learning active learning (LAL) and AdaBoost algorithm aiming to reduce the costly manual work of labeling and improve the classification of anomaly patterns. Furthermore, the problem of biased classification brought by imbalanced datasets has also been solved by the LAL. Wavelet packet transform was also utilized to extract features from the acceleration data. The methodologies were firstly introduced precisely in this paper followed by two study cases to verify the feasibility of the proposed framework for data anomaly detection. The first case was a dataset with the anomalies synthetically added to the acceleration time history data measured in dynamic tests of a grid structure, including five kinds of data abnormalities. Both the balanced and imbalanced datasets were studied and analyzed, where a comparative study was carried out between the LAL-AdaBoost and uncertainty sampling-based AdaBoost with the same training and testing sets. The results showed that LAL-AdaBoost outperformed in both scenarios with higher accuracies and faster convergence speed. Then, a further study was carried out using acceleration data collected from a long-span bridge. By querying only limited amount of the training set, the proposed framework could accurately detect and classify 97.95% anomaly patterns of the testing set, showing great potential for further and broader application in the field of SHM data processing.

A Novel Boosting-Based Anomaly Detection Scheme

A Mixed Intrusion Detection System Utilizing K-means and Extreme Gradient Boosting

Using Naive Bayes with AdaBoost to Enhance Network Anomaly Intrusion Detection

AdaBoost Algorithm in Trustworthy Network for Anomaly Intrusion Detection

Novel Intrusion Detection using Probabilistic Neural Network and Adaptive Boosting

AdaBoost-Based Algorithm for Network Intrusion Detection

Boosting algorithms for network intrusion detection: A comparative evaluation of Real AdaBoost, Gentle AdaBoost and Modest AdaBoost

A Novel Unsupervised Anomaly Detection Approach for Intrusion Detection System

Anomaly detection in wide area network mesh using two machine learning anomaly detection algorithms

Anomaly Detection Method Based on Clustering Undersampling and Ensemble Learning

A novel and robust data anomaly detection framework using LAL-AdaBoost for structural health monitoring

Anomaly-Based Intrusion Detection System using Supervised Learning Algorithm Artificial Neural Network and Ant Colony Optimization with Feature Selection

A Novel Fuzzy Anomaly Detection Algorithm Based on Artificial Immune System

Boosting Anomaly Detection Using Unsupervised Diverse Test-Time Augmentation

UADB: Unsupervised Anomaly Detection Booster

Optimizing Network Anomaly Detection Scheme Using Instance Selection Mechanism

A BOOSTING APPROACH FOR INTRUSION DETECTION

AnomalyNCD: Towards Novel Anomaly Class Discovery in Industrial Scenarios

Efficacy Improvement of Anomaly Detection by Using Intelligence Sharing Scheme

Research of Anomaly detection based on Dynamic Anomaly Detection Enhancement Framework

A Hybrid Approach: Utilising Kmeans Clustering and Naive Bayes for IoT Anomaly Detection