Abstract:The Active Traffic Management (ATM) system has been widely used in the United States and the European countries to improve the traffic safety of urban expressways. The accurate real-time crash risk prediction is fundamental to the system running well. Crash data are characterized by small probability, which poses a typical Imbalanced Data Classification problem. Most previous studies mainly improved the prediction methods only in data level or algorithm level, which may be inadequate to predict the crash risk accurately especially in a continuous real-time traffic data environment. The comprehensive imbalanced classification algorithm was examined in this research to build more accurate real-time traffic crash risk prediction model. At the output level, the Youden index method has been proved to be of the best ability to divide the prediction results and Probability Calibration Method was proposed to optimize the prediction results in further. At the data level, Under-sampling and Synthetic Minority Oversampling Technique(SMOTE) methods were compared to solve the imbalanced data classification problem by changing the data distribution. At the algorithm level, the cost-sensitive MLP algorithm and Adaboost algorithm were examined and finally the random sampling cost-sensitive MLP model(RCSMLP) and Rusboost model were constructed by synthesizing the optimization methods from three levels. The sensitivity of the RCSMLP model reached 78.10 % and the specificity of the model reached 81.44 %. The AUC and sensitivity of the Rusboost model reached 0.892 and 0.842 while the specificity of the model reached 0.816, which shows the better performance in dealing with the imbalanced traffic crash risk prediction problem compared to existed prediction models. The proposed method of improving prediction accuracy in this study is universal and can be applied to many other prediction models to predict real-time traffic crash risk.

BalancedBoost: A hybrid approach for real-time network traffic classification

Imbalanced Data Sets Classification Method Based on Over-Sampling Technique

Imbalanced Data Classification Algorithm Based on Integrated Sampling and Ensemble Learning.

WOTBoost: Weighted Oversampling Technique in Boosting for imbalanced learning

A Classfication Method For Imbalance Data Set Based on Kernel SMOTE

Hybrid Firefly Optimised Ensemble Classification for Drifting Data Streams with Imbalance

Hybrid approach with Membership-density based oversampling for handling multi-class imbalance in internet traffic identification with overlapping and noise

A weighted hybrid ensemble method for classifying imbalanced data

A cluster-based SMOTE both-sampling (CSBBoost) ensemble algorithm for classifying imbalanced data

Hybrid SVM algorithm oriented to classifying imbalanced datasets

Internet traffic prediction by w-boost: classification and regression

Examining imbalanced classification algorithms in predicting real-time traffic crash risk

Group & Reweight: A Novel Cost-Sensitive Approach to Mitigating Class Imbalance in Network Traffic Classification

Imbalanced Data Classification Based on Improved Random-SMOTE and Feature Standard Deviation

Research and application of XGBoost in imbalanced data

A Method to Boost Naïve Bayesian Classifiers

A network anomaly detection algorithm based on semi-supervised learning and adaptive multiclass balancing

CUSBoost: Cluster-based Under-sampling with Boosting for Imbalanced Classification

Fast and Robust Online Traffic Classification Supporting Unseen Applications

Over-sampling Algorithm Based on Adaboost in Unbalanced Data Set

To Balance or Not to Balance: A Simple-yet-Effective Approach for Learning with Long-Tailed Distributions