Abstract:Traffic congestion incurs long delay in travel time, which seriously affects our daily travel experiences. Exploring why traffic congestion occurs is significantly important to effectively address the problem of traffic congestion and improve user experience. Traditional approaches to mine the congestion causes depend on human efforts, which is time consuming and cost-intensive. Hence, we aim at discovering the known and unknown causes of traffic congestion in a systematic way. However, to achieve it, there are three challenges: (1) traffic congestion is affected by several factors with complex spatio-temporal relations; (2) there are a few samples of congestion data with known causes due to the limitation of human label; (3) more unknown congestion causes are unexplored since several factors contribute to traffic congestion. To address above challenges, we design a congestion cause discovery system consisting of two modules: (1) congestion feature extraction module, which extracts the important features distinguishing between different causes of congestion; and (2) congestion cause discovery module, which designs a deep semi-supervised learning based framework to discover the causes of traffic congestion with limited labeled data. Specifically, in pre-training stage, it first leverages a few labeled data as prior knowledge to pre-train the model. Then, in clustering stage, we propose two different clustering methods to discover the congestion causes. For the first clustering method, we extend the classic deep embedded clustering model to produce clusters via soft assignment. For the second one, we iteratively use k -means to group the latent features extracted from the pre-trained model, and use the cluster results as pseudo-labels to fine-tune the network. Extensive experiments show that the performance of our methods is superior to the state-of-the-art baselines, which demonstrates the effectiveness of the proposed cause discovery system. Additionally, our system is deployed and used in the practical production environment at Amap.

A Semi-Supervised Clustering Algorithm for Real Network Traffic with Concept Drift

An enhanced self-learning-based clustering scheme for real-time traffic data distribution in wireless networks

Information Entropy Based Clustering Method for Unsupervised Internet Traffic Classification

Identifying Traffic Clusters in Urban Networks Based on Graph Theory Using License Plate Recognition Data

Network Traffic Classification Algorithm Based on Spectral Clustering

Improved Clustering Algorithm Based on Modified Fuzzy C-means Applied to the Traffic

Clustering Network Traffic Using Semi-Supervised Learning

Towards a Profiling View for Unsupervised Traffic Classification by Exploring the Statistic Features and Link Patterns

Unsupervised Anomalous Traffic Detection Based on Cascading Representation and Multiple-Clustering

A network anomaly detection algorithm based on semi-supervised learning and adaptive multiclass balancing

A Neighbor-Searching Discrepancy-based Drift Detection Scheme for Learning Evolving Data

Network traffic analysis over clustering-based collective anomaly detection

A Classification and Novel Class Detection Algorithm for Concept Drift Data Stream Based on the Cohesiveness and Separation Index of Mahalanobis Distance

Traffic Classification - Towards Accurate Real Time Network Applications

Concept Drift Detection in Federated Networked Systems

Modeling and clustering network-level urban traffic status based on traffic flow assignment ratios

Network traffic classification based on federated semi-supervised learning

Novel Dynamic Multiple Classification System for Network Traffic

Online Semi-Supervised Concept Drift Detection with Density Estimation

Discovering Causes of Traffic Congestion Via Deep Transfer Clustering

ReCDA: Concept Drift Adaptation with Representation Enhancement for Network Intrusion Detection