Abstract:Accurate traffic prediction faces significant challenges, necessitating a deep understanding of both temporal and spatial cues and their complex interactions across multiple variables. Recent advancements in traffic prediction systems are primarily due to the development of complex sequence-centric models. However, existing approaches often embed multiple variables and spatial relationships at each time step, which may hinder effective variable-centric learning, ultimately leading to performance degradation in traditional traffic prediction tasks. To overcome these limitations, we introduce variable-centric and prior knowledge-centric modeling techniques. Specifically, we propose a Heterogeneous Mixture of Experts (TITAN) model for traffic flow prediction. TITAN initially consists of three experts focused on sequence-centric modeling. Then, designed a low-rank adaptive method, TITAN simultaneously enables variable-centric modeling. Furthermore, we supervise the gating process using a prior knowledge-centric modeling strategy to ensure accurate routing. Experiments on two public traffic network datasets, METR-LA and PEMS-BAY, demonstrate that TITAN effectively captures variable-centric dependencies while ensuring accurate routing. Consequently, it achieves improvements in all evaluation metrics, ranging from approximately 4.37\% to 11.53\%, compared to previous state-of-the-art (SOTA) models. The code is open at \href{<a class="link-external link-https" href="https://github.com/sqlcow/TITAN" rel="external noopener nofollow">this https URL</a>}{<a class="link-external link-https" href="https://github.com/sqlcow/TITAN" rel="external noopener nofollow">this https URL</a>}.

What problem does this paper attempt to address?

### What problems does this paper attempt to solve? This paper aims to solve several key challenges in traffic flow prediction: 1. **Spatio - temporal heterogeneity and complex variable interactions**: - Traffic data has significant spatio - temporal heterogeneity, that is, the data not only changes over time but also shows different characteristics in spatial locations. Traditional time - series prediction methods (such as support vector regression, random forest, and gradient - boosted decision trees) are difficult to effectively capture these complex spatio - temporal relationships. 2. **Limitations of existing models**: - Existing traffic prediction models mainly rely on complex sequence - centric models. These models embed multiple variables and spatial relationships at each time step, which may hinder effective variable - centric learning, resulting in performance degradation. 3. **Balance between sequence - centric and variable - centric modeling**: - Recent research shows that methods that simply combine sequence - centric and variable - centric modeling (for example, through weighted averaging) are difficult to adapt to complex real - world scenarios. Therefore, a method that can effectively perform sequence - centric and variable - centric modeling simultaneously is needed. 4. **Challenges in applying the Mixture of Experts (MoE) in spatio - temporal tasks**: - In spatio - temporal prediction tasks, traditional MoE models face the problem of sub - optimal routing in the early training stage. Especially when encountering unpredictable events, MoE has difficulty querying and retrieving appropriate information from memory, resulting in ineffective routing decisions. ### Proposed solutions To solve the above problems, the authors propose a heterogeneous mixture of experts model named TITAN. The main features of TITAN include: - **Three sequence - centric experts**: Focus on learning temporal dependencies and capturing temporal patterns. - **One variable - centric expert**: Emphasizes the learning of cross - variable relationships to ensure a more comprehensive understanding of the data. - **One prior - knowledge - centric guiding expert**: Supervises the routing process to ensure more informed decisions in uncertain situations. Through these designs, TITAN can effectively capture variable - centric dependencies in traffic flow prediction tasks while ensuring accurate routing, thus achieving significant improvements (an increase of 4.37% to 11.53%) on all evaluation metrics. ### Summary By introducing variable - centric and prior - knowledge - centric modeling techniques, TITAN overcomes the limitations of existing traffic prediction models and provides a more flexible and efficient solution for complex spatio - temporal prediction tasks.

A Time Series is Worth Five Experts: Heterogeneous Mixture of Experts for Traffic Flow Prediction

Spatial-Temporal Graph Multi-Gate Mixture-of-Expert Model for Traffic Prediction

Urban Traffic Flow Forecasting Based on Memory Time-Series Network.

TESTAM: A Time-Enhanced Spatio-Temporal Attention Model with Mixture of Experts

Navigating Spatio-Temporal Heterogeneity: A Graph Transformer Approach for Traffic Forecasting

Lane-Level Traffic Flow Prediction with Heterogeneous Data and Dynamic Graphs

Meta Graph Transformer: A Novel Framework for Spatial–Temporal Traffic Prediction

Continual Traffic Forecasting via Mixture of Experts

ST-ExpertNet: A Deep Expert Framework for Traffic Prediction

Adaptive Graph Convolutional Recurrent Network with Transformer and Whale Optimization Algorithm for Traffic Flow Prediction

Cross Space and Time: A Spatio-Temporal Unitized Model for Traffic Flow Forecasting

Toward Directed Spatiotemporal Graph: A New Idea for Heterogeneous Traffic Prediction

Causally-Aware Spatio-Temporal Multi-Graph Convolution Network for Accurate and Reliable Traffic Prediction

Residual Attention Enhanced Time-varying Multi-Factor Graph Convolutional Network for Traffic Flow Prediction

Incorporating Multivariate Auxiliary Information for Traffic Prediction on Highways

Spatial-temporal multi-factor fusion graph neural network for traffic prediction

Context-Aware Traffic Prediction Framework Based on Series Decomposition

Interpretable Cascading Mixture-of-Experts for Urban Traffic Congestion Prediction

An adaptive composite time series forecasting model for short-term traffic flow

Traffic Prediction with Transfer Learning: A Mutual Information-based Approach

Machine Learning-Based Traffic Flow Prediction and Intelligent Traffic Management