Abstract:The Multipath TCP (MPTCP) protocol has been standardized by the IETF as an extension of conventional TCP, which enables multi-homed devices to establish multiple paths for simultaneous data transmission. Congestion control is a fundamental mechanism for the design and implementation of MPTCP. Due to the diverse QoS characteristics of heterogeneous links, existing multipath congestion control mechanisms suffer from a number of performance problems such as bufferbloat, suboptimal bandwidth usage, etc. In this paper, we propose a learning-based multipath congestion control approach called <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">SmartCC</italic> to deal with the diversities of multiple communication path in heterogeneous networks. SmartCC adopts an asynchronous reinforcement learning framework to learn a set of congestion rules, which allows the sender to observe the environment and take actions to adjust the subflows’ congestion windows adaptively to fit different network situations. To deal with the problem of infinite states in high-dimensional space, we propose a hierarchical tile coding algorithm for state aggregation and a function estimation approach for <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$Q$ </tex-math></inline-formula> -learning, which can derive the optimal policy efficiently. Due to the asynchronous design of SmartCC, the processes of model training and execution are decoupled, and the learning process will not introduce extra delay and overhead on the decision making process in MPTCP congestion control. We conduct extensive experiments for performance evaluation, which show that SmartCC improves the aggregate throughput significantly and outperforms the state-of-the-art mechanisms on a variety of performance metrics.

Breaking the Inertial Thinking: Non-Blocking Multipath Congestion Control Based on the Single-Subflow Reinforcement Learning Model

A Multi-objective Reinforcement Learning Perspective on Internet Congestion Control

TRCC: Transferable Congestion Control with Reinforcement Learning

MPCC: Optimal Congestion Control for Multipath Routing in Ad Hoc Networks

SmartCC: A Reinforcement Learning Approach for Multipath TCP Congestion Control in Heterogeneous Networks

A Dynamic Packet Scheduling Method for Multipath TCP in Heterogeneous Wireless Networks

Multipath TCP Meets Reinforcement Learning: A Novel Energy-Efficient Scheduling Approach in Heterogeneous Wireless Networks

DeepCC: Multi-Agent Deep Reinforcement Learning Congestion Control for Multi-Path TCP Based on Self-Attention

Achieving Flexible and Lightweight Multipath Congestion Control Through Online Learning

When Classic Meets Intelligence: A Hybrid Multipath Congestion Control Framework

A Reinforcement Learning Approach for Multipath TCP Data Scheduling

Shared Bottleneck-Based Congestion Control and Packet Scheduling for Multipath TCP

Modeling Multi-Path Tcp Throughput With Coupled Congestion Control And Flow Control

Receive Buffer Pre-division Based Flow Control for MPTCP.

A Survey on Congestion Control and Scheduling for Multipath TCP: Machine Learning vs Classical Approaches

BBR-Based Congestion Control and Packet Scheduling for Bottleneck Fairness Considered Multipath TCP in Heterogeneous Wireless Networks

MA-CC: Cross-Layer Congestion Control via Multi-agent Reinforcement Learning

Model Predictive Congestion Control for TCP Endpoints

RAN Information-Assisted TCP Congestion Control Using Deep Reinforcement Learning with Reward Redistribution

Pacc: A Path Associativity Congestion Control and Throughput Model for Multi-Path Tcp

Adaptive subflow allocation for multipath data transmission in mobile edge networks