Abstract:The Multipath TCP (MPTCP) protocol has been standardized by the IETF as an extension of conventional TCP, which enables multi-homed devices to establish multiple paths for simultaneous data transmission. Congestion control is a fundamental mechanism for the design and implementation of MPTCP. Due to the diverse QoS characteristics of heterogeneous links, existing multipath congestion control mechanisms suffer from a number of performance problems such as bufferbloat, suboptimal bandwidth usage, etc. In this paper, we propose a learning-based multipath congestion control approach called <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">SmartCC</italic> to deal with the diversities of multiple communication path in heterogeneous networks. SmartCC adopts an asynchronous reinforcement learning framework to learn a set of congestion rules, which allows the sender to observe the environment and take actions to adjust the subflows’ congestion windows adaptively to fit different network situations. To deal with the problem of infinite states in high-dimensional space, we propose a hierarchical tile coding algorithm for state aggregation and a function estimation approach for <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$Q$ </tex-math></inline-formula> -learning, which can derive the optimal policy efficiently. Due to the asynchronous design of SmartCC, the processes of model training and execution are decoupled, and the learning process will not introduce extra delay and overhead on the decision making process in MPTCP congestion control. We conduct extensive experiments for performance evaluation, which show that SmartCC improves the aggregate throughput significantly and outperforms the state-of-the-art mechanisms on a variety of performance metrics.

When Classic Meets Intelligence: A Hybrid Multipath Congestion Control Framework

MPLibra: Complementing the Benefits of Classic and Learning-based Multipath Congestion Control

A Multi-objective Reinforcement Learning Perspective on Internet Congestion Control

A Dynamic Packet Scheduling Method for Multipath TCP in Heterogeneous Wireless Networks

MPCC: Optimal Congestion Control for Multipath Routing in Ad Hoc Networks

SmartCC: A Reinforcement Learning Approach for Multipath TCP Congestion Control in Heterogeneous Networks

Achieving Flexible and Lightweight Multipath Congestion Control Through Online Learning

Shared Bottleneck-Based Congestion Control and Packet Scheduling for Multipath TCP

Pacc: A Path Associativity Congestion Control and Throughput Model for Multi-Path Tcp

A Unified Congestion Control Framework for Diverse Application Preferences and Network Conditions.

Modeling Multi-Path Tcp Throughput With Coupled Congestion Control And Flow Control

Breaking the Inertial Thinking: Non-Blocking Multipath Congestion Control Based on the Single-Subflow Reinforcement Learning Model

BBR-Based Congestion Control and Packet Scheduling for Bottleneck Fairness Considered Multipath TCP in Heterogeneous Wireless Networks

Leveraging Coupled BBR and Adaptive Packet Scheduling to Boost MPTCP

Multipath TCP Meets Reinforcement Learning: A Novel Energy-Efficient Scheduling Approach in Heterogeneous Wireless Networks

Low Priority Congestion Control for Multipath TCP

Transparent Multipath: Using Double MPTCP Proxies to Enhance Transport Performance for Traditional TCP

A Survey on Congestion Control and Scheduling for Multipath TCP: Machine Learning vs Classical Approaches

Wbbr: A Bottleneck Estimation-Based Congestion Control for Multipath TCP

Survey of MPTCP-Based Multipath Transmission Optimization

Improve MPTCP with SDN: from the Perspective of Resource Pooling