Abstract:The Multipath TCP (MPTCP) protocol has been standardized by the IETF as an extension of conventional TCP, which enables multi-homed devices to establish multiple paths for simultaneous data transmission. Congestion control is a fundamental mechanism for the design and implementation of MPTCP. Due to the diverse QoS characteristics of heterogeneous links, existing multipath congestion control mechanisms suffer from a number of performance problems such as bufferbloat, suboptimal bandwidth usage, etc. In this paper, we propose a learning-based multipath congestion control approach called <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">SmartCC</italic> to deal with the diversities of multiple communication path in heterogeneous networks. SmartCC adopts an asynchronous reinforcement learning framework to learn a set of congestion rules, which allows the sender to observe the environment and take actions to adjust the subflows’ congestion windows adaptively to fit different network situations. To deal with the problem of infinite states in high-dimensional space, we propose a hierarchical tile coding algorithm for state aggregation and a function estimation approach for <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$Q$ </tex-math></inline-formula> -learning, which can derive the optimal policy efficiently. Due to the asynchronous design of SmartCC, the processes of model training and execution are decoupled, and the learning process will not introduce extra delay and overhead on the decision making process in MPTCP congestion control. We conduct extensive experiments for performance evaluation, which show that SmartCC improves the aggregate throughput significantly and outperforms the state-of-the-art mechanisms on a variety of performance metrics.

Multi-Channel Scatter (MCS): Traffic Balancing Based on Edge-Switching in Datacenter Networks

Channel-Aware Rate Adaptation for Backscatter Networks

VMS: Traffic Balancing Based on Virtual Switches in Datacenter Networks.

Elastically Scaling Control Channels in Network Measurement with $\Mathsf{escala}$

CSMA/CA-based Medium Access Control for Indoor Millimeter Wave Networks.

VMS: Load Balancing Based on the Virtual Switch Layer in Datacenter Networks

Escala: Timely Elastic Scaling of Control Channels in Network Measurement

Traffic Splitting over ECMPs Using ACC Scheme

Low-Cost Datacenter Load Balancing With Multipath Transport and Top-of-Rack Switches

DCMPTCP: Host-Based Load Balancing for Datacenters

Dual-Nat: Dynamic Multipath Flow Scheduling for Data Center Networks

Responsive multipath TCP in SDN-based datacenters

MultiCS: Circuit Switched NoC with Multiple Sub-Networks and Sub-Channels.

Explicit Multipath Congestion Control for Data Center Networks

MTSS: multi-path traffic scheduling mechanism based on SDN

Network Traffic Control for Multi-homed End-hosts via SDN

Multi-tenant Multi-Objective Bandwidth Allocation in Datacenters Using Stacked Congestion Control

SmartCC: A Reinforcement Learning Approach for Multipath TCP Congestion Control in Heterogeneous Networks

Dual Channel Adaptive Congestion Control For Datacenters

Dual-structure Data Center Multicast Using Software Defined Networking

Scmptcp: SDN Cooperated Multipath Transfer for Satellite Network with Load Awareness