Abstract:Because of reinforcement learning's (RL) ability to automatically create more adaptive controlling logics beyond the hand-crafted heuristics, numerous effort has been made to apply RL to congestion control (CC) design for real time video communication (RTC) applications and has successfully shown promising benefits over the rule-based RTC CCs. Online reinforcement learning is often adopted to train the RL models so the models can directly adapt to real network environments. However, its trail-and-error manner can also cause catastrophic degradation of the quality of experience (QoE) of RTC application at run time. Thus, safeguard strategies such as falling back to hand-crafted heuristics can be used to run along with RL models to guarantee the actions explored in the training sensible, despite that these safeguard strategies interrupt the learning process and make it more challenging to discover optimal RL policies. The recent emergence of loss-tolerant neural video codecs (NVC) naturally provides a layer of protection for the online learning of RL-based congestion control because of its resilience to packet losses, but such packet loss resilience have not been fully exploited in prior works yet. In this paper, we present a reinforcement learning (RL) based congestion control which can be aware of and takes advantage of packet loss tolerance characteristic of NVCs via reward in online RL learning. Through extensive evaluation on various videos and network traces in a simulated environment, we demonstrate that our NVC-aware CC running with the loss-tolerant NVC reduces the training time by 41\% compared to other prior RL-based CCs. It also boosts the mean video quality by 0.3 to 1.6dB, lower the tail frame delay by 3 to 200ms, and reduces the video stalls by 20\% to 77\% in comparison with other baseline RTC CCs.

Learning-based Congestion Control for Internet Video Communication over Wireless Networks

A Multi-objective Reinforcement Learning Perspective on Internet Congestion Control

Deep Reinforcement Learning for Smart City Communication Networks

ETCC: A congestion control scheme for real-time multimedia streaming

Statistical Learning Based Congestion Control for Real-time Video Communication

Delay Constrained Self Congestion Control for FGS Video Streams

Reinforcement Learning Based Cross-Layer Congestion Control for Real-Time Communication

Congestion Control Design of Multicast QUIC Based on Reinforcement Learning

A New Bandwidth Prediction Based Wireless Ad-Hoc Video Communication

Delay-constrained rate control for real-time video streaming over wireless networks

Upload Your Data Faster: Driver-Queue Based Congestion Control for Wireless Networks

Towards Fair and Efficient Learning-based Congestion Control

Context-Aware Cross-Layer Congestion Control for Large-Scale Live Streaming

PACC: Perception Aware Congestion Control for Real-time Communication

HybridRTS: A Hybrid Congestion Control Framework with Rule and Reinforcement Learning for Low-Latency WebRTC Live Video Streaming

A Congestion Control Algorithm for Live Video Streaming in Dynamic Network

Cross-Layer Design of Source Rate Control and Congestion Control for Wireless Video Streaming

Learned Internet Congestion Control for Short Video Uploading

Loss-tolerant neural video codec aware congestion control for real time video communication

A Unified Congestion Control Framework for Diverse Application Preferences and Network Conditions.

Reinforcement learning for bandwidth estimation and congestion control in real-time communications