VCMaker: Content-aware configuration adaptation for video streaming and analysis in live augmented reality

Ning Chen,Sheng Zhang,Siyi Quan,Zhi Ma,Zhuzhong Qian,Sanglu Lu

DOI: https://doi.org/10.1016/j.comnet.2021.108513

IF: 5.493

2021-01-01

Computer Networks

Abstract:The emergence of edge computing has enabled mobile Augmented Reality (AR) on edge servers. We notice that the video configurations, i.e., frames per second (fps) and resolution, significantly affect the key metrics such as detection accuracy, data transmission latency and energy consumption in real AR application. Besides the time-varying bandwidth, we observe that the video contents, such as moving velocities of target objects, have remarkable impacts on the configuration selection. In addition, we take the energy consumption on data transmission into consideration. In this paper, we propose VCMaker, a system that generates video configuration decisions using reinforcement learning (RL). VCMaker trains a neural network model that selects configuration for future video chunks based on the collected observations. Rather than rely on any preprogrammed models, VCMaker learns to make configuration decisions solely through empirical observations of the resulting performances of historical decisions. In addition, we leverage the dynamic Region of Interest (RoI) encoding and motion vector-based object detection mechanisms to advance VCMaker. We implemented VCMaker and conducted extensive evaluations. The results show that VCMaker achieves a 20.5%-32.8% higher detection accuracy, and 25.2%-45.7% lower energy consumption than several state-of-the-art schemes.