Sparkle: User-Aware Viewport Prediction in 360-Degree Video Streaming

Jinyu Chen,Xianzhuo Luo,Miao Hu,Di Wu,Yipeng Zhou

DOI: https://doi.org/10.1109/tmm.2020.3033127

IF: 7.3

2021-01-01

IEEE Transactions on Multimedia

Abstract:In 360-degree video streaming, users commonly watch a video scene within a Field of View (FoV). Such observation provides an opportunity to save bandwidth consumption by predicting and then prefetching video tiles within the FoV. However, existing FoV prediction methods seldom consider the diversity among user behaviors and the impact of different video genres. Thus, previous one-size-fits-all models cannot make accurate prediction for users with different behavior patterns. In this paper, we propose a user-aware viewport prediction algorithm called Sparkle, which is a practical whitebox approach for FoV prediction. Instead of training a single learning model to predict the behaviors for all users, our proposed algorithm is tailored to fit each individual user. In particular, unlike other learning models, our prediction model is completely explainable and all the parameters have their physical meanings. We first conduct a measurement study to analyze real user behaviors and observe that there exists sharp fluctuation of view orientation and user posture has significant impact on the viewport movement of users. Moreover, cross-user similarity is diverse across different video genres. Inspired by these insights, we further design a user-aware viewport prediction algorithm by mimicking a user's viewport movement on the tile map, and determine how a user will change the viewport angle based on his (or her) trajectory and other similar users' behaviors in the past time window. Extensive evaluations with real datasets demonstrate that, our proposed algorithm significantly outperforms the state-of-the-art benchmark methods (e.g., LSTM-based methods) by over <span class="mjpage"><svg xmlns:xlink="http://www.w3.org/1999/xlink" width="5.255ex" height="2.176ex" style="vertical-align: -0.338ex;" viewBox="0 -791.3 2262.5 936.9" role="img" focusable="false" xmlns="http://www.w3.org/2000/svg"><g stroke="currentColor" fill="currentColor" stroke-width="0" transform="matrix(1 0 0 -1 0 0)"> <use xlink:href="#MJMATHI-74" x="0" y="0"></use> <use xlink:href="#MJMATHI-65" x="361" y="0"></use> <use xlink:href="#MJMATHI-78" x="828" y="0"></use> <use xlink:href="#MJMATHI-74" x="1400" y="0"></use> <use xlink:href="#MJMAIN-35" x="1762" y="0"></use></g></svg></span>, and the prediction accuracy is much more stable on various types of 360-degree videos than previous methods.<svg xmlns="http://www.w3.org/2000/svg" style="display: none;"><defs id="MathJax_SVG_glyphs"><path stroke-width="1" id="MJMATHI-74" d="M26 385Q19 392 19 395Q19 399 22 411T27 425Q29 430 36 430T87 431H140L159 511Q162 522 166 540T173 566T179 586T187 603T197 615T211 624T229 626Q247 625 254 615T261 596Q261 589 252 549T232 470L222 433Q222 431 272 431H323Q330 424 330 420Q330 398 317 385H210L174 240Q135 80 135 68Q135 26 162 26Q197 26 230 60T283 144Q285 150 288 151T303 153H307Q322 153 322 145Q322 142 319 133Q314 117 301 95T267 48T216 6T155 -11Q125 -11 98 4T59 56Q57 64 57 83V101L92 241Q127 382 128 383Q128 385 77 385H26Z"></path><path stroke-width="1" id="MJMATHI-65" d="M39 168Q39 225 58 272T107 350T174 402T244 433T307 442H310Q355 442 388 420T421 355Q421 265 310 237Q261 224 176 223Q139 223 138 221Q138 219 132 186T125 128Q125 81 146 54T209 26T302 45T394 111Q403 121 406 121Q410 121 419 112T429 98T420 82T390 55T344 24T281 -1T205 -11Q126 -11 83 42T39 168ZM373 353Q367 405 305 405Q272 405 244 391T199 357T170 316T154 280T149 261Q149 260 169 260Q282 260 327 284T373 353Z"></path><path stroke-width="1" id="MJMATHI-78" d="M52 289Q59 331 106 386T222 442Q257 442 286 424T329 379Q371 442 430 442Q467 442 494 420T522 361Q522 332 508 314T481 292T458 288Q439 288 427 299T415 328Q415 374 465 391Q454 404 425 404Q412 404 406 402Q368 386 350 336Q290 115 290 78Q290 50 306 38T341 26Q378 26 414 59T463 140Q466 150 469 151T485 153H489Q504 153 504 145Q504 144 502 134Q486 77 440 33T333 -11Q263 -11 227 52Q186 -10 133 -10H127Q78 -10 57 16T35 71Q35 103 54 123T99 143Q142 143 142 101Q142 81 130 66T107 46T94 41L91 40Q91 39 97 36T113 29T132 26Q168 26 194 71Q203 87 217 139T245 247T261 313Q266 340 266 352Q266 380 251 392T217 404Q177 404 142 372T93 290Q91 281 88 280T72 278H58Q52 284 52 289Z"></path><path stroke-width="1" id="MJMAIN-35" d="M164 157Q164 133 148 117T109 101H102Q148 22 224 22Q294 22 326 82Q345 115 345 210Q345 313 318 349Q292 382 260 382H254Q176 382 136 314Q132 307 129 306T114 304Q97 304 95 310Q93 314 93 485V614Q93 664 98 664Q100 666 102 666Q103 666 123 658T178 642T253 634Q324 634 389 662Q397 666 402 666Q410 666 410 648V635Q328 538 205 538Q174 538 149 544L139 546V374Q158 388 169 396T205 412T256 420Q337 420 393 355T449 201Q449 109 385 44T229 -22Q148 -22 99 32T50 154Q50 178 61 192T84 210T107 214Q132 214 148 197T164 157Z"></path></defs></svg>

computer science, information systems,telecommunications, software engineering