Video Decoding Energy Reduction Using Temporal-Domain Filtering

Christian Herglotz,Matthias Kränzler,Robert Ludwig,André Kaup

DOI: https://doi.org/10.1145/3593908.3593948

2023-06-12

Abstract:In this paper, we study decoding energy reduction opportunities using temporal-domain filtering and subsampling methods. In particular, we study spatiotemporal filtering using a contrast sensitivity function and temporal downscaling, i.e., frame rate reduction. We apply these concepts as a pre-filtering to the video before compression and evaluate the bitrate, the decoding energy, and the visual quality with a dedicated metric targeting temporally down-scaled sequences. We find that decoding energy savings yield 35% when halving the frame rate and that spatiotemporal filtering can lead to up to 5% of additional savings, depending on the content.

Image and Video Processing

What problem does this paper attempt to address?

This paper aims to explore methods for reducing video decoding energy consumption through time - domain filtering and frame - rate reduction techniques. Specifically, the researchers analyzed the effects of spatio - temporal filtering using the Contrast Sensitivity Function (CSF) and frame - rate reduction on video bit rate, decoding energy consumption, and visual quality. They used these methods as pre - processing steps before video compression, evaluated their performance on different video contents, and evaluated visual quality through metrics specifically for time - domain downscaling sequences.

The main objectives of the paper can be summarized as follows:

1. **Explore opportunities for reducing video decoding energy consumption**: Use time - domain filtering and frame - rate reduction methods to study how to significantly reduce the energy consumption in the video decoding process while maintaining or slightly affecting the visual quality of the video.
2. **Evaluate the effect of spatio - temporal filtering**: Identify and remove the frequency components invisible to the human eye in the video through the Contrast Sensitivity Function (CSF), thereby reducing the entropy of the video signal, with the expectation of reducing the bit rate and ultimately reducing the decoding complexity.
3. **Analyze the impact of frame - rate reduction**: Study the specific impact of frame - rate reduction on decoding energy consumption and visual quality, especially how much percentage the decoding energy consumption can be reduced when the frame rate is halved.
4. **Propose an effective video processing strategy**: Combine the above two methods to propose a video processing strategy that can effectively reduce decoding energy consumption without significantly affecting video quality.

Through these studies, the author hopes to provide theoretical support and technical guidance for the development of low - power and high - efficiency online video applications, especially on mobile devices, which is particularly important for battery - powered portable devices.