Abstract:In recent years, deep learning has attracted substantial research attention for video restoration. Among the existing contributions, the single-frame based approaches purely rely on one reference frame and neglect the rest neighboring frames when enhancing a target frame. By contrast, the multi-frame based contributions exploit temporal information in a sliding window and the existing recurrent design only employ a single preceding enhanced frame. It is intuitive to exploit both multiple original neighboring frames and the preceding enhanced frames for video quality enhancement. In this paper, we propose a Recurrent video quality Enhancement framework with Multi-granularity frame-fusion and frame Difference based attention (REMD). Firstly, we devise a three-dimensional convolutional neural network based encoder-decoder fusion model, which fuses multiple frames in multi-granularity. Secondly, severe compression artifacts tend to emerge on the edges and textures of the compressed frames. We propose a frame difference based spatial attention method to intensify the edges and textures of motioning regions. Finally, a recurrent sliding window design is conceived for exploiting the temporal information in preceding enhanced frames and subsequent neighboring frames. Experiments demonstrate that our method achieves superior performance in comparison to the state-of-the-art contributions with substantially reduced spatial and computational complexity.

Video Enhancement Based on Multiple Region-of-Interests Fusion

A new global-based video enhancement algorithm by fusing features of multiple region-of-interests

Multi-Frame Image Fusion Method Combining Spatial-Temporal Saliency Detection and Nsct

Region-aware RGB and Near-Infrared Image Fusion

Dual Band Image Fusion Method Based on Region Saliency

Intra-and-Inter-Constraint-Based Video Enhancement Based on Piecewise Tone Mapping

An Improved ROI Scalable Video Coding Scheme Based on FMO

LEARNING-BASED MULTI-FRAME VIDEO QUALITY ENHANCEMENT

A Recurrent Video Quality Enhancement Framework with Multi-Granularity Frame-Fusion and Frame Difference Based Attention

Image-based Fusion for Video Enhancement of Night-Time Surveillance

A multilevel region-of-interest based rate control scheme for video communication

Multi-Frame Compressed Video Quality Enhancement by Spatio-Temporal Information Balance

A New Algorithm for Compressing Massive Region-of-interest Location Information in Videos.

Video Frame Interpolation with Region-Distinguishable Priors from SAM

Infrared and Visible Light Image Sequences Fusion Scheme Based on Region Target Detection

Progressive Fusion Video Super-Resolution Network Via Exploiting Non-Local Spatio-Temporal Correlations.

Deep Regional Feature Pooling for Video Matching

Temporal RoI Align for Video Object Recognition

An Image Fusion Approach Based on Object Region

Compressed Video Quality Enhancement With Temporal Group Alignment and Fusion