RBSR: Efficient and Flexible Recurrent Network for Burst Super-Resolution

Renlong Wu,Zhilu Zhang,Shuohao Zhang,Hongzhi Zhang,Wangmeng Zuo

DOI: https://doi.org/10.48550/arXiv.2306.17595

2023-06-30

Computer Vision and Pattern Recognition

Abstract:Burst super-resolution (BurstSR) aims at reconstructing a high-resolution (HR) image from a sequence of low-resolution (LR) and noisy images, which is conducive to enhancing the imaging effects of smartphones with limited sensors. The main challenge of BurstSR is to effectively combine the complementary information from input frames, while existing methods still struggle with it. In this paper, we suggest fusing cues frame-by-frame with an efficient and flexible recurrent network. In particular, we emphasize the role of the base-frame and utilize it as a key prompt to guide the knowledge acquisition from other frames in every recurrence. Moreover, we introduce an implicit weighting loss to improve the model's flexibility in facing input frames with variable numbers. Extensive experiments on both synthetic and real-world datasets demonstrate that our method achieves better results than state-of-the-art ones. Codes and pre-trained models are available at https://github.com/ZcsrenlongZ/RBSR.

What problem does this paper attempt to address?

### Problems Addressed by the Paper

The paper aims to address the issue of multi-frame information fusion in Burst Super-Resolution (BurstSR). The goal of BurstSR is to reconstruct a high-resolution (HR) image from a series of low-resolution (LR) and noisy images, which is very beneficial for improving the imaging performance of smartphones under limited sensor conditions. However, existing BurstSR methods still face challenges in effectively combining complementary information from input frames. Specifically, the paper focuses on the following issues:

1. **Effective fusion of multi-frame information**: Existing methods have limitations in merging multi-frame information, especially when dealing with different numbers of input frames.
2. **Role of the base frame**: The importance of the base frame in multi-frame fusion has not been fully explored. How to use the base frame to guide the information acquisition of other frames is a key issue.
3. **Model flexibility**: How to design a model that can efficiently handle different numbers of input frames while maintaining high performance.

To address these issues, the authors propose an efficient Recurrent Network, named RBSR (Recurrent Burst Super-Resolution). This network fuses information frame by frame and uses the base frame as a key hint to guide the knowledge acquisition of other frames, thereby improving the performance and flexibility of the model. Additionally, the authors introduce an Implicit Weighting Loss to enhance the model's ability to handle different numbers of input frames. Experimental results show that RBSR outperforms existing methods on both synthetic and real-world datasets, not only in terms of performance but also in inference efficiency.