A Stronger Stitching Algorithm for Fisheye Images based on Deblurring and Registration

Jing Hao,Jingming Xie,Jinyuan Zhang,Moyun Liu
2023-07-22
Abstract:Fisheye lens, which is suitable for panoramic imaging, has the prominent advantage of a large field of view and low cost. However, the fisheye image has a severe geometric distortion which may interfere with the stage of image registration and stitching. Aiming to resolve this drawback, we devise a stronger stitching algorithm for fisheye images by combining the traditional image processing method with deep learning. In the stage of fisheye image correction, we propose the Attention-based Nonlinear Activation Free Network (ANAFNet) to deblur fisheye images corrected by Zhang calibration method. Specifically, ANAFNet adopts the classical single-stage U-shaped architecture based on convolutional neural networks with soft-attention technique and it can restore a sharp image from a blurred image effectively. In the part of image registration, we propose the ORB-FREAK-GMS (OFG), a comprehensive image matching algorithm, to improve the accuracy of image registration. Experimental results demonstrate that panoramic images of superior quality stitching by fisheye images can be obtained through our method.
Computer Vision and Pattern Recognition
What problem does this paper attempt to address?
This paper attempts to address the severe geometric distortion problem in fisheye image stitching. Specifically, while fisheye lenses have the advantages of a wide field of view and low cost, the images they capture suffer from severe geometric distortion, which affects the quality of image registration and stitching. To overcome this drawback, the authors propose a more robust fisheye image stitching algorithm that combines traditional image processing methods with deep learning techniques. ### Main Issues and Solutions: 1. **Fisheye Image Correction**: - **Problem**: After fisheye images are corrected using Zhang’s calibration method, there is still a stretching blur phenomenon, which significantly affects the subsequent image registration process. - **Solution**: The authors propose an Attention-based Nonlinear Activation Free Network (ANAFNet) to deblur the corrected fisheye images. ANAFNet adopts a classic single-stage U-shaped architecture and incorporates soft attention techniques, effectively recovering clear images from blurred ones. 2. **Image Registration**: - **Problem**: Traditional image registration methods have low matching accuracy when dealing with fisheye images due to severe radial distortion. - **Solution**: The authors propose the ORB-FREAK-GMS (OFG) comprehensive image matching algorithm to improve the accuracy of feature point matching. The OFG algorithm combines ORB feature point detection, FREAK descriptor generation, and GMS matching methods, maintaining high matching accuracy under various transformations. ### Experimental Results: - **Deblurring Effect**: Experimental results show that ANAFNet outperforms traditional deblurring methods in both visual quality and quantitative metrics. - **Image Registration Effect**: The OFG algorithm significantly outperforms the traditional ORB+RANSAC method in matching accuracy across multiple datasets, especially demonstrating higher robustness under rotation, scale, illumination, and blur transformations. - **Panoramic Image Stitching Effect**: By combining ANAFNet and the OFG algorithm, the final generated panoramic images achieve satisfactory results in both visual quality and quantitative metrics. In conclusion, the methods proposed in this paper effectively solve the geometric distortion problem in fisheye image stitching, improving the quality and robustness of the stitched images.