Abstract:Wide field-of-view (FOV) cameras, which capture a larger scene area than narrow FOV cameras, are used in many applications including 3D reconstruction, autonomous driving, and video surveillance. However, wide-angle images contain distortions that violate the assumptions underlying pinhole camera models, resulting in object distortion, difficulties in estimating scene distance, area, and direction, and preventing the use of off-the-shelf deep models trained on undistorted images for downstream computer vision tasks. Image rectification, which aims to correct these distortions, can solve these problems. In this paper, we comprehensively survey progress in wide-angle image rectification from transformation models to rectification methods. Specifically, we first present a detailed description and discussion of the camera models used in different approaches. Then, we summarize several distortion models including radial distortion and projection distortion. Next, we review both traditional geometry-based image rectification methods and deep learning-based methods, where the former formulates distortion parameter estimation as an optimization problem and the latter treats it as a regression problem by leveraging the power of deep neural networks. We evaluate the performance of state-of-the-art methods on public datasets and show that although both kinds of methods can achieve good results, these methods only work well for specific camera models and distortion types. We also provide a strong baseline model and carry out an empirical study of different distortion models on synthetic datasets and real-world wide-angle images. Finally, we discuss several potential research directions that are expected to further advance this area in the future.

DQN-based gradual fisheye image rectification

DaFIR: Distortion-aware Representation Learning for Fisheye Image Rectification

RoFIR: Robust Fisheye Image Rectification Framework Impervious to Optical Center Deviation

FishEyeRecNet: A Multi-Context Collaborative Deep Network for Fisheye Image Rectification.

Object Detection and Localization in 3D Environment by Fusing Raw Fisheye Image and Attitude Data

SimFIR: A Simple Framework for Fisheye Image Rectification with Self-supervised Representation Learning

Self-Supervised Deep Learning for Fisheye Image Rectification.

Progressively Complementary Network for Fisheye Image Rectification Using Appearance Flow

Unsupervised Fisheye Image Correction Through Bidirectional Loss with Geometric Prior

Downside Hemisphere Object Detection and Localization of MAV by Fisheye Camera

Fish-eye image correction method and technique based on geometrical imaging model

Two distortion correcting methods for fisheye images

SIR: Self-supervised Image Rectification Via Seeing the Same Scene from Multiple Different Lenses

An Underwater Image Enhancement Model for Domain Adaptation

RecRecNet: Rectangling Rectified Wide-Angle Images by Thin-Plate Spline Model and DoF-based Curriculum Learning

FisheyeDepth: A Real Scale Self-Supervised Depth Estimation Model for Fisheye Camera

Pixel-variant Local Homography for Fisheye Stereo Rectification Minimizing Resampling Distortion

Improved Face Detector on Fisheye Images Via Spherical-Domain Attention

Adapting CNNs for Fisheye Cameras without Retraining

Location-guided Head Pose Estimation for Fisheye Image

Wide-Angle Image Rectification: A Survey