Fisheye lens distortion correction

Dmitry Pozdnyakov

DOI: https://doi.org/10.48550/arXiv.2010.10295

2020-10-19

Image and Video Processing

Abstract:A new distortion correction algorithm for fisheye lens with equidistant mapping function is considered in the present study. The algorithm is much more data lossless and accurate than such a classical approach like Brown-Conrady model

What problem does this paper attempt to address?

Linear fitting interpolation based on FOV division for correcting wide angle fish-eye lens distortion

an li,yisi wu,chi chen,zhenrong zheng

DOI: https://doi.org/10.1117/12.2186738

2015-01-01

Abstract:The wide angle lens, like fish eye lens, suffers great optical distortion that causes severe deformation of the real world. A method to correct the strong distortion was presented in this work. Due to the nonlinear distribution of the distortion, linear algorithms are generally not under consideration to establish the math model of distorted-to-ideal images. However, this method employed the calibration pattern that comprised of regular array of dots to divides the full field of view (FOV) to subsections, each subsection is a small FOV, the mapping parameters between the distorted image and ideal image in each small FOV can be calculated by employing the very simple linear polynomial. Thus, applying the determined parameters to their corresponding sub-FOVs, respectively, all the ideal pixel coordinates of the distorted image can be obtained. The method employed linear polynomial characterizes the geometric deformation between the distorted and ideal images directly. Therefore, it contains both of radial distortion and tangential distortion and there is no need of concerning any intrinsic or extrinsic parameters of the optical systems. So, this algorithm reliefs the computational work that employed by conventional radial models and other mathematical models. Experiments performed on off-axis optical systems which exist complicated distortion, such as the head mounted displays (HMDs), had already yielded accurate correcting results. Likewise, in this paper, the experiments refer to the fish-eye lens also verify the effectiveness and flexibility of this method, as well, high correcting accuracy is achieved.
Estimating Radial Distortion for Fish-eye Lens Based on Collinear Constraint

ZHU Yunfang,DU Xin

DOI: https://doi.org/10.3969/j.issn.1003-501x.2014.11.006

2014-01-01

Abstract:A method of calibrating radial distortion for fish-eye lens based on collinear constraint is proposed. The method only needs a single image which contains at least two sets of parallel lines in 3D world. Under single parameter division model, these lines are imaged as circular arcs in the distorted image. These circular arcs imaged from the same set of parallel lines will intersect at two common points, and thus their centers are collinear. Under this constraint, the calculation of parameters of the single parameter division model is then proposed, which can be solved by using the Levenberg-Marquardt algorithm effectively. Experimental results of both synthetic and real images show that the proposed method is simple, robust and accurate.
Two distortion correcting methods for fisheye images

Zhi Wang,Zhaofeng Cen,Xiaotong Li

DOI: https://doi.org/10.1117/12.2283532

2017-01-01

Abstract:Fisheye lens forms a circular image on the image plane which is different from people's perspective view. This paper introduces two practical methods to correct the fisheye image from distortion. Distorted images can be corrected to the perspective views which are suitable for people to watch by rectilinear projection based on lens functions. This paper extended this method so that it can correct images taken by fisheye lens over 180 degrees. Since the rectilinear projection method has the limitation that it can only correct a ROI, the second method based on an improved cylinder projection is proposed to overcome this limitation. The picture corrected by this method has less deformation compared with traditional cylinder projection method.
Fish-eye image correction method and technique based on geometrical imaging model

LI Gen,FEI Zhang-jun,YANG Shi-you

2013-01-01

Abstract:Fish-eye lenses with wide field of view(FOV)are widely used in the field of large scene video surveillance,but nonlinear radial distortion of fish-eye image violated the human visual habits and brought inconvenience to measurement and pattern recognition applications. In order to correct the fish-eye images,a new correction method and technique based on geometrical imaging model that corrects the images form different types of fish- eye lens by corresponding geometrical imaging model was proposed. Also,the implementation method of equidistant imaging model was developed. Finally,the optimization methods based on the cache and a fast bilinear interpolation implementation ensuring the accuracy aiming at the real-time application were presented. The results indicate that the new method works efficiently in a larger scale,and easier to implement.
Distortion correction of fisheye lenses using a nonparametric imaging model

Xianghua Ying,Zhanyi Hu

2004-01-01

Abstract:Fisheye lenses are often used to enlarge the field of view of a conventional camera. But the images taken with a camera with fisheye lenses have severe distortions. This paper presents a new non-parametric imaging modeling method to transform the distorted images into the quasiperspective images. The non-parametric model is a mapping from the rays captured by the camera to their corresponding image points. The non-parametric model of the fisheye camera can be conveniently obtained using the inner surface of a hemispherical calibration pattern. The key distinct characteristic between the parametric method and the non-parametric method is that: the parametric distortion model does not often fit the real distortions in the imaging system well, and the nonparametric model can fit the imaging system better if there are more control points on the hemispherical pattern. Our experimental results show that the …
A Comprehensive Overview of Fish-Eye Camera Distortion Correction Methods

Jian Xu,De-Wei Han,Kang Li,Jun-Jie Li,Zhao-Yuan Ma

2024-05-13

Abstract:The fisheye camera, with its unique wide field of view and other characteristics, has found extensive applications in various fields. However, the fisheye camera suffers from significant distortion compared to pinhole cameras, resulting in distorted images of captured objects. Fish-eye camera distortion is a common issue in digital image processing, requiring effective correction techniques to enhance image quality. This review provides a comprehensive overview of various methods used for fish-eye camera distortion correction. The article explores the polynomial distortion model, which utilizes polynomial functions to model and correct radial distortions. Additionally, alternative approaches such as panorama mapping, grid mapping, direct methods, and deep learning-based methods are discussed. The review highlights the advantages, limitations, and recent advancements of each method, enabling readers to make informed decisions based on their specific needs.

Computer Vision and Pattern Recognition
Correction of Barrel Distortion in Fisheye Lens Images Using Image-Based Estimation of Distortion Parameters

Minjung Lee,Hyungtae Kim,Joonki Paik

DOI: https://doi.org/10.1109/access.2019.2908451

IF: 3.9

2019-01-01

IEEE Access

Abstract:Images acquired by a fisheye lens camera contain geometric distortion that results in deformation of the object’s shape. To correct the lens distortion, existing methods use prior information, such as calibration patterns or lens design specifications. However, the use of a calibration pattern works only when an input scene is a 2-D plane at a prespecified position. On the other hand, the lens design specifications can be understood only by optical experts. To solve these problems, we present a novel image-based algorithm that corrects the geometric distortion. The proposed algorithm consists of three stages: i) feature detection, ii) distortion parameter estimation, and iii) selection of the optimally corrected image out of multiple corrected candidates. The proposed method can automatically select the optimal amount of correction for a fisheye lens distortion by analyzing characteristics of the distorted image using neither prespecified lens design parameters nor calibration patterns. Furthermore, our method performs not only on-line correction by using facial landmark points, but also off-line correction described in subsection III-C. As a result, the proposed method can be applied to a virtual reality (VR) or augmented reality (AR) camera with two fisheye lenses in a field-of-view (FOV) of 195°, autonomous vehicle vision systems, wide-area visual surveillance systems, and unmanned aerial vehicle (UAV) cameras.

computer science, information systems,telecommunications,engineering, electrical & electronic
Camera lens radial distortion correction using two-view projective invariants.

Xin Du,Hongdong Li,Yunfang Zhu

DOI: https://doi.org/10.1364/OL.36.004734

IF: 3.6

2011-01-01

Optics Letters

Abstract:This Letter proposes a new method for automatically removing radial lens distortion from image feature point correspondences of two views. Based on the projective geometric relationship between two views of a planar scene, we have derived a system of algebraic equations that relates the invariants to the distortion parameters to be found. We then propose a noniterative procedure to solve the equations system, and a kernel-voting scheme to select the best root. Being a noniterative approach, our method overcomes many problems with the conventional iterative approach. It also largely decouples the estimation of the distortion from the estimation of other camera parameters and, therefore, delivers more reliable results. Experiments on both synthetic data and real images have provided satisfactory results. (C) 2011 Optical Society of America
Calibration of Fisheye Cameras Based on the Viewing Sphere

林颖,龚小谨,刘济林

DOI: https://doi.org/10.3785/j.issn.1008-973x.2013.08.026

2013-01-01

Abstract:With respect to the large field of view(FOV) of a fisheye camera,a new calibration method was proposed,This method takes advantage of two mutually orthogonal sets of parallel lines.The intrinsic parameters could be initialized by the field of view(FOV) and imaging boundary,and corners extracted in the image plane could be back-projected to a viewing sphere.Two geometric properties of the two line sets on the viewing sphere provided a closed-form solution for extrinsic parameters.Finally,all the parameters were optimized according to the re-projection errors of the corners.Three commonly used fisheye projection models are simulated with respect to the image number and noise level.The simulation shows that,within meaningful noise,the method can achieve high accuracy when the image number is larger than 5.A fisheye camera with 185°FOV is also calibrated,and a geometry-known trihedral object is further tested to validate the estimation of extrinsic parameters.The experiments compared to the popular fisheye calibration toolbox from Caltech prove that with a lower uncertainty,the proposed method is more accurate than Caltech's toolbox.
Fisheye image Distortion Correction Based on Spherical Perspective Projection Constraint

Yue Liu,Baofeng Zhang,Na Liu,Hongyan Li,Junchao Zhu

DOI: https://doi.org/10.1109/icma49215.2020.9233684

2020-10-13

Abstract:Fisheye lens has the prominent advantage of large field of view, which is suitable for the overall environmental monitoring during the restoration of polluted sites. But at the same time, the fisheye lens has a large distortion, which has a great influence on the image matching and stitching between different cameras. Aiming at the shortcomings of traditional wide-angle lens image correction methods, only correction is carried out in the longitude direction, and the correction effect is poor. A fisheye correction algorithm based on spherical perspective projection constraint was proposed, in the case of spherical perspective projection, the space line is no longer projected as a plane line of the image, but as a great circle on the sphere. First, adopt the scan line approximation method to extract the effective area of the fisheye image, to determine the center and radius; then in view of the shortcomings and deficiencies of the spherical coordinate positioning method and the latitude and longitude mapping method. A correction method based on spherical perspective projection constraint is quantized; finally, the experimental verification is carried out. Through the experiment, this method has a good effect on the correction of fisheye image, the shape of the object in the image has been restored, and several arcs have become parallel lines, which are very consistent with the reality.
A Fast Correction Algorithm for Inverse Longitude and Latitude Mapping of Fish Eye Image

Xingxing Li,Chao Duan,Panpan Yin,Xingning Wang

DOI: https://doi.org/10.1088/1742-6596/1982/1/012044

2021-07-01

Journal of Physics: Conference Series

Abstract:Abstract In order to solve the problem of large barrel distortion in fisheye lens, an image distortion correction algorithm based on double longitude is proposed to improve the visual effect. Firstly, in the case of unknown lens angle and non-circular fisheye image, the spherical center and its radius are obtained by using fisheye image features, and the radius is optimized to reduce pole distortion; on this basis, the fisheye image is mapped to the spherical surface by orthogonal projection strategy using double longitude model method, which is transformed into the transverse longitude and longitudinal longitude coordinates of the sphere The projection is a square plane image based on the horizontal and vertical double longitude coordinates to realize the fast and accurate image transformation.
Fisheye lens distortion correction using spherical perspective projection constraint

Xianghua Ying,Hu Zhan yi

2003-01-01

Jisuanji Xuebao/Chinese Journal of Computers

Abstract:Fisheye lenses are often used to enlarge the field of view of a camera. But the images taken with the fisheye camera often have severe distortions. This paper proposes a new fisheye lenses distortion correction method using spherical perspective projection (SPP) constraint. Space line are generally projected into curves on a fisheye image. The SPP constraint means that a space line must be projected into a great circle on a unit sphere under the SPP model. The reason of using the SPP model is to handle the situation of larger-than-180-degree view-angles, and our case happens to be true. Our proposed distortion correction process consists of the following two main steps: Firstly, polynomial models for radial and tangential distortions with free parameters are selected; Secondly, these parameters are estimated based on the fact that the projected image curves of space lines should be mapped to great circles on a unit spherical surface, i.e., the SPP constraint. By minimizing the sum of errors, the squares of spherical distances points to their corresponding best-fit great circles, the parameters are obtained. Our minimization process also consists of two steps, first, an initial estimation is obtained by the linearization of the two distortion models, then Levenberg-Marquardt algorithm is employed for the final nonlinear optimization. Finally, experimental results with synthetic data under different noise levels as well as with real fisheye images are reported, and the results appear satisfactory.
Unified Optical Distortion Correction Method for Imaging Systems Using A Concise Geometrical Transformation Model

Shengqian Chang,Siqi Liu,Fei Yuan,Zhenrong Zheng

DOI: https://doi.org/10.1016/j.optcom.2016.08.069

IF: 2.4

2017-01-01

Optics Communications

Abstract:Since optical distortion has been a big trouble for various kinds of imaging systems, finding a simple correction method with wide applications is of significant importance. In this paper, we propose a unified and simple correction method, performing well for both photographic and projective imaging systems. The basic idea is regarding the optical distortion as geometrical deformation between the object and image, without considering the specific features of an optical system. First of all, a calibration template is employed to establish the geometrical transformation model (GTM) for the distortion of a built optical system. Two alternative algorithms are given to estimate the GTM in algebraic form. The computation is very simple because no intrinsic parameters of the optical system are needed to establish the GTM. Besides, the errors introduced by the fabricating and assembling process can be eliminated. Then, the corrected image of the photographic system or the pre-distorted image of the projective systems can be obtained accordingly utilizing the GTM. Experiments are conducted to demonstrate the effectiveness of our method with wide applications.
Image Data Loss Minimized Geometric Correction for Asymmetric Distortion Fish-eye Lens

Jiyoung Park,Joong-Ryoul Lee,Sung-Hee Kim,Jin-Woo Son,Young-Ju Cho,Myoung-Hee Kim

Abstract:ABSTRACTDue to the fact that fisheye lens can provide super wide angles with the minimum number of cameras, field-of-view over 180 degrees, many vehicles are attempting to mount the camera system. Not only use the camera as a viewing system, but also as a camera sensor, camera calibration should be preceded, and geometrical correction on the radial distortion is needed to provide the images for the driver's assistance. In this thesis, we introduce a geometric correction technique to minimize the loss of the image data from a vehicle fish-eye lens having a field of view over 180°, and a asymmetric distortion. Geometric correction is a process in which a camera model with a distortion model is established, and then a corrected view is generated after camera parameters are calculated through a calibration process. First, the FOV model to imitate a asymmetric distortion configuration is used as the distortion model. Then, we need to unify the axis ratio because a horizontal view of the vehicle fish-eye lens is asymmetrically wide for the driver, and estimate the parameters by applying a non-linear optimization algorithm. Finally, we create a corrected view by a backward mapping, and provide a function to optimize the ratio for the horizontal and vertical axes. This minimizes image data loss and improves the visual perception when the input image is undistorted through a perspective projection.

Computer Science,Engineering,Mathematics
RoFIR: Robust Fisheye Image Rectification Framework Impervious to Optical Center Deviation

Zhaokang Liao,Hao Feng,Shaokai Liu,Wengang Zhou,Houqiang Li

2024-06-27

Abstract:Fisheye images are categorized fisheye into central and deviated based on the optical center position. Existing rectification methods are limited to central fisheye images, while this paper proposes a novel method that extends to deviated fisheye image rectification. The challenge lies in the variant global distortion distribution pattern caused by the random optical center position. To address this challenge, we propose a distortion vector map (DVM) that measures the degree and direction of local distortion. By learning the DVM, the model can independently identify local distortions at each pixel without relying on global distortion patterns. The model adopts a pre-training and fine-tuning training paradigm. In the pre-training stage, it predicts the distortion vector map and perceives the local distortion features of each pixel. In the fine-tuning stage, it predicts a pixel-wise flow map for deviated fisheye image rectification. We also propose a data augmentation method mixing central, deviated, and distorted-free images. Such data augmentation promotes the model performance in rectifying both central and deviated fisheye images, compared with models trained on single-type fisheye images. Extensive experiments demonstrate the effectiveness and superiority of the proposed method.

Computer Vision and Pattern Recognition
Orthorectification of Fisheye Image under Equidistant Projection Model

Guoqing Zhou,Huanxu Li,Ruhao Song,Qingyang Wang,Jiasheng Xu,Bo Song

DOI: https://doi.org/10.3390/rs14174175

IF: 5

2022-08-26

Remote Sensing

Abstract:The fisheye camera, with its large viewing angle, can acquire more spatial information in one shot and is widely used in many fields. However, a fisheye image contains large distortion, resulting in that many scholars have investigated its accuracy of orthorectification, i.e., generation of digital orthophoto map (DOM). This paper presents an orthorectification method, which first determines the transformation relationship between the fisheye image points and the perspective projection points according to the equidistant projection model, i.e., determines the spherical distortion of the fisheye image; then introduces the transformation relationship and the fisheye camera distortion model into the collinearity equation to derive the fisheye image orthorectification model. To verify the proposed method, high accuracy of the fisheye camera 3D calibration field is established to obtain the interior and exterior orientation parameters (IOPs/EOPs) and distortion parameters of the fisheye lens. Three experiments are used to verify the proposed orthorectification method. The root mean square errors (RMSEs) of the three DOMs are averagely 0.003 m, 0.29 m, and 0.61 m, respectively. The experimental results demonstrate that the proposed method is correct and effective.

environmental sciences,imaging science & photographic technology,remote sensing,geosciences, multidisciplinary
Unsupervised Fisheye Image Correction Through Bidirectional Loss with Geometric Prior

Shangrong Yang,Chunyu Lin,Kang Liao,Yao Zhao,Meiqin Liu

DOI: https://doi.org/10.1016/j.jvcir.2019.102692

IF: 2.887

2019-01-01

Journal of Visual Communication and Image Representation

Abstract:Neural network based methods for fisheye distortion correction are effective and increasingly popular, although training network require a high amount of labeled data. In this paper, we propose an unsupervised fisheye correction network to address the aforementioned issue. During the training process, the predicted parameters are employed to correct strong distortion that exists in the fisheye image and synthesize the corresponding distortion using the original distortion-free image. Thus, the network is constrained with bidirectional loss to obtain more accurate distortion parameters. We calculate the two losses at the image level as opposed to directly minimizing the difference between the predicted and ground truth of distortion parameters. Additionally, we leverage the geometric prior that the distortion distribution depends on the geometric regions of fisheye images and the straight line should be straight in the corrected images. The network focuses more on the geometric prior regions as opposed to equally perceiving the whole image without any attention mechanisms. To generate more appealing corrected results in visual appearance, we introduce a coarse-to-fine inpainting network to fill the hole regions caused by the irreversible mapping function using distortion parameters. Each module of the proposed network is differentiable, and thus the entire framework is completely end-to-end. When compared with the previous supervised methods, our method is more flexible and shows better practical applications for distortion rectification. The experiment results demonstrate that our proposed method outperforms state-of-the-art methods on the correction performance without any labeled distortion parameters.
Improved interpolation with sub-pixel relocation method for strong barrel distortion

Xuan Yuwen,Silong Zhang,Long Chen,Hui Zhang

DOI: https://doi.org/10.1016/j.sigpro.2022.108795

IF: 4.729

2023-02-01

Signal Processing

Abstract:In this paper, we propose an improved interpolation method with sub-pixel relocation method to correct fisheye images with strong barrel distortion. The proposed method takes as inputs the fisheye image and the projection model, and outputs the corrected perspective image. In the method, we project the fisheye image pixels to perspective image plane using the backward projection. The projected pixels are treated as scatter data, and their location is refined by the sub-pixel relocation method. The method is designed to aim at the sub-pixel bias phenomenon of the projected pixels. The sub-pixel bias phenomenon is that there is a sub-pixel bias between the projected pixel center point and the center point of the projected pixel. This phenomenon is caused by the uneven distortion of the fisheye image plane, and it will affect the accuracy of the interpolation method. The sub-pixel relocation method relocates the projected pixel centers to the centers of the projected pixels. Then the relocated points are meshed by the Voronoi diagram method. The mesh is used to select the neighbors of the interpolation points as natural neighbor selection method. The weights of the neighbors are calculated using the inverse distance method. The indexes of the neighbors and their weights are stored in a look-up-table structure for reuse. For each fisheye image, the proposed method can quickly find the neighbors of the interpolation points and the weight of each neighbor through the look-up-table, and the perspective image can be calculated in real-time. Test results illustrate that the proposed method achieves the best accuracy among the tested methods for fisheye images with strong barrel distortion.

engineering, electrical & electronic
Exploiting the Distortion-Semantic Interaction in Fisheye Data

Kiran Kokilepersaud,Mohit Prabhushankar,Yavuz Yarici,Ghassan AlRegib,Armin Parchami

2023-05-07

Abstract:In this work, we present a methodology to shape a fisheye-specific representation space that reflects the interaction between distortion and semantic context present in this data modality. Fisheye data has the wider field of view advantage over other types of cameras, but this comes at the expense of high radial distortion. As a result, objects further from the center exhibit deformations that make it difficult for a model to identify their semantic context. While previous work has attempted architectural and training augmentation changes to alleviate this effect, no work has attempted to guide the model towards learning a representation space that reflects this interaction between distortion and semantic context inherent to fisheye data. We introduce an approach to exploit this relationship by first extracting distortion class labels based on an object's distance from the center of the image. We then shape a backbone's representation space with a weighted contrastive loss that constrains objects of the same semantic class and distortion class to be close to each other within a lower dimensional embedding space. This backbone trained with both semantic and distortion information is then fine-tuned within an object detection setting to empirically evaluate the quality of the learnt representation. We show this method leads to performance improvements by as much as 1.1% mean average precision over standard object detection strategies and .6% improvement over other state of the art representation learning approaches.

Computer Vision and Pattern Recognition,Artificial Intelligence
DaFIR: Distortion-aware Representation Learning for Fisheye Image Rectification

Zhaokang Liao,Wengang Zhou,Houqiang Li

DOI: https://doi.org/10.1109/tcsvt.2023.3315967

IF: 5.859

2023-01-01

IEEE Transactions on Circuits and Systems for Video Technology

Abstract:This paper focuses on fisheye image rectification. Existing learning-based solutions learn image representations that mix distortion features and content features. Since the distortion feature dominates the rectification process, we propose a novel distortion-aware representation learning framework, which decouples the distortion feature from the content feature, for fisheye image rectification. Specifically, we first pre-train a Vision Transformer with a supervised pre-text task, which regresses the distortion distribution map of a distorted image. The pre-training equips the Vision Transformer with the ability to capture distortion-related patterns. After that, the pre-trained model is fine-tuned to predict the pixel-wise flow map to rectify the fisheye images. Extensive experiments are conducted to evaluate our approach and verify our idea of feature decoupling. The experiment results demonstrate the state-of-the-art performance of our approach compared to existing algorithms, as well as its generality on real-world images. Our source code is publicly available at https://github.com/lzk9508/DaFIR.

Fisheye lens distortion correction

Linear fitting interpolation based on FOV division for correcting wide angle fish-eye lens distortion

Estimating Radial Distortion for Fish-eye Lens Based on Collinear Constraint

Two distortion correcting methods for fisheye images

Fish-eye image correction method and technique based on geometrical imaging model

Distortion correction of fisheye lenses using a nonparametric imaging model

A Comprehensive Overview of Fish-Eye Camera Distortion Correction Methods

Correction of Barrel Distortion in Fisheye Lens Images Using Image-Based Estimation of Distortion Parameters

Camera lens radial distortion correction using two-view projective invariants.

Calibration of Fisheye Cameras Based on the Viewing Sphere

Fisheye image Distortion Correction Based on Spherical Perspective Projection Constraint

A Fast Correction Algorithm for Inverse Longitude and Latitude Mapping of Fish Eye Image

Fisheye lens distortion correction using spherical perspective projection constraint

Unified Optical Distortion Correction Method for Imaging Systems Using A Concise Geometrical Transformation Model

Image Data Loss Minimized Geometric Correction for Asymmetric Distortion Fish-eye Lens

RoFIR: Robust Fisheye Image Rectification Framework Impervious to Optical Center Deviation

Orthorectification of Fisheye Image under Equidistant Projection Model

Unsupervised Fisheye Image Correction Through Bidirectional Loss with Geometric Prior

Improved interpolation with sub-pixel relocation method for strong barrel distortion

Exploiting the Distortion-Semantic Interaction in Fisheye Data

DaFIR: Distortion-aware Representation Learning for Fisheye Image Rectification