Abstract:In this work, we present a methodology to shape a fisheye-specific representation space that reflects the interaction between distortion and semantic context present in this data modality. Fisheye data has the wider field of view advantage over other types of cameras, but this comes at the expense of high radial distortion. As a result, objects further from the center exhibit deformations that make it difficult for a model to identify their semantic context. While previous work has attempted architectural and training augmentation changes to alleviate this effect, no work has attempted to guide the model towards learning a representation space that reflects this interaction between distortion and semantic context inherent to fisheye data. We introduce an approach to exploit this relationship by first extracting distortion class labels based on an object's distance from the center of the image. We then shape a backbone's representation space with a weighted contrastive loss that constrains objects of the same semantic class and distortion class to be close to each other within a lower dimensional embedding space. This backbone trained with both semantic and distortion information is then fine-tuned within an object detection setting to empirically evaluate the quality of the learnt representation. We show this method leads to performance improvements by as much as 1.1% mean average precision over standard object detection strategies and .6% improvement over other state of the art representation learning approaches.

Beautifying Fisheye Images Using Orientation and Shape Cues.

Two distortion correcting methods for fisheye images

Object Detection and Localization in 3D Environment by Fusing Raw Fisheye Image and Attitude Data

Fish-eye image correction method and technique based on geometrical imaging model

Calibration of Fisheye Cameras Based on the Viewing Sphere

A New Approach for Distorted Image Restoration by Distance-based Correction and Cubic Spline Interpolation

Downside Hemisphere Object Detection and Localization of MAV by Fisheye Camera

Fisheye image Distortion Correction Based on Spherical Perspective Projection Constraint

Improved Face Detector on Fisheye Images Via Spherical-Domain Attention

Orientation-aware People Detection and Counting Method Based on Overhead Fisheye Camera

A Comprehensive Overview of Fish-Eye Camera Distortion Correction Methods

Correction method for fisheye image based on the virtual small-field camera.

Study on Fisheye Image Correction Based on Cylinder Model

A Stronger Stitching Algorithm for Fisheye Images based on Deblurring and Registration

Exploiting the Distortion-Semantic Interaction in Fisheye Data

Orthorectification of Fisheye Image under Equidistant Projection Model

Pixel-variant Local Homography for Fisheye Stereo Rectification Minimizing Resampling Distortion

FSD-BRIEF: A Distorted BRIEF Descriptor for Fisheye Image Based on Spherical Perspective Model

Digital deformation model for fisheye image rectification.

Correction of Barrel Distortion in Fisheye Lens Images Using Image-Based Estimation of Distortion Parameters

FishEyeRecNet: A Multi-Context Collaborative Deep Network for Fisheye Image Rectification.