Abstract:Thanks to the wide view field, the fisheye camera can get much more visual information. Thus, it is widely used in the field of computer vision. However, projection is often required for fisheye images to be used for object detection. Meanwhile, the projection will lead to distortion in fisheye images, and the discontinuous image edges will make the objects incomplete. Fisheye images are characterized by objects that are large near and small far. These problems are still challenges for the existing advanced object detector YOLOv7. Therefore, in this paper, we propose an improved YOLOv7 model. First, Modulated Deformable Convolution is introduced into the YOLOv7 model to automatically adapt to distortion changes of distorted objects in fisheye images. It not only adjusts the sampling position of the convolutional kernel but also further extends the deformation range. The improved model can efficiently extract features of distorted and edge-discontinuous objects. In addition, fisheye images are characterized by objects close to the fisheye lens being large, while objects farther away from the fisheye lens will be smaller. To further optimize the detection performance of small objects in fisheye images, Swin Transformer is also introduced into the YOLOv7 model, and Swin Transformer Block with Window Multi-head Self-Attention (W-MSA) Effectively enhances Network Local Perception. Finally, our proposed model achieves up to 2.4% improvement in mAP compared to the original YOLOv7 model on the ERP-360 dataset. Also, the proposed model achieves the best results compared to other state-of-the-art object detection methods for equirectangular projection images. On the VOC-360 dataset, our proposed model improves the mAP by up to 5.9% compared to the original YOLOv7 model. The experimental results show that the proposed models achieve good results for object detection in both fisheye images and equirectangular projection images. The ERP-360 dataset, source code and pre-trained models for related tasks can be found at https://github.com/xiaoxiaomichong/ERP-360dataset.

Fisheye Object Detection with Visual Prompting-Aided Fine-Tuning

Object Detection and Localization in 3D Environment by Fusing Raw Fisheye Image and Attitude Data

Downside Hemisphere Object Detection and Localization of MAV by Fisheye Camera

Calibration of Fisheye Cameras Based on the Viewing Sphere

Fish Keypoints Detection for Ecology Monitoring Based on Underwater Visual Intelligence

Coarse-To-Fine Segmentation Refinement and Missing Shape Recovery for Halibut Fish

Improved YOLOv7 models based on modulated deformable convolution and swin transformer for object detection in fisheye images

Fish-eye image correction method and technique based on geometrical imaging model

Swin‐fisheye: Object detection for fisheye images

Correction of Barrel Distortion in Fisheye Lens Images Using Image-Based Estimation of Distortion Parameters

Exploiting the Distortion-Semantic Interaction in Fisheye Data

Object ground lines regression and mapping from fisheye images to around view image for the AVP

Improved Face Detector on Fisheye Images Via Spherical-Domain Attention

Efficient Face Detection in the Fisheye Image Domain

RoFIR: Robust Fisheye Image Rectification Framework Impervious to Optical Center Deviation

Two distortion correcting methods for fisheye images

Moving Object Detection Using an In-Vehicle Fish-Eye Camera

FisheyeDetNet: 360° Surround view Fisheye Camera based Object Detection System for Autonomous Driving

3D Object Detection from a Single Fisheye Image Without a Single Fisheye Training Image

An adversarial pedestrian detection model based on virtual fisheye image training

Self-supervised Interest Point Detection and Description for Fisheye and Perspective Images