Abstract:Thanks to the wide view field, the fisheye camera can get much more visual information. Thus, it is widely used in the field of computer vision. However, projection is often required for fisheye images to be used for object detection. Meanwhile, the projection will lead to distortion in fisheye images, and the discontinuous image edges will make the objects incomplete. Fisheye images are characterized by objects that are large near and small far. These problems are still challenges for the existing advanced object detector YOLOv7. Therefore, in this paper, we propose an improved YOLOv7 model. First, Modulated Deformable Convolution is introduced into the YOLOv7 model to automatically adapt to distortion changes of distorted objects in fisheye images. It not only adjusts the sampling position of the convolutional kernel but also further extends the deformation range. The improved model can efficiently extract features of distorted and edge-discontinuous objects. In addition, fisheye images are characterized by objects close to the fisheye lens being large, while objects farther away from the fisheye lens will be smaller. To further optimize the detection performance of small objects in fisheye images, Swin Transformer is also introduced into the YOLOv7 model, and Swin Transformer Block with Window Multi-head Self-Attention (W-MSA) Effectively enhances Network Local Perception. Finally, our proposed model achieves up to 2.4% improvement in mAP compared to the original YOLOv7 model on the ERP-360 dataset. Also, the proposed model achieves the best results compared to other state-of-the-art object detection methods for equirectangular projection images. On the VOC-360 dataset, our proposed model improves the mAP by up to 5.9% compared to the original YOLOv7 model. The experimental results show that the proposed models achieve good results for object detection in both fisheye images and equirectangular projection images. The ERP-360 dataset, source code and pre-trained models for related tasks can be found at https://github.com/xiaoxiaomichong/ERP-360dataset.

Improved Face Detector on Fisheye Images Via Spherical-Domain Attention

Object Detection and Localization in 3D Environment by Fusing Raw Fisheye Image and Attitude Data

Efficient Face Detection in the Fisheye Image Domain

Downside Hemisphere Object Detection and Localization of MAV by Fisheye Camera

Orientation-aware People Detection and Counting Method Based on Overhead Fisheye Camera

Improved YOLOv7 models based on modulated deformable convolution and swin transformer for object detection in fisheye images

Fish-eye image correction method and technique based on geometrical imaging model

Swin‐fisheye: Object detection for fisheye images

FishFormer: Annulus Slicing-based Transformer for Fisheye Rectification with Efficacy Domain Exploration

Two distortion correcting methods for fisheye images

FisheyeDetNet: 360° Surround view Fisheye Camera based Object Detection System for Autonomous Driving

Fisheye Object Detection Based on Standard Image Datasets with 24-Points Regression Strategy

A Stronger Stitching Algorithm for Fisheye Images based on Deblurring and Registration

Robust Keypoint Detection and Matching on Fisheye Images by Self-Supervised Learning.

Fisheye image Distortion Correction Based on Spherical Perspective Projection Constraint

DSA: Deformable Segmentation Attention for Multi-Scale Fisheye Image Segmentation

Location-guided Head Pose Estimation for Fisheye Image

AST: Annulus Swin Transformer for Pedestrians Detection under Top-view Fisheye Image.

Beautifying Fisheye Images Using Orientation and Shape Cues.

Extraction of Feature Point Pseudo-Neighborhood for High-Accuracy of Fisheye Image Matching

Omnidirectional Depth Estimation With Hierarchical Deep Network for Multi-Fisheye Navigation Systems