Abstract:Thanks to the wide view field, the fisheye camera can get much more visual information. Thus, it is widely used in the field of computer vision. However, projection is often required for fisheye images to be used for object detection. Meanwhile, the projection will lead to distortion in fisheye images, and the discontinuous image edges will make the objects incomplete. Fisheye images are characterized by objects that are large near and small far. These problems are still challenges for the existing advanced object detector YOLOv7. Therefore, in this paper, we propose an improved YOLOv7 model. First, Modulated Deformable Convolution is introduced into the YOLOv7 model to automatically adapt to distortion changes of distorted objects in fisheye images. It not only adjusts the sampling position of the convolutional kernel but also further extends the deformation range. The improved model can efficiently extract features of distorted and edge-discontinuous objects. In addition, fisheye images are characterized by objects close to the fisheye lens being large, while objects farther away from the fisheye lens will be smaller. To further optimize the detection performance of small objects in fisheye images, Swin Transformer is also introduced into the YOLOv7 model, and Swin Transformer Block with Window Multi-head Self-Attention (W-MSA) Effectively enhances Network Local Perception. Finally, our proposed model achieves up to 2.4% improvement in mAP compared to the original YOLOv7 model on the ERP-360 dataset. Also, the proposed model achieves the best results compared to other state-of-the-art object detection methods for equirectangular projection images. On the VOC-360 dataset, our proposed model improves the mAP by up to 5.9% compared to the original YOLOv7 model. The experimental results show that the proposed models achieve good results for object detection in both fisheye images and equirectangular projection images. The ERP-360 dataset, source code and pre-trained models for related tasks can be found at https://github.com/xiaoxiaomichong/ERP-360dataset.

An adversarial pedestrian detection model based on virtual fisheye image training

Object Detection and Localization in 3D Environment by Fusing Raw Fisheye Image and Attitude Data

Downside Hemisphere Object Detection and Localization of MAV by Fisheye Camera

Pedestrian Detection Method Based on Improved YOLOv5s for Densely Occluded Scenarios

Pedestrian Feature Generation in Fish-Eye Images Via Adversary.

Spatial Focal Loss for Pedestrian Detection in Fisheye Imagery

Orientation-aware People Detection and Counting Method Based on Overhead Fisheye Camera

Few-Shot Supervised Prototype Alignment for Pedestrian Detection on Fisheye Images

Oriented Spatial Transformer Network for Pedestrian Detection Using Fish-Eye Camera

Adversarial Training-Based Hard Example Mining for Pedestrian Detection in Fish-Eye Images

Improved YOLOv7 models based on modulated deformable convolution and swin transformer for object detection in fisheye images

Self-Adapting Part-Based Pedestrian Detection Using A Fish-Eye Camera

Streaming Object Detection on Fisheye Cameras for Automatic Parking

Moving Object Detection Using an In-Vehicle Fish-Eye Camera

A Detection and Tracking System for Fisheye Videos from Traffic Intersections

Fisheye Object Detection Based on Standard Image Datasets with 24-Points Regression Strategy

OARPD: occlusion-aware rotated people detection in overhead fisheye images

Improved Face Detector on Fisheye Images Via Spherical-Domain Attention

FishEye8K: A Benchmark and Dataset for Fisheye Camera Object Detection

Towards Oriented Fisheye Object Detection: Dataset and Baseline

FisheyePP4AV: A privacy-preserving method for autonomous vehicles on fisheye camera images