Abstract:Panoramic cycling videos can record 360° views around the cyclists. Thus, it is essential to conduct automatic road user analysis on them using computer vision models to provide data for studies on cycling safety. However, the features of panoramic data such as severe distortions, large number of small objects and boundary continuity have brought great challenges to the existing CV models, including poor performance and evaluation methods that are no longer applicable. In addition, due to the lack of data with annotations, it is not easy to re-train the models. In response to these problems, the project proposed and implemented a three-step methodology: (1) improve the prediction performance of the pre-trained object detection models on panoramic data by projecting the original image into 4 perspective sub-images; (2) introduce supports for boundary continuity and category information into DeepSORT, a commonly used multiple object tracking model, and set an improved detection model as its detector; (3) using the tracking results, develop an application for detecting the overtaking behaviour of the surrounding vehicles. Evaluated on the panoramic cycling dataset built by the project, the proposed methodology improves the average precision of YOLO v5m6 and Faster RCNN-FPN under any input resolution setting. In addition, it raises MOTA and IDF1 of DeepSORT by 7.6\% and 9.7\% respectively. When detecting the overtakes in the test videos, it achieves the F-score of 0.88. The code is available on GitHub at <a class="link-external link-http" href="http://github.com/cuppp1998/360_object_tracking" rel="external noopener nofollow">this http URL</a> to ensure the reproducibility and further improvements of results.

Object Detection on Panoramic Images Based on Deep Learning

Object Detection and Localization in 3D Environment by Fusing Raw Fisheye Image and Attitude Data

Real-Time Object Detection for 360-Degree Panoramic Image Using CNN

Downside Hemisphere Object Detection and Localization of MAV by Fisheye Camera

Aerial-PASS: Panoramic Annular Scene Segmentation in Drone Videos

Omnisupervised Omnidirectional Semantic Segmentation

Learning Feature Transforms for Object Detection from Panoramic Images

3D Object Detection Algorithm for Panoramic Images with Multi-Scale Convolutional Neural Network.

DeepPanoContext: Panoramic 3D Scene Understanding with Holistic Scene Context Graph and Relation-based Optimization

PHD: A Deep Learning Based Human Detection Framework for Panoramic Videos

PADENet: an Efficient and Robust Panoramic Monocular Depth Estimation Network for Outdoor Scenes.

Applications of Deep Learning for Top-View Omnidirectional Imaging: A Survey

Universal Object Detection with Large Vision Model

Panoramic depth estimation via supervised and unsupervised learning in indoor scenes

A Panoramic Video Face Detection System Design and Implement

Automatic Salient Object Detection for Panoramic Images Using Region Growing and Fixation Prediction Model

Deep learning based object detection from multi-modal sensors: an overview

What's in my Room? Object Recognition on Indoor Panoramic Images

Multiple Object Detection and Tracking in Panoramic Videos for Cycling Safety Analysis

3D Object Detection for Point Cloud in Virtual Driving Environment

Fully Convolutional One-Stage 3D Object Detection on LiDAR Range Images