Abstract:This article primarily focuses on the localization and extraction of multiple moving objects in images taken from a moving camera platform, such as image sequences captured by drones. The positions of moving objects in the images are influenced by both the camera's motion and the movement of the objects themselves, while the background position in the images is related to the camera's motion. The main objective of this article was to extract all moving objects from the background in an image. We first constructed a motion feature space containing motion distance and direction, to map the trajectories of feature points. Subsequently, we employed a clustering algorithm based on trajectory distinctiveness to differentiate between moving objects and the background, as well as feature points corresponding to different moving objects. The pixels between the feature points were then designated as source points. Within local regions, complete moving objects were segmented by identifying these pixels. We validated the algorithm on some sequences in the Video Verification of Identity (VIVID) program database and compared it with relevant algorithms. The experimental results demonstrated that, in the test sequences when the feature point trajectories exceed 10 frames, there was a significant difference in the feature space between the feature points on the moving objects and those on the background. Correctly classified frames with feature points accounted for 67% of the total frames.The positions of the moving objects in the images were accurately localized, with an average IOU value of 0.76 and an average contour accuracy of 0.57. This indicated that our algorithm effectively localized and segmented the moving objects in images captured by moving cameras.

What problem does this paper attempt to address?

### Problems Addressed by the Paper The paper mainly focuses on the problem of locating and extracting multiple moving objects from mobile camera platforms (such as image sequences captured by drones). Specifically, the goal of the paper is to extract all moving objects from the background. This task is challenging because both the movement of the camera and the movement of the objects themselves affect the positions of moving objects in the images, while the background position is related to the camera's movement. ### Solution Overview 1. **Constructing Motion Feature Space**: - The authors first constructed a motion feature space that includes motion distance and direction to map the trajectories of feature points. 2. **Clustering Algorithm**: - A clustering algorithm based on the uniqueness of trajectories is used to distinguish between moving objects and the background, as well as the feature points corresponding to different moving objects. 3. **Local Area Segmentation**: - Within local areas, the complete segmentation of moving objects is achieved by identifying the pixels between these feature points. ### Experimental Validation - The authors validated the algorithm on certain sequences in the Video Verification of Identity (VIVID) program database and compared it with related algorithms. - Experimental results show that in the test sequences, when the feature point trajectories exceed 10 frames, there is a significant difference in the feature space between the feature points on moving objects and those on the background. - Correctly classified frames account for 67% of the total frames, the positions of moving objects in the images are accurately located, with an average IOU value of 0.76 and an average contour accuracy of 0.57. ### Main Contributions 1. **Dual-Threshold DBSCAN Algorithm**: - Used for temporal classification and distinguishing moving objects with inconsistent motion. 2. **Iterative Algorithm Based on Sample Mean and Variance**: - Used for distinguishing spatial objects. 3. **Improved Two-Step Contour Generation Algorithm**: - Used for extracting the contours of objects. ### Conclusion The method proposed in this paper effectively solves the problem of locating and segmenting multiple moving objects in images captured by mobile camera platforms, performing particularly well when the feature point trajectories exceed 10 frames.

Dynamic Multiple Object Segmentation with Spatio-Temporal Filtering

Motionobject segmentation algorithm based on spatial-temporal information

Detection and Segmentation of Moving Objects Using Temporal and Spatial Cues

Automatic Extraction Of Moving Objects In Video Sequences Based On Spatio-Temporal Information

Automatic Extraction of Video Object Plane by Integrating Spatio-Temporal Information

Detecting and Tracking Dynamic Objects in Complex Environments

Motion Objects Segmentation Using a New Level Set Based Method

Motion Segmentation from a Moving Monocular Camera

Moving Object Extraction with a Hand-held Camera

Spatio-Temporal Video Segmentation of Static Scenes and Its Applications

Detecting and segmenting multiple moving objects using level-set method

Automatic Segmentation of Moving Objects in Video Sequences Based on Dynamic Background Construction

An automatic segmentation algorithm for moving objects in video sequences under multi-constraints

Video-object segmentation and 3D-trajectory estimation for monocular video sequences

A Novel Moving Object Detection Algorithm Based on Robust Image Feature Threshold Segmentation with Improved Optical Flow Estimation

Automatic Extraction of Moving Objects Using Multiple Features and Multiple Frames

A Moving Object Segmentation Technique Using Dynamic Programming

A Spatiotemporal Multiple Moving Objects Segmentation and Tracking with Level Set

Improved Video Object Segmentation Algorithm Based on Temporal-Spatial Information

Detection and Segmentation of Moving Objects Based on Support Vector Machine

Detecting moving objects under a moving camera in complex environments