Abstract:Motion segmentation from a single moving camera presents a significant challenge in the field of computer vision. This challenge is compounded by the unknown camera movements and the lack of depth information of the scene. While deep learning has shown impressive capabilities in addressing these issues, supervised models require extensive training on massive annotated datasets, and unsupervised models also require training on large volumes of unannotated data, presenting significant barriers for both. In contrast, traditional methods based on optical flow do not require training data, however, they often fail to capture object-level information, leading to over-segmentation or under-segmentation. In addition, they also struggle in complex scenes with substantial depth variations and non-rigid motion, due to the overreliance of optical flow. To overcome these challenges, we propose an innovative hybrid approach that leverages the advantages of both deep learning methods and traditional optical flow based methods to perform dense motion segmentation without requiring any training. Our method initiates by automatically generating object proposals for each frame using foundation models. These proposals are then clustered into distinct motion groups using both optical flow and relative depth maps as motion cues. The integration of depth maps derived from state-of-the-art monocular depth estimation models significantly enhances the motion cues provided by optical flow, particularly in handling motion parallax issues. Our method is evaluated on the DAVIS-Moving and YTVOS-Moving datasets, and the results demonstrate that our method outperforms the best unsupervised method and closely matches with the state-of-theart supervised methods.

Semi-dense Motion Segmentation for Moving Cameras by Discrete Energy Minimization

Motion Segmentation from a Moving Monocular Camera

Dense Monocular Motion Segmentation Using Optical Flow and Pseudo Depth Map: A Zero-Shot Approach

Contour Based Automatic Scene Segmentation in Image Sequences

Spatio-Temporal Video Segmentation of Static Scenes and Its Applications

Motion Objects Segmentation Using a New Level Set Based Method

A Spatiotemporal Multiple Moving Objects Segmentation and Tracking with Level Set

Spatio-Temporal Segmentation with Depth-Inferred Videos of Static Scenes

Motion Segmentation Via a Sparsity Constraint

Motion Detection for Rapidly Moving Cameras in Fully 3D Scenes

Multi-Motion Segmentation: Combining Geometric Model-Fitting and Optical Flow for RGB Sensors

A Spatial-Motion-Segmentation Algorithm by Fusing EDPA and Motion Compensation.

Spatiotemporal Strategies for Joint Segmentation and Motion Tracking from Cardiac Image Sequences

Event Camera-based Motion Segmentation Via Depth Estimation and 3D Motion Compensation

Motion Segmentation with Event Camera: N-patches Optical Flow Estimation and Pairwise Markov Random Fields

Contour-Based Motion Segmentation Using Few Priors

3D Video Based Segmentation and Motion Estimation with Active Surface Evolution

A Novel Automatic Motion Segmentation Method Based on Optical Flow

A New Approach To Two-View Motion Segmentation Using Global Dimension Minimization

Motionrfcn: Motion Segmentation Using Consecutive Dense Depth Maps

Motion Segmentation Via Robust Subspace Separation in the Presence of Outlying, Incomplete, or Corrupted Trajectories