Abstract:Motion segmentation from a single moving camera presents a significant challenge in the field of computer vision. This challenge is compounded by the unknown camera movements and the lack of depth information of the scene. While deep learning has shown impressive capabilities in addressing these issues, supervised models require extensive training on massive annotated datasets, and unsupervised models also require training on large volumes of unannotated data, presenting significant barriers for both. In contrast, traditional methods based on optical flow do not require training data, however, they often fail to capture object-level information, leading to over-segmentation or under-segmentation. In addition, they also struggle in complex scenes with substantial depth variations and non-rigid motion, due to the overreliance of optical flow. To overcome these challenges, we propose an innovative hybrid approach that leverages the advantages of both deep learning methods and traditional optical flow based methods to perform dense motion segmentation without requiring any training. Our method initiates by automatically generating object proposals for each frame using foundation models. These proposals are then clustered into distinct motion groups using both optical flow and relative depth maps as motion cues. The integration of depth maps derived from state-of-the-art monocular depth estimation models significantly enhances the motion cues provided by optical flow, particularly in handling motion parallax issues. Our method is evaluated on the DAVIS-Moving and YTVOS-Moving datasets, and the results demonstrate that our method outperforms the best unsupervised method and closely matches with the state-of-theart supervised methods.

A Novel Automatic Motion Segmentation Method Based on Optical Flow

Motionobject segmentation algorithm based on spatial-temporal information

An Efficient Optical Flow Based Motion Detection Method for Non-stationary Scenes

A Novel Moving Object Detection Algorithm Based on Robust Image Feature Threshold Segmentation with Improved Optical Flow Estimation

Flow2seg: Motion-Aided Semantic Segmentation

Multi-Motion Segmentation: Combining Geometric Model-Fitting and Optical Flow for RGB Sensors

Dense Monocular Motion Segmentation Using Optical Flow and Pseudo Depth Map: A Zero-Shot Approach

An effective motion object detection method using optical flow estimation under a moving camera

Research on Motion Estimation Algorithm in Image Segmentation

Optical Flow Estimation from Layered Nearest Neighbor Flow Fields.

Abnormal Behavior Detection Based on Regional Optical Flow

Contour Based Automatic Scene Segmentation in Image Sequences

Automatic Dynamic Texture Segmentation Using Local Descriptors and Optical Flow

Moving Object Segmentation: All You Need Is SAM (and Flow)

Motion Object Detection Based on Optical Flow Field and Level Set

Direct Method for Motion Estimation and Structure Reconstruction Based on Optical Flow

Optical Flow Computation Based On Bayesian Selecting Scheme

Optical Flow Based Real-time Moving Object Detection in Unconstrained Scenes.

Optical Flow in Mostly Rigid Scenes

&Lt;title>semi-Automatic Video Object Segmentation Based on Hierarchy Optical Flow</title>

Three-dimensional motion estimation by straight-line and endpoint optical flow