Abstract:Moving object segmentation (MOS) in dynamic scenes is an important, challenging, but under-explored research topic for autonomous driving, especially for sequences obtained from moving ego vehicles. Most segmentation methods leverage motion cues obtained from optical flow maps. However, since these methods are often based on optical flows that are pre-computed from successive RGB frames, this neglects the temporal consideration of events occurring within the inter-frame, consequently constraining its ability to discern objects exhibiting relative staticity but genuinely in motion. To address these limitations, we propose to exploit event cameras for better video understanding, which provide rich motion cues without relying on optical flow. To foster research in this area, we first introduce a novel large-scale dataset called DSEC-MOS for moving object segmentation from moving ego vehicles, which is the first of its kind. For benchmarking, we select various mainstream methods and rigorously evaluate them on our dataset. Subsequently, we devise EmoFormer, a novel network able to exploit the event data. For this purpose, we fuse the event temporal prior with spatial semantic maps to distinguish genuinely moving objects from the static background, adding another level of dense supervision around our object of interest. Our proposed network relies only on event data for training but does not require event input during inference, making it directly comparable to frame-only methods in terms of efficiency and more widely usable in many application cases. The exhaustive comparison highlights a significant performance improvement of our method over all other methods. The source code and dataset are publicly available at: <a class="link-external link-https" href="https://github.com/ZZY-Zhou/DSEC-MOS" rel="external noopener nofollow">this https URL</a>.

SSF-MOS: Semantic Scene Flow Assisted Moving Object Segmentation for Autonomous Vehicles

Motionobject segmentation algorithm based on spatial-temporal information

MF-MOS: A Motion-Focused Model for Moving Object Segmentation

3D-SeqMOS: A Novel Sequential 3D Moving Object Segmentation in Autonomous Driving

FlowMOT: 3D Multi-Object Tracking by Scene Flow Association

Event-Free Moving Object Segmentation from Moving Ego Vehicle

CV-MOS: A Cross-View Model for Motion Segmentation

MambaMOS: LiDAR-based 3D Moving Object Segmentation with Motion-aware State Space Model

MV-MOS: Multi-View Feature Fusion for 3D Moving Object Segmentation

TOSS: Real-time Tracking and Moving Object Segmentation for Static Scene Mapping

Efficient Spatial-Temporal Information Fusion for LiDAR-Based 3D Moving Object Segmentation

Multi-Motion Segmentation: Combining Geometric Model-Fitting and Optical Flow for RGB Sensors

Flow2seg: Motion-Aided Semantic Segmentation

StreamMOS: Streaming Moving Object Segmentation with Multi-View Perception and Dual-Span Memory

Real-time Moving Object Segmentation with Tracking and Tracklet Belief

Semantics-Guided Moving Object Segmentation with 3D LiDAR

Fast Multi-frame Stereo Scene Flow with Motion Segmentation

Moving Object Segmentation: All You Need Is SAM (and Flow)

SeFlow: A Self-Supervised Scene Flow Method in Autonomous Driving

Semantic geometric fusion multi-object tracking and lidar odometry in dynamic environment