Abstract:Moving object segmentation (MOS) in dynamic scenes is an important, challenging, but under-explored research topic for autonomous driving, especially for sequences obtained from moving ego vehicles. Most segmentation methods leverage motion cues obtained from optical flow maps. However, since these methods are often based on optical flows that are pre-computed from successive RGB frames, this neglects the temporal consideration of events occurring within the inter-frame, consequently constraining its ability to discern objects exhibiting relative staticity but genuinely in motion. To address these limitations, we propose to exploit event cameras for better video understanding, which provide rich motion cues without relying on optical flow. To foster research in this area, we first introduce a novel large-scale dataset called DSEC-MOS for moving object segmentation from moving ego vehicles, which is the first of its kind. For benchmarking, we select various mainstream methods and rigorously evaluate them on our dataset. Subsequently, we devise EmoFormer, a novel network able to exploit the event data. For this purpose, we fuse the event temporal prior with spatial semantic maps to distinguish genuinely moving objects from the static background, adding another level of dense supervision around our object of interest. Our proposed network relies only on event data for training but does not require event input during inference, making it directly comparable to frame-only methods in terms of efficiency and more widely usable in many application cases. The exhaustive comparison highlights a significant performance improvement of our method over all other methods. The source code and dataset are publicly available at: <a class="link-external link-https" href="https://github.com/ZZY-Zhou/DSEC-MOS" rel="external noopener nofollow">this https URL</a>.

Real-time Moving Object Segmentation with Tracking and Tracklet Belief

LiDAR Video Object Segmentation with Dynamic Kernel Refinement

Object tracking with 3D LIDAR via multi-task sparse learning

MambaMOS: LiDAR-based 3D Moving Object Segmentation with Motion-aware State Space Model

Real-time dynamic obstacle detection and tracking using 3D Lidar

3D-SeqMOS: A Novel Sequential 3D Moving Object Segmentation in Autonomous Driving

MF-MOS: A Motion-Focused Model for Moving Object Segmentation

Semantics-Guided Moving Object Segmentation with 3D LiDAR

Multi-sensor fusion for robust localization with moving object segmentation in complex dynamic 3D scenes

Efficient Spatial-Temporal Information Fusion for LiDAR-Based 3D Moving Object Segmentation

Moving Object Segmentation in 3D LiDAR Data: A Learning-based Approach Exploiting Sequential Data

LIFTS: Lidar and Monocular Image Fusion for Multi-Object Tracking and Segmentation

An Effective Motion-Centric Paradigm for 3D Single Object Tracking in Point Clouds

Object-Level Pseudo-3D Lifting for Distance-Aware Tracking

Receding Moving Object Segmentation in 3D LiDAR Data Using Sparse 4D Convolutions

Semantic geometric fusion multi-object tracking and lidar odometry in dynamic environment

Dynamic Object Tracking for Self-Driving Cars Using Monocular Camera and LIDAR.

TOSS: Real-time Tracking and Moving Object Segmentation for Static Scene Mapping

MotionBEV: Attention-Aware Online LiDAR Moving Object Segmentation with Bird's Eye View based Appearance and Motion Features

Beyond 3D Siamese Tracking: A Motion-Centric Paradigm for 3D Single Object Tracking in Point Clouds

Event-Free Moving Object Segmentation from Moving Ego Vehicle