Abstract:Building 3D geometric maps of man-made spaces is a well-established and active field that is fundamental to computer vision and robotics. However, considering the evolving nature of built environments, it is essential to question the capabilities of current mapping efforts in handling temporal changes. In addition, spatiotemporal mapping holds significant potential for achieving sustainability and circularity goals. Existing mapping approaches focus on small changes, such as object relocation or self-driving car operation; in all cases where the main structure of the scene remains fixed. Consequently, these approaches fail to address more radical changes in the structure of the built environment, such as geometry and topology. To this end, we introduce the Nothing Stands Still (NSS) benchmark, which focuses on the spatiotemporal registration of 3D scenes undergoing large spatial and temporal change, ultimately creating one coherent spatiotemporal map. Specifically, the benchmark involves registering two or more partial 3D point clouds (fragments) from the same scene but captured from different spatiotemporal views. In addition to the standard pairwise registration, we assess the multi-way registration of multiple fragments that belong to any temporal stage. As part of NSS, we introduce a dataset of 3D point clouds recurrently captured in large-scale building indoor environments that are under construction or renovation. The NSS benchmark presents three scenarios of increasing difficulty, to quantify the generalization ability of point cloud registration methods over space (within one building and across buildings) and time. We conduct extensive evaluations of state-of-the-art methods on NSS. The results demonstrate the necessity for novel methods specifically designed to handle large spatiotemporal changes. The homepage of our benchmark is at http://nothing-stands-still.com.

Semantic-aware Spatiotemporal Alignment of Natural Outdoor Surveys

Spatiotemporal Alignment of Multi-Sensor Aerial Video Sequences

Mapping in unstructured natural environment: a sensor fusion framework for wearable sensor suites

Object-aware Semantic Mapping of Indoor Scenes Using Octomap

Spatio-Temporal Video Segmentation of Static Scenes and Its Applications

Aligning Across Large Gaps in Time

Real-Time Globally Consistent 3D Reconstruction with Semantic Priors

Deep Soft Procrustes for Markerless Volumetric Sensor Alignment

Maps from Motion (MfM): Generating 2D Semantic Maps from Sparse Multi-view Images

RTSfM: Real-Time Structure From Motion for Mosaicing and DSM Mapping of Sequential Aerial Images With Low Overlap

Hierarchical Context Alignment with Disentangled Geometric and Temporal Modeling for Semantic Occupancy Prediction

OpenPifPaf: Composite Fields for Semantic Keypoint Detection and Spatio-Temporal Association

Visual-Inertial Odometry-enhanced Geometrically Stable ICP for Mapping Applications using Aerial Robots

Real-time Image-based 6-DOF Localization in Large-Scale Environments

High-resolution Ecosystem Mapping in Repetitive Environments Using Dual Camera SLAM

Semantic Monocular SLAM for Highly Dynamic Environments

Semi-Dense 3D Semantic Mapping from Monocular SLAM

Solving Short-Term Relocalization Problems In Monocular Keyframe Visual SLAM Using Spatial And Semantic Data

DepthSSC: Depth-Spatial Alignment and Dynamic Voxel Resolution for Monocular 3D Semantic Scene Completion

Nothing Stands Still: A Spatiotemporal Benchmark on 3D Point Cloud Registration Under Large Geometric and Temporal Change

Probabilistic Data Association for Semantic SLAM at Scale