Abstract:Mobile robots rely on maps to navigate through an environment. In the absence of any map, the robots must build the map online from partial observations as they move in the environment. Traditional methods build a map using only direct observations. In contrast, humans identify patterns in the observed environment and make informed guesses about what to expect ahead. Modeling these patterns explicitly is difficult due to the complexity of the environments. However, these complex models can be approximated well using learning-based methods in conjunction with large training data. By extracting patterns, robots can use direct observations and predictions of what lies ahead to better navigate an unknown environment. In this dissertation, we present several learning-based methods to equip mobile robots with prediction capabilities for efficient and safer operation. In the first part of the dissertation, we learn to predict using geometrical and structural patterns in the environment. Partially observed maps provide invaluable cues for accurately predicting the unobserved areas. We first demonstrate the capability of general learning-based approaches to model these patterns for a variety of overhead map modalities. Then we employ task-specific learning for faster navigation in indoor environments by predicting 2D occupancy in the nearby regions. This idea is further extended to 3D point cloud representation for object reconstruction. Predicting the shape of the full object from only partial views, our approach paves the way for efficient next-best-view planning. In the second part of the dissertation, we learn to predict using spatiotemporal patterns in the environment. We focus on dynamic tasks such as target tracking and coverage where we seek decentralized coordination between robots. We first show how graph neural networks can be used for more scalable and faster inference.

Augmented Environment Representations with Complete Object Models

Object-aware Semantic Mapping of Indoor Scenes Using Octomap

Extending Maps with Semantic and Contextual Object Information for Robot Navigation: a Learning-Based Framework using Visual and Depth Cues

Object-Oriented 3D Semantic Mapping Based on Instance Segmentation

Multimodal sensor-based semantic 3D mapping for a large-scale environment

An Approach for Construct Semantic Map with Scene Classification and Object Semantic Segmentation

Volumetric Instance-Aware Semantic Mapping and 3D Object Discovery

Efficient and Robust Semantic Mapping for Indoor Environments

Mapping High-level Semantic Regions in Indoor Environments without Object Recognition

Hybrid Topological and 3D Dense Mapping through Autonomous Exploration for Large Indoor Environments

Indoor Semantic Scene Understanding using Multi-modality Fusion

Object-Oriented Semantic Mapping for Reliable UAVs Navigation

Language-guided Semantic Mapping and Mobile Manipulation in Partially Observable Environments

Artifacts Mapping: Multi-Modal Semantic Mapping for Object Detection and 3D Localization

Multi-Objective Location and Mapping Based on Deep Learning and Visual Slam

Enhancing Graph Representation of the Environment through Local and Cloud Computation

Multi-LiDAR Mapping for Scene Segmentation in Indoor Environments for Mobile Robots

Real-time 3D Semantic Scene Perception for Egocentric Robots with Binocular Vision

Semi-Dense 3D Semantic Mapping from Monocular SLAM

A semantic SLAM-based dense mapping approach for large-scale dynamic outdoor environment

Enhanced Robot Planning and Perception through Environment Prediction