Abstract:End-to-end autonomous driving has made impressive progress in recent years. Existing methods usually adopt the decoupled encoder-decoder paradigm, where the encoder extracts hidden features from raw sensor data, and the decoder outputs the ego-vehicle's future trajectories or actions. Under such a paradigm, the encoder does not have access to the intended behavior of the ego agent, leaving the burden of finding out safety-critical regions from the massive receptive field and inferring about future situations to the decoder. Even worse, the decoder is usually composed of several simple multi-layer perceptrons (MLP) or GRUs while the encoder is delicately designed (e.g., a combination of heavy ResNets or Transformer). Such an imbalanced resource-task division hampers the learning process. In this work, we aim to alleviate the aforementioned problem by two principles: (1) fully utilizing the capacity of the encoder; (2) increasing the capacity of the decoder. Concretely, we first predict a coarse-grained future position and action based on the encoder features. Then, conditioned on the position and action, the future scene is imagined to check the ramification if we drive accordingly. We also retrieve the encoder features around the predicted coordinate to obtain fine-grained information about the safety-critical region. Finally, based on the predicted future and the retrieved salient feature, we refine the coarse-grained position and action by predicting its offset from ground-truth. The above refinement module could be stacked in a cascaded fashion, which extends the capacity of the decoder with spatial-temporal prior knowledge about the conditioned future. We conduct experiments on the CARLA simulator and achieve state-of-the-art performance in closed-loop benchmarks. Extensive ablation studies demonstrate the effectiveness of each proposed module.

Secure Encoded Instruction Graphs for End-to-End Data Validation in Autonomous Robots

Learning Observation-Based Certifiable Safe Policy for Decentralized Multi-Robot Navigation

A Computer Vision for Navigation of Mobile Robots

Key-Scan-Based Mobile Robot Navigation: Integrated Mapping, Planning, and Control using Graphs of Scan Regions

Formal Verification of Obstacle Avoidance and Navigation of Ground Robots

Semantically Safe Robot Manipulation: From Semantic Scene Understanding to Motion Safeguards

Safe and Efficient Navigation in Extreme Environments using Semantic Belief Graphs

Closing the Perception-Action Loop for Semantically Safe Navigation in Semi-Static Environments

Semantic Belief Behavior Graph: Enabling Autonomous Robot Inspection in Unknown Environments

Asymmetric Information Enhanced Mapping Framework for Multirobot Exploration based on Deep Reinforcement Learning

An Evidential Filter for Indoor Navigation of a Mobile Robot in Dynamic Environment

Survey on Embedding Economical Autonomous Navigation System for Mobile Robots and UAV

Toward Fully Automated Inspection of Critical Assets Supported by Autonomous Mobile Robots, Vision Sensors, and Artificial Intelligence

Think Twice Before Driving: Towards Scalable Decoders for End-to-End Autonomous Driving

Safe Networked Robotics with Probabilistic Verification

Learning Autonomous Navigation in Unmapped and Unknown Environments

Robot Navigation in Unseen Spaces using an Abstract Map

Autonomous Mobile Robot Navigation in Uneven and Unstructured Indoor Environments

Securing Autonomous Service Robots through Fuzzing, Detection, and Mitigation

SafeEmbodAI: a Safety Framework for Mobile Robots in Embodied AI Systems

Updating Robot Safety Representations Online from Natural Language Feedback