Abstract:End-to-end autonomous driving has made impressive progress in recent years. Existing methods usually adopt the decoupled encoder-decoder paradigm, where the encoder extracts hidden features from raw sensor data, and the decoder outputs the ego-vehicle's future trajectories or actions. Under such a paradigm, the encoder does not have access to the intended behavior of the ego agent, leaving the burden of finding out safety-critical regions from the massive receptive field and inferring about future situations to the decoder. Even worse, the decoder is usually composed of several simple multi-layer perceptrons (MLP) or GRUs while the encoder is delicately designed (e.g., a combination of heavy ResNets or Transformer). Such an imbalanced resource-task division hampers the learning process. In this work, we aim to alleviate the aforementioned problem by two principles: (1) fully utilizing the capacity of the encoder; (2) increasing the capacity of the decoder. Concretely, we first predict a coarse-grained future position and action based on the encoder features. Then, conditioned on the position and action, the future scene is imagined to check the ramification if we drive accordingly. We also retrieve the encoder features around the predicted coordinate to obtain fine-grained information about the safety-critical region. Finally, based on the predicted future and the retrieved salient feature, we refine the coarse-grained position and action by predicting its offset from ground-truth. The above refinement module could be stacked in a cascaded fashion, which extends the capacity of the decoder with spatial-temporal prior knowledge about the conditioned future. We conduct experiments on the CARLA simulator and achieve state-of-the-art performance in closed-loop benchmarks. Extensive ablation studies demonstrate the effectiveness of each proposed module.

A NEW TRAINING ALGORITHM OF AUTOENCODER COMBINING LATERAL INHIBITION MECHANISM

A Label Noise Robust Stacked Auto-Encoder Algorithm for Inaccurate Supervised Classification Problems

An Efficient Learning Algorithm for Direct Training Deep Spiking Neural Networks

Improved AutoEncoder with LSTM module and KL divergence

An informative dual ForkNet for video anomaly detection

Analyzing noise in autoencoders and deep networks

Closed-Loop Transcription via Convolutional Sparse Coding

Sparse Code Formation with Linear Inhibition

Leaky Training: A Robustness Improvement Scheme for Autoencoder Based Network Intrusion Detection System under Online Learning

Diversity encouraged learning of unsupervised LSTM ensemble for neural activity video prediction

A New Modal Autoencoder for Functionally Independent Feature Extraction

A New Training Principle for Stacked Denoising Autoencoders

EncodeNet: A Framework for Boosting DNN Accuracy with Entropy-driven Generalized Converting Autoencoder

Robust and structural sparsity auto-encoder with L21-norm minimization

Think Twice Before Driving: Towards Scalable Decoders for End-to-End Autonomous Driving

A comprehensive survey on design and application of autoencoder in deep learning

Adaptive ProductAE: SNR-Aware Adaptive Decoding of Neural Product Codes

AENEA: A novel autoencoder-based network embedding algorithm

Wi-Fi Signal Noise Reduction And Multipath Elimination Based On Autoencoder

Do not Interfere but Cooperate: A Fully Learnable Code Design for Multi-Access Channels with Feedback

Deep Learning of Nonnegativity-Constrained Autoencoders for Enhanced Understanding of Data