Abstract:This paper proposes a strategy for visual prediction in the context of autonomous driving. Humans, when not distracted or drunk, are still the best drivers you can currently find. For this reason we take inspiration from two theoretical ideas about the human mind and its neural organization. The first idea concerns how the brain uses a hierarchical structure of neuron ensembles to extract abstract concepts from visual experience and code them into compact representations. The second idea suggests that these neural perceptual representations are not neutral but functional to the prediction of the future state of affairs in the environment. Similarly, the prediction mechanism is not neutral but oriented to the current planning of a future action. We identify within the deep learning framework two artificial counterparts of the aforementioned neurocognitive theories. We find a correspondence between the first theoretical idea and the architecture of convolutional autoencoders, while we translate the second theory into a training procedure that learns compact representations which are not neutral but oriented to driving tasks, from two distinct perspectives. From a static perspective, we force groups of neural units in the compact representations to distinctly represent specific concepts crucial to the driving task. From a dynamic perspective, we encourage the compact representations to be predictive of how the current road scenario will change in the future. We successfully learn compact representations that use as few as 16 neural units for each of the two basic driving concepts we consider: car and lane. We prove the efficiency of our proposed perceptual representations on the SYNTHIA dataset. Our source code is available at <a class="link-external link-https" href="https://github.com/3lis/rnn_vae" rel="external noopener nofollow">this https URL</a>

Foresee: Attentive Future Projections of Chaotic Road Environments with Online Training

Dynamic Environment Prediction in Urban Scenes using Recurrent Representation Learning

Predicting Future Lane Changes of Other Highway Vehicles using RNN-based Deep Models

Real-time forecasting of driver-vehicle dynamics on 3D roads: A deep-learning framework leveraging Bayesian optimisation

Looking Ahead: Anticipating Pedestrians Crossing with Future Frames Prediction

Probabilistic vehicle trajectory prediction over occupancy grid map via recurrent neural network

An End-to-End Vehicle Trajcetory Prediction Framework

Jointy Predicting Future Sequence And Steering Angles For Dynamic Driving Scenes

Improving Movement Predictions of Traffic Actors in Bird's-Eye View Models using GANs and Differentiable Trajectory Rasterization

PredictionNet: Real-Time Joint Probabilistic Traffic Prediction for Planning, Control, and Simulation

Deep learning based predicting urban traffic congestion with RGB-coded images using GRU-CNN and LSTM

INFER: INtermediate representations for FuturE pRediction

An End-to-End Future Frame Prediction Method for Vehicle-Centric Driving Videos

Traffic Flow Forecast of Road Networks with Recurrent Neural Networks

Regularizing Neural Networks for Future Trajectory Prediction via Inverse Reinforcement Learning Framework

Epurate-Net: Efficient Progressive Uncertainty Refinement Analysis for Traffic Environment Urban Road Detection

"If you could see me through my eyes": Predicting Pedestrian Perception

On the Road with 16 Neurons: Mental Imagery with Bio-inspired Deep Neural Networks

Spatiotemporal Dynamic Forecasting and Analysis of Regional Traffic Flow in Urban Road Networks Using Deep Learning Convolutional Neural Network

Deep learning-based vehicle trajectory prediction based on generative adversarial network for autonomous driving applications

Forecasting Trajectory and Behavior of Road-Agents Using Spectral Clustering in Graph-LSTMs