Abstract:Modern autonomous driving techniques harness a variety of visual cues for performing key tasks like vehicle steering and collision avoidance. This work defines for the first time and proposes a solution to the following problem: can one precisely estimate an autonomous vehicle’s orientation relative to the road by taking a picture from within the vehicle? Vehicle orientation supplies crucial information for making various driving-related decisions. The key challenge of this problem is the lack of training data, since accurately annotating vehicle’s orientation information in an image is in general difficult. To circumvent the data scarcity issue, we propose to leverage publicly-available street panoramas as those found in GoogleMaps. Street panoramas are essentially defined by 2-D plenoptic functions that map the visual environment to a sphere. One can thus effortlessly collect annotated data for the interested task by specifying a direction and reading out corresponding pixels from a panorama. This way we collect a city-scale large benchmark TB500K for studying this problem. Our second contribution is the exposition of a two-stream deep network for attacking this problem. Intuitively, visual/semantic and geometric (e.g., the direction of road lanes) cues are both crucial and complement each other. Our proposed two-stream network can jointly learn from both cues and serves as a reasonable baseline for this novel problem. In addition, several experiments are also conducted to reveal key factors in estimating the vehicle orientation.

Better Road Following By Integrating Omni-View Images And Neural Nets

Omnidirectional Vision for Road Understanding *

Fast Road Classification and Orientation Estimation Using Omni-View Images and Neural Networks.

Combining Rotation-Invariance Images and Neural Networks for Road Scene Understanding

Design of an Enhanced Visual Odometry by Building and Matching Compressive Panoramic Landmarks Online

Realization of odometry system using monocular vision

Integrating Visual and Range Data for Road Detection.

A Computer Vision for Navigation of Mobile Robots

Unifying Terrain Awareness Through Real-Time Semantic Segmentation

Scene-Adaptive Off-Road Detection Using a Monocular Camera

Vision-Based Road Detection using Contextual Blocks

Unstructured Road Edge Detection and Initial Positioning Approach based on Monocular Vision

Monocular BEV Perception of Road Scenes Via Front-to-Top View Projection

Comprehensive understanding of multiple scale, omnidirectional spatiotemporal images for visual navigation

Robust Appearance Based Visual Route Following for Navigation in Large-scale Outdoor Environments

Sequential Monocular Road Detection by Fusing Appearance and Geometric Information

A Hybrid Approach to Real-Time Robotic Visual Navigation: Integrating Detection and Scene Segmentation

Learning from Maps: Visual Common Sense for Autonomous Driving

An Efficient Approach to Building Image-Based Road Network Model.

Learning single-shot vehicle orientation estimation from large-scale street panoramas.

Road-aware Monocular Structure from Motion and Homography Estimation