Abstract:The safety of an automated vehicle hinges crucially upon the accuracy of perception and decision-making latency. Under these stringent requirements, future automated cars are usually equipped with multi-modal sensors such as cameras and LiDARs. The sensor fusion is adopted to provide a confident context of driving scenarios for better decision-making. A promising sensor fusion technique is middle fusion that combines the feature representations from intermediate layers that belong to different sensing modalities. However, achieving both the accuracy and latency efficiency is challenging for middle fusion, which is critical for driving automation applications. We present A3Fusion, a software-hardware system specialized for an adaptive, agile, and aligned fusion in driving automation. A3Fusion achieves a high efficiency for the middle fusion of multiple CNN-based modalities by proposing an adaptive multi-modal learning network architecture and a latency-aware, agile network architecture optimization algorithm that enhances semantic segmentation accuracy while taking the inference latency as a key trade-off. In addition, A3Fusion proposes a FPGA-based accelerator that captures unique data flow patterns of our middle fusion algorithm while reducing the overall compute overheads. We enable these contributions by co-designing the neural network, algorithm, and the accelerator architecture.

A Stable Design of the Architecture of Single Platform Multi-sensor Data Fusion System

Building A Generic Platform For Big Sensor Data Application

Distributed Sensor Layout Optimization for Target Detection with Data Fusion.

Design of campus comprehensive access management platform

A Multi-sensor Data Fusion Method Based on the Set-Pair Identity Degree and Connection Number

Survey of multisensor data fusion models

A High Performance Track Fusion System with Feedback

Architecture of Sensor Management and Simulation of Micro Sensor Management

Multi-hierarchical Blackboard Model for Communication Intercept Information Fusion

Research on Ontology-Based Data Fusion.

Target Tracking System for Multi-Sensor Data Fusion

A SVD-Based Multi-Sensor Data Fusion Algorithm

DISTRIBUTED INFORMATION FUSION ALGORITHM FOR SINGLE SENSOR DYNAMIC SYSTEM ON THE BASIS OF MULTISCALE DYNAMIC MODELS

Towards Efficient Architecture and Algorithms for Sensor Fusion

Performance analysis of serial data fusion algorithm under typical condition of multi-radar

Hierarchical Fusion Algorithm for Multisensor Integrated Navigation System

A Design of a Multi-Sensor Platform by System-Level Integration in the Package

A hierarchical software architecture for smart firefighting platform

Multi-sensor Data Fusion Based on Similitude Degree

An architecture of traffic state analysis based on multi-sensor fusion

A Novel Multi-Sensor Hybrid Fusion Framework