Abstract:Event-based vision represents a paradigm shift in how vision information is captured and processed. By only responding to dynamic intensity changes in the scene, event-based sensing produces far less data than conventional frame-based cameras, promising to springboard a new generation of high-speed, low-power machines for edge intelligence. However, processing such dynamically sparse input originated from event cameras efficiently in real time, particularly with complex deep neural networks (DNN), remains a formidable challenge. Existing solutions that employ GPUs and other frame-based DNN accelerators often struggle to efficiently process the dynamically sparse event data, missing the opportunities to improve processing efficiency with sparse data. To address this, we propose ESDA, a composable dynamic sparse dataflow architecture that allows customized DNN accelerators to be constructed rapidly on FPGAs for event-based vision tasks. ESDA is a modular system that is composed of a set of parametrizable modules for each network layer type. These modules share a uniform sparse token-feature interface and can be connected easily to compose an all-on-chip dataflow accelerator on FPGA for each network model. To fully exploit the intrinsic sparsity in event data, ESDA incorporates the use of submanifold sparse convolutions that largely enhance the activation sparsity throughout the layers while simplifying hardware implementation. Finally, a network architecture and hardware implementation co-optimizing framework that allows tradeoffs between accuracy and performance is also presented. Experimental results demonstrate that when compared with existing GPU and hardware-accelerated solutions, ESDA achieves substantial speedup and improvement in energy efficiency across different applications, and it allows much wider design space for real-world deployments.

Visual language based on event-triggered concurrent dataflow model

The Design of Vpp Software Development Environment

A visual programming language for drawing and executing flowcharts

Infrastructure of Web-based VR-form virtual laboratory

A Labview-Based, Interactive Virtual Laboratory for Electronic Engineering Education

A virtual data language and system for scientific workflow management in data grid environments

Visual Programming for Message-Passing Systems.

A VR Simulation Framework Integrated with Multisource CAE Analysis Data for Mechanical Equipment Working Process.

A comprehensive study of visual event computing

Design of Simulation Software for Experiments with Visual Basic

Design and Implementation of Virtual Test System Based on LabWindows/CVI

Development of Distributed On-line Experimental System Based on Virtual Instruments

An Architecture for Application-Specific Visualization Environment(ASVE)

GIVE: a general interactive visualization environment.

Simulation Realization of Lv Svg Control Algorithm Based on Emtdc and C Language Mixed Programming

Design of EMAT Data Acquisition and Analysis Software Based on Virtual Instrumentation Technology

General-Purpose Visual Language and Information System with Case-Studies in Developing Business Applications

Visual Life Sciences Workflow Design Using Distributed and Heterogeneous Resources

A Composable Dynamic Sparse Dataflow Architecture for Efficient Event-based Vision Processing on FPGA

Virtual Instrument Software Labview and Data Acquisition

Design of Electronic Virtual Experiment System Based on LabVIEW