Abstract:The use of deep learning has grown at an exponential rate, giving rise to numerous specialized hardware and software systems for deep learning. Because the design space of deep learning software stacks and hardware accelerators is diverse and vast, prior work considers software optimizations separately from hardware architectures, effectively reducing the search space. Unfortunately, this bifurcated approach means that many profitable design points are never explored. This paper instead casts the problem as hardware/software co-design, with the goal of automatically identifying desirable points in the joint design space. The key to our solution is a new constrained Bayesian optimization framework that avoids invalid solutions by exploiting the highly constrained features of this design space, which are semi-continuous/semi-discrete. We evaluate our optimization framework by applying it to a variety of neural models, improving the energy-delay product by 18% (ResNet) and 40% (DQN) over hand-tuned state-of-the-art systems, as well as demonstrating strong results on other neural network architectures, such as MLPs and Transformers.

What problem does this paper attempt to address?

The problem that this paper attempts to solve is how to automatically optimize the co - design between hardware and software in the design of deep - learning accelerators to improve the Energy - Delay Product (EDP) when executing specific neural - network models. Specifically, traditional methods usually optimize software and hardware designs separately, which has led to many potentially effective design schemes remaining unexplored. This paper proposes a new constrained Bayesian - optimization framework for jointly searching the feasible - solution space of hardware architectures and software mappings, thereby automatically identifying the ideal design points in the joint - design space. Through this method, the paper aims to overcome the problem of poor performance of standard - optimization techniques in existing methods due to the highly - constrained characteristics (i.e., semi - continuous / semi - discrete) of the design space. The key contributions of the paper are as follows: - Proposing the first system capable of automatically optimizing both the hardware and software parts of deep - neural - network accelerators simultaneously, using a principle - and - system - based search algorithm. - Proposing a constrained formal representation of hardware and software designs applicable to Bayesian optimization, which is a challenging problem because invalid hardware and software designs account for 90%. - Proposing a nested hardware / software Bayesian - optimization formula that can be extended to other hardware - accelerator designs. - Providing specific hardware designs for multiple models and state - of - the - art results. Through these contributions, the paper shows that its proposed Bayesian - optimization - based method provides significant improvements in the Energy - Delay Product (EDP) on a series of neural networks compared to existing manually - designed hardware accelerators, with an improvement of 16.0% to 40.2%. This is mainly due to its powerful Bayesian - optimization software optimizer, and the consistent efficiency of this optimizer enables the method to be extended to this huge search space.

Learned Hardware/Software Co-Design of Neural Accelerators

Software-defined Design Space Exploration for an Efficient DNN Accelerator Architecture

Design Automation for Efficient Deep Learning Computing

Invited: Algorithm-Software-Hardware Co-Design for Deep Learning Acceleration

Algorithm and Hardware Co-design for Reconfigurable CNN Accelerator

Woodpecker-DL: Accelerating Deep Neural Networks via Hardware-Aware Multifaceted Optimizations

HAO: Hardware-aware neural Architecture Optimization for Efficient Inference

Software-Hardware Codesign for Efficient Neural Network Acceleration

Unleashing Network/Accelerator Co-Exploration Potential on FPGAs: A Deeper Joint Search

Hardware/Software Co-Exploration of Neural Architectures.

CODEBench: A Neural Architecture and Hardware Accelerator Co-Design Framework

Hardware Accelerated Optimization of Deep Learning Model on Artificial Intelligence Chip

Towards Ultra-High Performance and Energy Efficiency of Deep Learning Systems: An Algorithm-Hardware Co-Optimization Framework

Beyond Training: A Zero-Shot Framework to Neural Architecture and Accelerator Co-Exploration

HSCoNAS: Hardware-Software Co-Design of Efficient DNNs Via Neural Architecture Search

Efficient Hardware Optimization Strategies For Deep Neural Networks Acceleration Chip

GANDSE: Generative Adversarial Network based Design Space Exploration for Neural Network Accelerator Design

The Case for Co-Designing Model Architectures with Hardware

Recent Advances in Efficient Computation of Deep Convolutional Neural Networks

Exploring Resource-Aware Deep Neural Network Accelerator and Architecture Design