Abstract:In recent years, there has been a significant advancement in memristor-based neural networks, positioning them as a pivotal processing-in-memory deployment architecture for a wide array of deep learning applications. Within this realm of progress, the emerging parallel analog memristive platforms are prominent for their ability to generate multiple feature maps in a single processing cycle. However, a notable limitation is that they are specifically tailored for neural networks with fixed structures. As an orthogonal direction, recent research reveals that neural architecture should be specialized for tasks and deployment platforms. Building upon this, the neural architecture search (NAS) methods effectively explore promising architectures in a large design space. However, these NAS-based architectures are generally heterogeneous and diversified, making it challenging for deployment on current single-prototype, customized, parallel analog memristive hardware circuits. Therefore, investigating memristive analog deployment that overrides the full search space is a promising and challenging problem. Inspired by this, and beginning with the DARTS search space, we study the memristive hardware design of primitive operations and propose the memristive all-inclusive hypernetwork that covers 2×1025 network architectures. Our computational simulation results on 3 representative architectures (DARTS-V1, DARTS-V2, PDARTS) show that our memristive all-inclusive hypernetwork achieves promising results on the CIFAR10 dataset (89.2% of PDARTS with 8-bit quantization precision), and is compatible with all architectures in the DARTS full-space. The hardware performance simulation indicates that the memristive all-inclusive hypernetwork costs slightly more resource consumption (nearly the same in power, 22%∼25% increase in Latency, 1.5× in Area) relative to the individual deployment, which is reasonable and may reach a tolerable trade-off deployment scheme for industrial scenarios.

Hardware architecture and routing-aware training for optimal memory usage: a case study

An Efficient, Low-Cost Routing Architecture for Spiking Neural Network Hardware Implementations

Mapping Very Large Scale Spiking Neuron Network to Neuromorphic Hardware.

Towards Efficient Neural Networks On-a-chip: Joint Hardware-Algorithm Approaches

Hardware-friendly Neural Network Architecture for Neuromorphic Computing

Optimizing Routerless Network-on-Chip Designs: An Innovative Learning-Based Framework

Approaching the mapping limit with closed-loop mapping strategy for deploying neural networks on neuromorphic hardware

Efficient Network Construction Through Structural Plasticity

Scalable NoC-based Neuromorphic Hardware Learning and Inference

Co-Exploring Neural Architecture and Network-on-Chip Design for Real-Time Artificial Intelligence

HFNet: A CNN Architecture Co-designed for Neuromorphic Hardware With a Crossbar Array of Synapses

HAO: Hardware-aware neural Architecture Optimization for Efficient Inference

Hardware-aware training for large-scale and diverse deep learning inference workloads using in-memory computing-based accelerators

An Ultra-Low Latency Multicast Router for Large-Scale Multi-Chip Neuromorphic Processing

Design-Technology Co-Optimization for NVM-based Neuromorphic Processing Elements

Core interface optimization for multi-core neuromorphic processors

Hardware-aware Approach to Deep Neural Network Optimization

Uncontrolled learning: co-design of neuromorphic hardware topology for neuromorphic algorithms

Echo State Graph Neural Networks with Analogue Random Resistive Memory Arrays

A memristive all-inclusive hypernetwork for parallel analog deployment of full search space architectures

Memory optimization of CNN Heterogeneous Multi-Core Architecture