Abstract:Neuro-inspired in-memory computing offers a promising solution to overcome the limitations of traditional von Neumann architectures by emulating brain activities. This approach takes advantage of parallel processing while minimizing power consumption and area overhead. However, maximizing the performance of neuromorphic hardware, particularly for increasingly deep and complex NN models, is a challenging task. Existing design space exploration methods focus primarily on layer placement and resource allocation, ignoring hardware-level configurations that directly influence performance, power, and area (PPA) trade-offs. Additionally, current tiled neuromorphic architectures lack support for size heterogeneity, making optimal resource utilization difficult. To address these challenges, we propose a multi-objective architecture search and optimization mechanism for neuromorphic architectures. Our approach introduces heterogeneous architectures with multiple tile/processing element/synaptic array sizes and provides a comprehensive end-to-end design automation tool to support them. We define a heterogeneous neuromorphic architecture, as exemplified by a “big-tile, little-tile” architecture with mesh interconnects. Our search mechanism expands the search space by considering candidates for both homogeneous and heterogeneous architectures and performs Pareto-front searches guided by user-defined weights or constraints on performance metrics. We also implement a hierarchical beam search technique to explore the vast search space of heterogeneous architecture candidates more effectively. Our mechanism identifies numerous heterogeneous architectures that outperform the baseline for different convolutional neural network (CNN) models. For EfficientNetB0, we achieve PPA improvements of 40.1%, 19.3%, and 4.4% over the baseline. Our tool is available at https://github.com/wntjd9805/hetero-neurosim-search.

Examination of the Multimodal Nature of Multi-Objective Neural Architecture Search.

Deep Multimodal Neural Architecture Search

Multi-objective Differentiable Neural Architecture Search

Multi-Objective Neural Architecture Search by Learning Search Space Partitions

Efficient Multi-Objective Neural Architecture Search via Pareto Dominance-based Novelty Search

A Comprehensive Survey of Neural Architecture Search: Challenges and Solutions

Prioritized Architecture Sampling with Monto-Carlo Tree Search

Surrogate-assisted Multi-objective Neural Architecture Search for Real-time Semantic Segmentation

Advances in neural architecture search

Learning Interpretable Models Through Multi-Objective Neural Architecture Search

MO-EMT-NAS: Multi-Objective Continuous Transfer of Architectural Knowledge Between Tasks from Different Datasets

Manas: multi-agent neural architecture search

A Pairwise Comparison Relation-assisted Multi-objective Evolutionary Neural Architecture Search Method with Multi-population Mechanism

Neural Architecture Search Based on A Multi-objective Evolutionary Algorithm With Probability Stack

Neural Architecture Search Survey: A Computer Vision Perspective

Neural-Architecture-Search-Based Multiobjective Cognitive Automation System

Towards a configurable and non-hierarchical search space for NAS

BM-NAS: Bilevel Multimodal Neural Architecture Search

Multi-Objective Architecture Search and Optimization for Heterogeneous Neuromorphic Architecture

Neural Architecture Search with an Efficient Multiobjective Evolutionary Framework

Multi-Objective Neural Architecture Search for In-Memory Computing