Abstract:Advancements in adapting deep convolution architectures for spiking neural networks (SNNs) have significantly enhanced image classification performance and reduced computational burdens. However, the inability of multiplication-free inference (MFI) to align with attention and transformer mechanisms, which are critical to superior performance on high-resolution vision tasks, imposes limitations on these gains. To address this, our research explores a new pathway, drawing inspiration from the progress made in multilayer perceptrons (MLPs). We propose an innovative spiking MLP architecture that uses batch normalization (BN) to retain MFI compatibility and introduce a spiking patch encoding (SPE) layer to enhance local feature extraction capabilities. As a result, we establish an efficient multistage spiking MLP network that blends effectively global receptive fields with local feature extraction for comprehensive spike-based computation. Without relying on pretraining or sophisticated SNN training techniques, our network secures a top-one accuracy of 66.39% on the ImageNet-1K dataset, surpassing the directly trained spiking ResNet-34 by 2.67%. Furthermore, we curtail computational costs, model parameters, and simulation steps. An expanded version of our network compares with the performance of the spiking VGG-16 network with a 71.64% top-one accuracy, all while operating with a model capacity 2.1 times smaller. Our findings highlight the potential of our deep SNN architecture in effectively integrating global and local learning abilities. Interestingly, the trained receptive field in our network mirrors the activity patterns of cortical cells.

Feature Importance Measure of a Multilayer Perceptron Based on the Presingle-Connection Layer

Brain-inspired Multilayer Perceptron with Spiking Neurons

Computational interpretability of multilayer preceptron used for SAR image target recognition

Deep PLS: A Lightweight Deep Learning Model for Interpretable and Efficient Data Analytics

Feature Importance Measure for Non-linear Learning Algorithms

Geometrical Interpretation and Design of Multilayer Perceptrons

InfoMLP: Unlocking the Potential of MLPs for Semi-Supervised Learning with Structured Data

Multi-Layer Perceptrons With Embedded Feature Selection With Application In Cancer Classification

Convolution Neural Network Feature Importance Analysis and Feature Selection Enhanced Model

An Importance-and-Semantics-Aware Approach for Entity Resolution Using MLP.

Investigating Layer Importance in Large Language Models

Effective Model Integration Algorithm for Improving Link and Sign Prediction in Complex Networks

Importance estimate of features via analysis of their weight and gradient profile

A Probabilistic Representation of Deep Learning for Improving The Information Theoretic Interpretability

Input Contribution Analysis in a Double Input Layered Neural Network

Efficient Deep Spiking Multilayer Perceptrons With Multiplication-Free Inference

A Feature Structure Based Interpretability Evaluation Approach for Deep Learning

CATSMLP: toward a robust and interpretable multilayer perceptron with sigmoid activation functions.

SS-MLP: A Novel Spectral-Spatial MLP Architecture for Hyperspectral Image Classification

Understanding Deep Neural Networks via Linear Separability of Hidden Layers

An Interpretable Neural Network Model Through Piecewise Linear Approximation