KirchhoffNet: A Scalable Ultra Fast Analog Neural Network

Zhengqi Gao,Fan-Keng Sun,Ron Rohrer,Duane S. Boning
2024-05-06
Abstract:In this paper, we leverage a foundational principle of analog electronic circuitry, Kirchhoff's current and voltage laws, to introduce a distinctive class of neural network models termed KirchhoffNet. Essentially, KirchhoffNet is an analog circuit that can function as a neural network, utilizing its initial node voltages as the neural network input and the node voltages at a specific time point as the output. The evolution of node voltages within the specified time is dictated by learnable parameters on the edges connecting nodes. We demonstrate that KirchhoffNet is governed by a set of ordinary differential equations (ODEs), and notably, even in the absence of traditional layers (such as convolution layers), it attains state-of-the-art performances across diverse and complex machine learning tasks. Most importantly, KirchhoffNet can be potentially implemented as a low-power analog integrated circuit, leading to an appealing property -- irrespective of the number of parameters within a KirchhoffNet, its on-chip forward calculation can always be completed within a short time. This characteristic makes KirchhoffNet a promising and fundamental paradigm for implementing large-scale neural networks, opening a new avenue in analog neural networks for AI.
Machine Learning,Artificial Intelligence,Hardware Architecture
What problem does this paper attempt to address?
The paper aims to propose a novel neural network model called KirchhoffNet by utilizing Kirchhoff’s current and voltage laws. This model is essentially an analog circuit that can operate as a neural network, with the initial node voltages serving as inputs and the node voltages at specific time points serving as outputs. KirchhoffNet controls the changes in node voltages through learnable parameters on the edges connecting the nodes, and these changes are described by a set of ordinary differential equations (ODEs). The paper points out that despite lacking traditional layer structures (such as convolutional layers), KirchhoffNet can still achieve state-of-the-art performance in various complex machine learning tasks. Additionally, since KirchhoffNet can potentially be implemented as a low-power analog integrated circuit, its forward computation can be completed in a short time regardless of the number of parameters in the network. This characteristic makes KirchhoffNet a promising and low-power analog computing paradigm for large-scale neural networks. The paper also explores the application of this model to tasks such as regression, image classification, generation, and density matching, demonstrating its competitiveness compared to existing methods.