Temporal credit assignment for one-shot learning utilizing a phase transition material

Alessandro R. Galloni,Yifan Yuan,Minning Zhu,Haoming Yu,Ravindra S. Bisht,Chung-Tse Michael Wu,Christine Grienberger,Shriram Ramanathan,Aaron D. Milstein
2023-09-30
Abstract:Design of hardware based on biological principles of neuronal computation and plasticity in the brain is a leading approach to realizing energy- and sample-efficient artificial intelligence and learning machines. An important factor in selection of the hardware building blocks is the identification of candidate materials with physical properties suitable to emulate the large dynamic ranges and varied timescales of neuronal signaling. Previous work has shown that the all-or-none spiking behavior of neurons can be mimicked by threshold switches utilizing phase transitions. Here we demonstrate that devices based on a prototypical metal-insulator-transition material, vanadium dioxide (VO2), can be dynamically controlled to access a continuum of intermediate resistance states. Furthermore, the timescale of their intrinsic relaxation can be configured to match a range of biologically-relevant timescales from milliseconds to seconds. We exploit these device properties to emulate three aspects of neuronal analog computation: fast (~1 ms) spiking in a neuronal soma compartment, slow (~100 ms) spiking in a dendritic compartment, and ultraslow (~1 s) biochemical signaling involved in temporal credit assignment for a recently discovered biological mechanism of one-shot learning. Simulations show that an artificial neural network using properties of VO2 devices to control an agent navigating a spatial environment can learn an efficient path to a reward in up to 4 fold fewer trials than standard methods. The phase relaxations described in our study may be engineered in a variety of materials, and can be controlled by thermal, electrical, or optical stimuli, suggesting further opportunities to emulate biological learning.
Disordered Systems and Neural Networks,Materials Science,Hardware Architecture,Neural and Evolutionary Computing
What problem does this paper attempt to address?
The problem that this paper attempts to solve is how to design a neuromorphic computing hardware based on phase - change materials (such as vanadium dioxide, VO2) to achieve one - shot learning. Specifically, the research aims to use the dynamic control characteristics of VO2 to simulate multiple internal state variables of neurons. These variables change on different biologically relevant time scales, enabling temporal credit assignment, a recently discovered biological mechanism that allows the brain to learn quickly from a single experience. The key points of the paper include: 1. **Material Selection and Characteristics**: VO2 was selected as the prototype material because it can transition between a metallic state and an insulating state, and this transition can be controlled by temperature or electrical stimulation. The resistance state of VO2 can be configured on different time scales from milliseconds to seconds, which matches the time scales of electrical and biochemical signals in neurons. 2. **Neuromorphic Computing Circuit Design**: A hybrid circuit was designed, combining emerging materials (such as VO2) with traditional silicon - based CMOS components to simulate the integration and firing characteristics of neurons. In the circuit design, VO2 devices are used to simulate different internal state variables of neurons, including fast somatic firing, slow dendritic firing, and ultraslow biochemical signals, which are crucial for the temporal credit assignment in one - shot learning. 3. **Achievement of One - shot Learning**: Through simulation, it was shown that a neural network using VO2 devices can learn the effective path to reach the reward in a spatial navigation reinforcement - learning task through a single trial, with a learning efficiency four times higher than that of the standard method. This result indicates that by engineering the unique properties of materials, artificial neural networks closer to biological learning mechanisms can be constructed. In summary, this paper explores how to use the progress in materials science to improve the learning efficiency of artificial neural networks, especially in terms of one - shot learning, by designing neuromorphic computing hardware based on VO2.