Abstract:The networks proposed here show how neurons can be connected to form flip-flops, the basic building blocks in sequential logic systems. The novel neural flip-flops (NFFs) are explicit, dynamic, and can generate known phenomena of short-term memory. For each network design, all neurons, connections, and types of synapses are shown explicitly. The neurons' operation depends only on explicitly stated, minimal properties of excitement and inhibition. This operation is dynamic in the sense that the level of neuron activity is the only cellular change, making the NFFs' operation consistent with the speed of most brain functions. Memory tests have shown that certain neurons fire continuously at a high frequency while information is held in short-term memory. These neurons exhibit seven characteristics associated with memory formation, retention, retrieval, termination, and errors. One of the neurons in each of the NFFs produces all of the characteristics. This neuron and a second neighboring neuron together predict eight unknown phenomena. These predictions can be tested by the same methods that led to the discovery of the first seven phenomena. NFFs, together with a decoder from a previous paper, suggest a resolution to the longstanding controversy of whether short-term memory depends on neurons firing persistently or in brief, coordinated bursts. Two novel NFFs are composed of two and four neurons. Their designs follow directly from a standard electronic flip-flop design by moving each negation symbol from one end of the connection to the other. This does not affect the logic of the network, but it changes the logic of each component to a logic function that can be implemented by a single neuron. This transformation is reversible and is apparently new to engineering as well as neuroscience.

Exploring Flip Flop memories and beyond: training Recurrent Neural Networks with key insights

Exploring Flip Flop memories and beyond: training recurrent neural networks with key insights

The flip-flop neuron: a memory efficient alternative for solving challenging sequence processing and decision-making problems

Exploring weight initialization, diversity of solutions, and degradation in recurrent neural networks trained for temporal and decision-making tasks

Rapid Feedforward Computation by Temporal Encoding and Learning with Spiking Neurons

A Brain-Inspired Spiking Neural Network Model with Temporal Encoding and Learning

Reversible Recurrent Neural Networks

A Novel Framework for Recurrent Neural Networks with Enhancing Information Processing and Transmission between Units

Understanding Hidden Memories of Recurrent Neural Networks

Expanding memory in recurrent spiking networks

TIME: A Training-in-Memory Architecture for RRAM-Based Deep Neural Networks

Optogenetics Inspired Transition Metal Dichalcogenide Neuristors for In-Memory Deep Recurrent Neural Networks

Investigating Recurrent Neural Network Memory Structures using Neuro-Evolution

Low-Rank Training of Deep Neural Networks for Emerging Memory Technology

Memory and Information Processing in Recurrent Neural Networks

In Situ Training of Feed-Forward and Recurrent Convolutional Memristor Networks

Computing of temporal information in spiking neural networks with ReRAM synapses

Teaching Recurrent Neural Networks to Modify Chaotic Memories by Example

Ordered Memory.

Sequence Modeling with Recurrent Neural Networks (RNNs) for Student Learning Behavior Pattern Recognition in a Flipped Classroom

Neural flip-flops I: Short-term memory