Preventing hearing loss due to excessive noise exposure.

J. Sataloff

1974-07-01

Abstract:

What problem does this paper attempt to address?

Trained recurrent neural networks develop phase-locked limit cycles in a working memory task

Matthijs Pals,Jakob H. Macke,Omri Barak

DOI: https://doi.org/10.1371/journal.pcbi.1011852

2024-02-07

PLoS Computational Biology

Abstract:Neural oscillations are ubiquitously observed in many brain areas. One proposed functional role of these oscillations is that they serve as an internal clock, or 'frame of reference'. Information can be encoded by the timing of neural activity relative to the phase of such oscillations. In line with this hypothesis, there have been multiple empirical observations of such phase codes in the brain. Here we ask: What kind of neural dynamics support phase coding of information with neural oscillations? We tackled this question by analyzing recurrent neural networks (RNNs) that were trained on a working memory task. The networks were given access to an external reference oscillation and tasked to produce an oscillation, such that the phase difference between the reference and output oscillation maintains the identity of transient stimuli. We found that networks converged to stable oscillatory dynamics. Reverse engineering these networks revealed that each phase-coded memory corresponds to a separate limit cycle attractor. We characterized how the stability of the attractor dynamics depends on both reference oscillation amplitude and frequency, properties that can be experimentally observed. To understand the connectivity structures that underlie these dynamics, we showed that trained networks can be described as two phase-coupled oscillators. Using this insight, we condensed our trained networks to a reduced model consisting of two functional modules: One that generates an oscillation and one that implements a coupling function between the internal oscillation and external reference. In summary, by reverse engineering the dynamics and connectivity of trained RNNs, we propose a mechanism by which neural networks can harness reference oscillations for working memory. Specifically, we propose that a phase-coding network generates autonomous oscillations which it couples to an external reference oscillation in a multi-stable fashion. Many of our actions are rhythmic—walking, breathing, digesting and more. It is not surprising that neural activity can have a strong oscillatory component. Indeed, such brain waves are common, and can even be measured using EEG from the scalp. Perhaps less obvious is the presence of such oscillations during non-rhythmic behavior—such as memory maintenance and other cognitive functions. Reports of these cognitive oscillations have accumulated over the years, and various theories were raised regarding their origin and utilization. In particular, oscillations have been proposed to serve as a clock signal that can be used for temporal-, or phase-coding of information in working memory. Here, we studied the dynamical systems underlying this kind of coding, by using trained artificial neural networks as hypothesis generators. We trained recurrent neural networks to perform a working memory task, while giving them access to a reference oscillation. We were then able to reverse engineer the learned dynamics of the networks. Our analysis revealed that phase-coded memories correspond to stable attractors in the dynamical landscape of the model. These attractors arose from the coupling of the external reference oscillation with oscillations generated internally by the network.

biochemical research methods,mathematical & computational biology
A Spiking Neural Network Model For Associative Memory Using Temporal Codes

jun hu,huajin tang,k c tan,sen bong gee

DOI: https://doi.org/10.1007/978-3-319-13359-1_43

2015-01-01

Abstract:Associative memory is defined as the ability to map input patterns to output patterns. Understanding how human brain performs association between unrelated patterns and stores this knowledge is one of the most important goals in computational intelligence. Although this problem has been widely studied using conventional neural networks, increasing biological findings suggest that spiking neural network can be an alternative. The proposed model encodes different memories using different subsets of encoding neurons with temporal codes. A spike-timing based learning algorithm and spike-timing-dependent plasticity (STDP) are used to form associative memory. Simulation results show that hetero-associative memory and auto-associative memory are achievable by the synaptic modification of connections between input layer and hidden layers, and recurrent connections of hidden layers, respectively.
Cross-Frequency Coupling Increases Memory Capacity in Oscillatory Neural Networks

Connor Bybee,Alexander Belsten,Friedrich T. Sommer

DOI: https://doi.org/10.48550/arXiv.2204.07163

2022-04-05

Neurons and Cognition

Abstract:An open problem in neuroscience is to explain the functional role of oscillations in neural networks, contributing, for example, to perception, attention, and memory. Cross-frequency coupling (CFC) is associated with information integration across populations of neurons. Impaired CFC is linked to neurological disease. It is unclear what role CFC has in information processing and brain functional connectivity. We construct a model of CFC which predicts a computational role for observed $\theta - \gamma$ oscillatory circuits in the hippocampus and cortex. Our model predicts that the complex dynamics in recurrent and feedforward networks of coupled oscillators performs robust information storage and pattern retrieval. Based on phasor associative memories (PAM), we present a novel oscillator neural network (ONN) model that includes subharmonic injection locking (SHIL) and which reproduces experimental observations of CFC. We show that the presence of CFC increases the memory capacity of a population of neurons connected by plastic synapses. CFC enables error-free pattern retrieval whereas pattern retrieval fails without CFC. In addition, the trade-offs between sparse connectivity, capacity, and information per connection are identified. The associative memory is based on a complex-valued neural network, or phasor neural network (PNN). We show that for values of $Q$ which are the same as the ratio of $\gamma$ to $\theta$ oscillations observed in the hippocampus and the cortex, the associative memory achieves greater capacity and information storage than previous models. The novel contributions of this work are providing a computational framework based on oscillator dynamics which predicts the functional role of neural oscillations and connecting concepts in neural network theory and dynamical system theory.
Associative memory by virtual oscillator network based on single spin-torque oscillator

Yusuke Imai,Tomohiro Taniguchi

DOI: https://doi.org/10.1038/s41598-023-42951-z

2024-03-29

Abstract:A coupled oscillator network may be able to perform an energy-efficient associative memory operation. However, its realization has been difficult because inhomogeneities unavoidably arise among the oscillators during fabrication and lead to an unreliable operation. This issue could be resolved if the oscillator network were able to be formed from a single oscillator. Here, we performed numerical simulations and theoretical analyses on an associative memory operation that uses a virtual oscillator network based on a spin-torque oscillator. The virtual network combines the concept of coupled oscillators with that of feedforward neural networks. Numerical experiments demonstrate successful associations of $60$-pixel patterns with various memorized patterns. Moreover, the origin of the associative memory is shown to be forced synchronization driven by feedforward input, where phase differences among oscillators are fixed and correspond to the colors of the pixels in the pattern.

Mesoscale and Nanoscale Physics,Adaptation and Self-Organizing Systems,Applied Physics,Computational Physics
A Hierarchical Organized Memory Model Using Spiking Neurons

Jun Hu,Huajin Tang,Kay Chen Tan

DOI: https://doi.org/10.1109/IJCNN.2013.6706891

2013-01-01

Abstract:The recent identification of neural cliques, which are network-level memory coding units in the hippocampus, enables population codes to be the neuronal representation of memory. It has been discovered that the timing of spikes plays an important role in the neural computation and information processing in the brain. Moreover, these memory-coding units have been observed organizing in a hierarchical manner in the brain. Inspired by these exciting findings, we present a hierarchically organized memory model with spiking neurons, which can store both associative memory and episodic memory with temporal population codes. The basic structure of the hierarchical model is composed of three layers with different functions and can be extended to more complicated networks by duplicating and connecting the basic three-layer network. With a spike-timing based learning algorithm, the spiking neural network with theta and gamma oscillations is able to store spatiotemporal memory items within gamma cycles, and links these memories into a sequence. The spiking-timing-dependent plasticity (STDP) contributes to the formation of both associative memory and episodic memory via fast and slow N-methyl-Daspartate (NMDA) channels, respectively.
Resonant synchronization and information retrieve from memorized Kuramoto network

Lin Zhang,Xv Li,Tingting Xue

DOI: https://doi.org/10.48550/arXiv.1809.01445

2018-09-05

Abstract:A new collective behavior of resonant synchronization is discovered and the ability to retrieve information from brain memory is proposed based on this mechanism. We use modified Kuramoto phase oscillator to simulate the dynamics of a single neuron in self-oscillation state, and investigate the collective responses of a neural network, which is composed of $N$ globally coupled Kuramoto oscillators, to the external stimulus signals in a critical state just below the synchronization threshold of Kuramoto model. The input signals at different driving frequencies, which are used to denote different neural stimuli, can drive the coupled oscillators into different synchronized groups locked to the same effective frequencies and recover different synchronized patterns emerged from their collective dynamics closely related to the predetermined frequency distributions of the oscillators (memory). This model is used to explain how brain stores and retrieves information by the synchronized patterns emerging in the neural network stimulated by the external inputs.

Adaptation and Self-Organizing Systems,Biological Physics
Spiking Neuron Based Cognitive Memory Model

Qiang Yu,Huajin Tang,Jun Hu,Kay Chen Tan

DOI: https://doi.org/10.1007/978-3-319-55310-8_8

2017-01-01

Abstract:Jensen et al. (Learn. Mem. 3(2-3), 245-246, 1996 [1]) proposed an auto-associative memory model using an integrated short-term memory (STM) and long-term memory (LTM) spiking neural network. Their model requires that distinct pyramidal cells encoding different STM patterns are fired in different high-frequency gamma subcycles within each low-frequency theta oscillation. Auto-associative LTM is formed by modifying the recurrent synaptic efficacy between pyramidal cells. In order to store auto-associative LTM correctly, the recurrent synaptic efficacy must be bounded. The synaptic efficacy must be upper bounded to prevent re-firing of pyramidal cells in subsequent gamma subcycles. If cells encoding one memory item were to re-fire synchronously with other cells encoding another item in subsequent gamma subcycle, LTM stored via modifiable recurrent synapses would be corrupted. The synaptic efficacy must also be lower bounded so that memory pattern completion can be performed correctly. This chapter uses the original model by Jensen et al. as the basis to illustrate the following points. Firstly, the importance of coordinated long-term memory (LTM) synaptic modification. Secondly, the use of a generic mathematical formulation (spiking response model) that can theoretically extend the results to other spiking network utilizing threshold-fire spiking neuron model. Thirdly, the interaction of long-term and short-term memory networks that possibly explains the asymmetric distribution of spike density in theta cycle through the merger of STM patterns with interaction of LTM network.
Associative Memory Model of Hippocampus CA3 Using Spike Response Neurons

Chin Hiong Tan,Eng Yeow Cheu,Jun Hu,Qiang Yu,Huajin Tang

DOI: https://doi.org/10.1007/978-3-642-24955-6_59

2011-01-01

Abstract:One of the functional roles of the hippocampus is the storage and recall of associative memories. The hippocampus CA3 region has been hypothesized to function as an associative network. Dual oscillations have been recorded in brain regions involved in memory function in which a low frequency theta oscillation is subdivided into about seven subcycles of high frequency gamma oscillation. In this paper, the computational model of hippocampus CA3 proposed by Jensen et al. is realized using the Spike Response Model (SRM). The SRM-based network is able to demonstrate the same memory storage capability with added simplicity and flexibility. Different short term memory items are encoded by different subset of principal neurons and long term associative memory is maintained in the synaptic modifications of recurrent collaterals. The formation of associative memory is demonstrated in simulations.
Synaptic Conditions for Auto-Associative Memory Storage and Pattern Completion in Jensen Et Al.’s Model of Hippocampal Area CA3

Eng Yeow Cheu,Jiali Yu,Chin Hiong Tan,Huajin Tang

DOI: https://doi.org/10.1007/s10827-012-0394-8

IF: 1.4529

2012-01-01

Journal of Computational Neuroscience

Abstract:Jensen et al. (Learn Memory 3(2–3):243–256, 1996b) proposed an auto-associative memory model using an integrated short-term memory (STM) and long-term memory (LTM) spiking neural network. Their model requires that distinct pyramidal cells encoding different STM patterns are fired in different high-frequency gamma subcycles within each low-frequency theta oscillation. Auto-associative LTM is formed by modifying the recurrent synaptic efficacy between pyramidal cells. In order to store auto-associative LTM correctly, the recurrent synaptic efficacy must be bounded. The synaptic efficacy must be upper bounded to prevent re-firing of pyramidal cells in subsequent gamma subcycles. If cells encoding one memory item were to re-fire synchronously with other cells encoding another item in subsequent gamma subcycle, LTM stored via modifiable recurrent synapses would be corrupted. The synaptic efficacy must also be lower bounded so that memory pattern completion can be performed correctly. This paper uses the original model by Jensen et al. as the basis to illustrate the following points. Firstly, the importance of coordinated long-term memory (LTM) synaptic modification. Secondly, the use of a generic mathematical formulation (spiking response model) that can theoretically extend the results to other spiking network utilizing threshold-fire spiking neuron model. Thirdly, the interaction of long-term and short-term memory networks that possibly explains the asymmetric distribution of spike density in theta cycle through the merger of STM patterns with interaction of LTM network.
A Brain-Inspired Spiking Neural Network Model with Temporal Encoding and Learning

Qiang Yu,Huajin Tang,Kay Chen Tan,Haoyong Yu

DOI: https://doi.org/10.1016/j.neucom.2013.06.052

IF: 6

2014-01-01

Neurocomputing

Abstract:Neural coding and learning are important components in cognitive memory system, by processing the sensory inputs and distinguishing different patterns to allow for higher level brain functions such as memory storage and retrieval. Benefitting from biological relevance, this paper presents a spiking neural network of leaky integrate-and-fire (LIF) neurons for pattern recognition. A biologically plausible supervised synaptic learning rule is used so that neurons can efficiently make a decision. The whole system contains encoding, learning and readout. Utilizing the temporal coding and learning, networks of spiking neurons can effectively and efficiently perform various classification tasks. It can classify complex patterns of activities stored in a vector, as well as the real-world stimuli. Our approach is also benchmarked on the nonlinearly separable Iris dataset. The proposed approach achieves a good generalization, with a classification accuracy of 99.63% for training and 92.55% for testing. In addition, the trained networks demonstrate that the temporal coding is a viable means for fast neural information processing.
Active oscillatory associative memory

Matthew Du,Agnish Kumar Behera,Suriyanarayanan Vaikuntanathan

2023-07-25

Abstract:Traditionally, physical models of associative memory assume conditions of equilibrium. Here, we consider a prototypical oscillator model of associative memory and study how active noise sources that drive the system out of equilibrium, as well as nonlinearities in the interactions between the oscillators, affect the associative memory properties of the system. Our simulations show that pattern retrieval under active noise is more robust to the number of learned patterns and noise intensity than under passive noise. To understand this phenomenon, we analytically derive an effective energy correction due to the temporal correlations of active noise in the limit of short correlation decay time. We find that active noise deepens the energy wells corresponding to the patterns by strengthening the oscillator couplings, where the more nonlinear interactions are preferentially enhanced. Using replica theory, we demonstrate qualitative agreement between this effective picture and the retrieval simulations. Our work suggests that nonlinearity in the oscillator couplings can improve memory under nonequilibrium conditions.

Disordered Systems and Neural Networks,Statistical Mechanics
A robust autoassociative memory with coupled networks of Kuramoto-type oscillators

Daniel Heger,Katharina Krischer

DOI: https://doi.org/10.48550/arXiv.1604.02085

2016-04-07

Adaptation and Self-Organizing Systems

Abstract:Uncertain recognition success, unfavorable scaling of connection complexity or dependence on complex external input impair the usefulness of current oscillatory neural networks for pattern recognition or restrict technical realizations to small networks. We propose a new network architecture of coupled oscillators for pattern recognition which shows none of the mentioned aws. Furthermore we illustrate the recognition process with simulation results and analyze the new dynamics analytically: Possible output patterns are isolated attractors of the system. Additionally, simple criteria for recognition success are derived from a lower bound on the basins of attraction.
Attractor networks and memory replay of phase coded spike patterns

Ferdinando Giacco,Silvia Scarpetta

DOI: https://doi.org/10.48550/arXiv.1210.6979

2012-10-25

Abstract:We analyse the storage and retrieval capacity in a recurrent neural network of spiking integrate and fire neurons. In the model we distinguish between a learning mode, during which the synaptic connections change according to a Spike-Timing Dependent Plasticity (STDP) rule, and a recall mode, in which connections strengths are no more plastic. Our findings show the ability of the network to store and recall periodic phase coded patterns a small number of neurons has been stimulated. The self sustained dynamics selectively gives an oscillating spiking activity that matches one of the stored patterns, depending on the initialization of the network.

Neurons and Cognition
Electrically Programmable Phase-Change Photonic Memory for Optical Neural Networks with Nanoseconds in Situ Training Capability

Maoliang Wei,Junying Li,Zequn Chen,Bo Tang,Zhiqi Jia,Peng Zhang,Kunhao Lei,Kai Xu,Jianghong Wu,Chuyu Zhong,Hui Ma,Yuting Ye,Jialing Jian,Chunlei Sun,Ruonan Liu,Ying Sun,Wei. E. I. Sha,Xiaoyong Hu,Jianyi Yang,Lan Li,Hongtao Lin

DOI: https://doi.org/10.1117/1.ap.5.4.046004

IF: 13.582

2023-01-01

Advanced Photonics

Abstract:Optical neural networks（ONNs）, enabling low latency and high parallel data processing without electromagnetic interference, have become a viable player for fast and energy-efficient processing and calculation to meet the increasing demand for hash rate. Photonic memories employing nonvolatile phasechange materials could achieve zero static power consumption, low thermal cross talk, large-scale, and high-energy-efficient photonic neural networks. Nevertheless, the switching speed and dynamic energy consumption of phase-change material-based photonic memories make them inapplicable for in situ training.Here, by integrating a patch of phase change thin film with a PIN-diode-embedded microring resonator,a bifunctional photonic memory enabling both 5-bit storage and nanoseconds volatile modulation was demonstrated. For the first time, a concept is presented for electrically programmable phase-change material-driven photonic memory integrated with nanosecond modulation to allow fast in situ training and zero static power consumption data processing in ONNs. ONNs with an optical convolution kernel constructed by our photonic memory theoretically achieved an accuracy of predictions higher than 95% when tested by the MNIST handwritten digit database. This provides a feasible solution to constructing large-scale nonvolatile ONNs with high-speed in situ training capability.
Sub-harmonic Injection Locking Memristor-based Oscillator Arrays Used for Pattern Recognition

Yubing Xu,Bo Wang

DOI: https://doi.org/10.1109/iwofc48002.2019.9078470

2019-01-01

Abstract:Oscillatory neural network (ONN) is a promising computing architecture that can realize pattern recognition or other intellectual applications in real time. Emerging memristor-based oscillator provides a good choice as building block and corresponding phase behavioral model can accelerate simulation for about 40 times comparing to transistor level simulation. However, phase unlocking due to the frequency detuning after recognition is prominent in memristor-based traditional ONN architecture. With such problem, phase deviation cannot be locked and the correct recognized image fails to be continuously presented after synchronization. In this paper, a novel sub-harmonic injection locking (SHIL) memristor-based ONN is proposed to handle this problem. Energy function is used to give a deep insight into our method. Simulation both in MATLAB and Cadence shows a consistent frequency, i.e., constant phase differences. The results of error standard deviation of recognized patterns reduce 80, 36 and 192 times respectively in three representative cases.
An Oscillator Ensemble Model of Sequence Learning

Alexander Maye,Peng Wang,Jonathan Daume,Xiaolin Hu,Andreas K. Engel

DOI: https://doi.org/10.3389/fnint.2019.00043

2019-01-01

Frontiers in Integrative Neuroscience

Abstract:Learning and memorizing sequences of events is an important function of the human brain and the basis for forming expectations and making predictions. Learning is facilitated by repeating a sequence several times, causing rhythmic appearance of the individual sequence elements. This observation invites to consider the resulting multitude of rhythms as a spectral ‘fingerprint’ which characterizes the respective sequence. Here we explore the implications of this perspective by developing a neurobiologically plausible computational model which captures this ‘fingerprint’ by attuning an ensemble of neural oscillators. In our model, this attuning process is based on a number of oscillatory phenomena that have been observed in electrophysiological recordings of brain activity like synchronization, phase locking and reset as well as cross-frequency coupling. We compare the learning properties of the model with behavioral results from a study in human participants and observe good agreement of the errors for different levels of complexity of the sequence to be memorized. Finally, we suggest an extension of the model for processing sequences that extend over several sensory modalities.
Short term memory by transient oscillatory dynamics in recurrent neural networks

Kohei Ichikawa,Kunihiko Kaneko

DOI: https://doi.org/10.1103/PhysRevResearch.3.033193

2021-08-14

Abstract:Despite the significance of short-term memory in cognitive function, the process of encoding and sustaining the input information in neural activity dynamics remains elusive. Herein, we unveiled the significance of transient neural dynamics to short-term memory. By training recurrent neural networks to short-term memory tasks and analyzing the dynamics, the characteristics of the short-term memory mechanism were obtained in which the input information was encoded in the amplitude of transient oscillations, rather than the stationary neural activities. This transient trajectory was attracted to a slow manifold, which permitted the discarding of irrelevant information. Additionally, we investigated the process by which the dynamics acquire robustness to noise. In this transient oscillation, the robustness to noise was obtained by a strong contraction of the neural states after perturbation onto the manifold. This mechanism works for several neural network models and tasks, which implies its relevance to neural information processing in general.

Neurons and Cognition,Adaptation and Self-Organizing Systems
Typhus in Calcutta

R. N. Chaudhuri,H. Chakravarti

1949-02-01

Abstract:ln this paper. The 'patients.?The present series included 17 Bengalees and 8 non-Bengalees of whom two ^ere Anglo-Indians. The youngest patient was H years and the oldest 61 years old, 18 (72 per cent) being between 11 and 30 years. This shows 'he maximum prevalence during adolescence and in young adults. There were 15 males and females indicating that both sexes were almost dually affected; female accommodation is more limited in our hospital. The occupation of 14 patients was as follows : Three medical men, 4 clerks, 1 darwan and 6 college students; the rest were women and children, or gave no information regarding their occupation. Twenty patients belonged to the city proper while the remaining five came from outside, of whom one patient, a woman, was on her way to Eastern Pakistan and contracted the disease during her short stay in Calcutta, but members of the family accompanying her were unaffected. Only 2 patients?nos. 4 and 9 in table I?gave a history of similar type of fever in their families. A reference to the dates of admission shows a definite seasonal incidence, 18 cases occurring from July to September with the maximum of 8 in the month of August.

Preventing hearing loss due to excessive noise exposure.

Trained recurrent neural networks develop phase-locked limit cycles in a working memory task

A Spiking Neural Network Model For Associative Memory Using Temporal Codes

Cross-Frequency Coupling Increases Memory Capacity in Oscillatory Neural Networks

Associative memory by virtual oscillator network based on single spin-torque oscillator

A Hierarchical Organized Memory Model Using Spiking Neurons

Resonant synchronization and information retrieve from memorized Kuramoto network

Spiking Neuron Based Cognitive Memory Model

Associative Memory Model of Hippocampus CA3 Using Spike Response Neurons

Synaptic Conditions for Auto-Associative Memory Storage and Pattern Completion in Jensen Et Al.’s Model of Hippocampal Area CA3

A Brain-Inspired Spiking Neural Network Model with Temporal Encoding and Learning

Active oscillatory associative memory

A robust autoassociative memory with coupled networks of Kuramoto-type oscillators

Attractor networks and memory replay of phase coded spike patterns

Electrically Programmable Phase-Change Photonic Memory for Optical Neural Networks with Nanoseconds in Situ Training Capability

Sub-harmonic Injection Locking Memristor-based Oscillator Arrays Used for Pattern Recognition

An Oscillator Ensemble Model of Sequence Learning

Short term memory by transient oscillatory dynamics in recurrent neural networks

Typhus in Calcutta