On Computational Modeling of Sleep-Wake Cycle

Xin Li

2024-05-18

Abstract:Why do mammals need to sleep? Neuroscience treats sleep and wake as default and perturbation modes of the brain. It is hypothesized that the brain self-organizes neural activities without environmental inputs. This paper presents a new computational model of the sleep-wake cycle (SWC) for learning and memory. During the sleep mode, the memory consolidation by the thalamocortical system is abstracted by a disentangling operator that maps context-dependent representations (CDR) to context-independent representations (CIR) for generalization. Such a disentangling operator can be mathematically formalized by an integral transform that integrates the context variable from CDR. During the wake mode, the memory formation by the hippocampal-neocortical system is abstracted by an entangling operator from CIR to CDR where the context is introduced by physical motion. When designed as inductive bias, entangled CDR linearizes the problem of unsupervised learning for sensory memory by direct-fit. The concatenation of disentangling and entangling operators forms a disentangling-entangling cycle (DEC) as the building block for sensorimotor learning. We also discuss the relationship of DEC and SWC to the perception-action cycle (PAC) for internal model learning and perceptual control theory for the ecological origin of natural languages.

Machine Learning

What problem does this paper attempt to address?

This paper attempts to solve the problems of over - fitting and under - fitting in machine learning by proposing a new geometric perspective to re - understand the data manifold structure in high - dimensional space. Specifically, the paper challenges traditional manifold learning methods, such as IsoMAP and Locally Linear Embedding (LLE), which rely on nonlinear dimensionality reduction techniques. The author proposes the concept of "the blessing of dimensionality", that is, it is easier to untangle low - dimensional manifolds in higher - dimensional spaces, which is guaranteed by Whitney's Embedding Theorem.

The core contribution of the paper lies in introducing a new framework - the Tangling - Untangling Cycle (TUC) for efficient learning. TUC transforms supervised learning into unsupervised learning by introducing context - dependence (such as class labels as context variables) in high - dimensional space, and proves that the untangling operation can lead to linearly separable classifiers. To correct the inevitable over - fitting problem, the paper also considers the dual process of untangling - tangling or aliasing, which is crucial for generalization ability.

The specific implementation of TUC includes:
1. **Untangling operation**: Map the context - independent representation (CIR) in the low - dimensional space to the context - dependent representation (CDR) in the high - dimensional space by introducing context as a hidden variable.
2. **Tangling operation**: Map CDR back to CIR through a simple integral transformation to achieve invariance and generalization.

In addition, the paper also explores the hierarchical expansion of TUC, based on the Cartesian product and fractal geometry, as well as the connection between TUC and biological neural networks, especially Polychronization Neural Groups (PNG) and the Sleep - Wake Cycle (SWC). These biological implementations not only support the effectiveness of the TUC theory but also provide a new perspective for understanding cognitive functions in the brain.

In conclusion, the main goal of this paper is to provide a new geometric method through the TUC framework to balance the specificity and generalization ability in machine learning, thereby improving the learning efficiency.