Finding emergence in data by maximizing effective information

Mingzhe Yang,Zhipeng Wang,Kaiwei Liu,Yingqi Rong,Bing Yuan,Jiang Zhang

2023-11-30

Abstract:Quantifying emergence and modeling emergent dynamics in a data-driven manner for complex dynamical systems is challenging due to the lack of direct observations at the micro-level. Thus, it's crucial to develop a framework to identify emergent phenomena and capture emergent dynamics at the macro-level using available data. Inspired by the theory of causal emergence (CE), this paper introduces a machine learning framework to learn macro-dynamics in an emergent latent space and quantify the degree of CE. The framework maximizes effective information, resulting in a macro-dynamics model with enhanced causal effects. Experimental results on simulated and real data demonstrate the effectiveness of the proposed framework. It quantifies degrees of CE effectively under various conditions and reveals distinct influences of different noise types. It can learn a one-dimensional coarse-grained macro-state from fMRI data, to represent complex neural activities during movie clip viewing. Furthermore, improved generalization to different test environments is observed across all simulation data.

Physics and Society,Machine Learning

What problem does this paper attempt to address?

### Problems the Paper Attempts to Solve

This paper aims to address the problem of discovering emergent behaviors from data. Specifically, it attempts to identify and quantify emergent phenomena in complex dynamic systems and capture emergent dynamics at the macro level.

#### Main Challenges:
1. **Lack of Micro-level Observations**: Due to the lack of direct micro-level data observations, it is difficult to directly observe and identify emergent patterns or dynamic behaviors in complex systems.
2. **Absence of a Unified Quantification Method**: Traditional measurement methods either rely on predefined macro variables or are designed for specific scenarios, lacking a unified approach to quantify emergence across different contexts.

#### Research Background:
- Complex systems (such as climate systems, ecosystems, bird flocks, etc.) consist of a large number of interacting elements, exhibiting a wide range of nonlinear dynamic behaviors.
- In recent years, with the increase in real dynamic system data, data-driven modeling methods have received widespread attention.
- Emergence, as a significant feature of complex systems, has always been a challenge to quantify and describe.

#### Solution:
The paper proposes a machine learning framework, Neural Information Squeezer Plus (NIS+), to learn macro dynamics in the emergent latent space and quantify the degree of causal emergence. This framework enhances the causal effect of the macro dynamic model by maximizing Effective Information (EI).

#### Experimental Results:
- Experimental results on both simulated and real data show that the proposed framework can effectively quantify the degree of causal emergence under different conditions and reveal the different impacts of various types of noise.
- For example, in fMRI data, a one-dimensional coarse-grained macro state can be learned to represent the complex neural activity when watching movie clips.
- All experimental results on simulated data also demonstrate improved generalization ability in different test environments.

#### Conclusion:
The main goal of the paper is to identify and quantify emergent phenomena in complex dynamic systems and propose a new machine learning framework, NIS+, which can learn macro dynamics in the emergent latent space, thereby better understanding and predicting the dynamic behaviors of complex systems.