Abstract:This paper presents a comprehensive introduction and systematic derivation of the evolutionary equations for absolute entropy H and relative entropy D, some of which exist sporadically in the literature in different forms under different subjects, within the framework of dynamical systems. In general, both H and D are dissipated, and the dissipation bears a form reminiscent of the Fisher information; in the absence of stochasticity, dH/dt is connected to the rate of phase space expansion, and D stays invariant, i.e., the separation of two probability density functions is always conserved. These formulas are validated with linear systems, and put to application with the Lorenz system and a large-dimensional stochastic quasi-geostrophic flow problem. In the Lorenz case, H falls at a constant rate with time, implying that H will eventually become negative, a situation beyond the capability of the commonly used computational technique like coarse-graining and bin counting. For the stochastic flow problem, it is first reduced to a computationally tractable low-dimensional system, using a reduced model approach, and then handled through ensemble prediction. Both the Lorenz system and the stochastic flow system are examples of self-organization in the light of uncertainty reduction. The latter particularly shows that, sometimes stochasticity may actually enhance the self-organization process.

Entropy Estimate for Degenerate SDEs with Applications to Nonlinear Kinetic Fokker–Planck Equations

Entropy Estimate for Degenerate SDEs with Applications to Nonlinear Kinetic Fokker-Planck Equations

Hypoelliptic entropy dissipation for stochastic differential equations

Thermodynamics of Markov Processes with Non-extensive Entropy and Free Energy

Entropy Solutions to the Dirichlet Problem for Nonlinear Diffusion Equations with Conservative Noise

Exponential Convergence in Entropy and Wasserstein Distance for McKean-Vlasov SDEs

Quantitative relative entropy estimates for interacting particle systems with common noise

SAV-based entropy-dissipative schemes for a class of kinetic equations

Error estimates of the Euler's method for stochastic differential equations with multiplicative noise via relative entropy

Krylov-Safonov estimates for a degenerate diffusion process

Generalized Kinetic Equations with Fractional Time-Derivative and Nonlinear Diffusion: H-Theorem and Entropy

Entropy Evolution and Uncertainty Estimation with Dynamical Systems

Non-asymptotic entropic bounds for non-linear kinetic Langevin sampler with second-order splitting scheme

Uniform-in-time estimates for mean-field type SDEs and applications

Coupling Methods and Applications on Path Dependent McKean-Vlasov SDEs

Entropy on the Path Space and Application to Singular Diffusions and Mean-field Models

Nonequilibrium entropy from density estimation

Quantitative estimates of propagation of chaos for stochastic systems with W^-1,∞ kernels

A hypocoercivity related ergodicity method with rate of convergence for singularly distorted degenerate Kolmogorov equations and applications

Probability Distance Estimates Between Diffusion Processes and Applications to Singular McKean-Vlasov SDEs

Long-time behavior for discretization schemes of Fokker-Planck equations via couplings