Adult multilineage mesenchymal progenitor cells isolated from the bursa subacromialis.

A. Steinert,A. Heymer,M. Kunz,F. Jakob,U. Noth,F. Gohlke

2007-05-16

Journal of Stem Cells and Regenerative Medicine

Abstract:

What problem does this paper attempt to address?

Learning Spin Liquids on a Honeycomb Lattice with Artificial Neural Networks.

Chang-Xiao Li,Sheng Yang,Jing-Bo Xu

DOI: https://doi.org/10.1038/s41598-021-95523-4

IF: 4.6

2021-01-01

Scientific Reports

Abstract:Machine learning methods provide a new perspective on the study of many-body system in condensed matter physics and there is only limited understanding of their representational properties and limitations in quantum spin liquid systems. In this work, we investigate the ability of the machine learning method based on the restricted Boltzmann machine in capturing physical quantities including the ground-state energy, spin-structure factor, magnetization, quantum coherence, and multipartite entanglement in the two-dimensional ferromagnetic spin liquids on a honeycomb lattice. It is found that the restricted Boltzmann machine can encode the many-body wavefunction quite well by reproducing accurate ground-state energy and structure factor. Further investigation on the behavior of multipartite entanglement indicates that the residual entanglement is richer in the gapless phase than the gapped spin-liquid phase, which suggests that the residual entanglement can characterize the spin-liquid phases. Additionally, we confirm the existence of a gapped non-Abelian topological phase in the spin liquids on a honeycomb lattice with a small magnetic field and determine the corresponding phase boundary by recognizing the rapid change of the local magnetization and residual entanglement.
Deep Learning the Ising Model Near Criticality

Alan Morningstar,Roger G. Melko

DOI: https://doi.org/10.48550/arXiv.1708.04622

2017-08-16

Abstract:It is well established that neural networks with deep architectures perform better than shallow networks for many tasks in machine learning. In statistical physics, while there has been recent interest in representing physical data with generative modelling, the focus has been on shallow neural networks. A natural question to ask is whether deep neural networks hold any advantage over shallow networks in representing such data. We investigate this question by using unsupervised, generative graphical models to learn the probability distribution of a two-dimensional Ising system. Deep Boltzmann machines, deep belief networks, and deep restricted Boltzmann networks are trained on thermal spin configurations from this system, and compared to the shallow architecture of the restricted Boltzmann machine. We benchmark the models, focussing on the accuracy of generating energetic observables near the phase transition, where these quantities are most difficult to approximate. Interestingly, after training the generative networks, we observe that the accuracy essentially depends only on the number of neurons in the first hidden layer of the network, and not on other model details such as network depth or model type. This is evidence that shallow networks are more efficient than deep networks at representing physical probability distributions associated with Ising systems near criticality.

Disordered Systems and Neural Networks,Machine Learning
Is Deep Learning a Renormalization Group Flow?

Ellen de Mello Koch,Robert de Mello Koch,Ling Cheng

DOI: https://doi.org/10.1109/ACCESS.2020.3000901

2020-06-10

Abstract:Although there has been a rapid development of practical applications, theoretical explanations of deep learning are in their infancy. Deep learning performs a sophisticated coarse graining. Since coarse graining is a key ingredient of the renormalization group (RG), RG may provide a useful theoretical framework directly relevant to deep learning. In this study we pursue this possibility. A statistical mechanics model for a magnet, the Ising model, is used to train an unsupervised restricted Boltzmann machine (RBM). The patterns generated by the trained RBM are compared to the configurations generated through an RG treatment of the Ising model. Although we are motivated by the connection between deep learning and RG flow, in this study we focus mainly on comparing a single layer of a deep network to a single step in the RG flow. We argue that correlation functions between hidden and visible neurons are capable of diagnosing RG-like coarse graining. Numerical experiments show the presence of RG-like patterns in correlators computed using the trained RBMs. The observables we consider are also able to exhibit important differences between RG and deep learning.

Machine Learning,Statistical Mechanics,Computational Physics
A Deep Dive into the Connections Between the Renormalization Group and Deep Learning in the Ising Model

Kelsie Taylor

DOI: https://doi.org/10.7907/ztpg-z092

2023-08-22

Abstract:The renormalization group (RG) is an essential technique in statistical physics and quantum field theory, which considers scale-invariant properties of physical theories and how these theories' parameters change with scaling. Deep learning is a powerful computational technique that uses multi-layered neural networks to solve a myriad of complicated problems. Previous research suggests the possibility that unsupervised deep learning may be a form of RG flow, by being a layer-by-layer coarse graining of the original data. We examined this connection on a more rigorous basis for the simple example of Kadanoff block renormalization of the 2D nearest-neighbor Ising model, with our deep learning accomplished via Restricted Boltzmann Machines (RBMs). We developed extensive renormalization techniques for the 1D and 2D Ising model to provide a baseline for comparison. For the 1D Ising model, we successfully used Adam optimization on a correlation length loss function to learn the group flow, yielding results consistent with the analytical model for infinite N. For the 2D Ising model, we successfully generated Ising model samples using the Wolff algorithm, and performed the group flow using a quasi-deterministic method, validating these results by calculating the critical exponent \nu. We then examined RBM learning of the Ising model layer by layer, finding a blocking structure in the learning that is qualitatively similar to RG. Lastly, we directly compared the weights of each layer from the learning to Ising spin renormalization, but found quantitative inconsistencies for the simple case of nearest-neighbor Ising models.

Statistical Mechanics,Machine Learning,Neural and Evolutionary Computing
Deep neural networks for direct, featureless learning through observation: the case of 2d spin models

Kyle Mills,Isaac Tamblyn

DOI: https://doi.org/10.1103/PhysRevE.97.032119

2018-03-17

Abstract:We demonstrate the capability of a convolutional deep neural network in predicting the nearest-neighbor energy of the 4x4 Ising model. Using its success at this task, we motivate the study of the larger 8x8 Ising model, showing that the deep neural network can learn the nearest-neighbor Ising Hamiltonian after only seeing a vanishingly small fraction of configuration space. Additionally, we show that the neural network has learned both the energy and magnetization operators with sufficient accuracy to replicate the low-temperature Ising phase transition. We then demonstrate the ability of the neural network to learn other spin models, teaching the convolutional deep neural network to accurately predict the long-range interaction of a screened Coulomb Hamiltonian, a sinusoidally attenuated screened Coulomb Hamiltonian, and a modified Potts model Hamiltonian. In the case of the long-range interaction, we demonstrate the ability of the neural network to recover the phase transition with equivalent accuracy to the numerically exact method. Furthermore, in the case of the long-range interaction, the benefits of the neural network become apparent; it is able to make predictions with a high degree of accuracy, and do so 1600 times faster than a CUDA-optimized exact calculation. Additionally, we demonstrate how the neural network succeeds at these tasks by looking at the weights learned in a simplified demonstration.

Materials Science
Restricted Boltzmann Machine, recent advances and mean-field theory

Aurélien Decelle,Cyril Furtlehner

DOI: https://doi.org/10.1088/1674-1056/abd160

2021-05-28

Abstract:This review deals with Restricted Boltzmann Machine (RBM) under the light of statistical physics. The RBM is a classical family of Machine learning (ML) models which played a central role in the development of deep learning. Viewing it as a Spin Glass model and exhibiting various links with other models of statistical physics, we gather recent results dealing with mean-field theory in this context. First the functioning of the RBM can be analyzed via the phase diagrams obtained for various statistical ensembles of RBM leading in particular to identify a {\it compositional phase} where a small number of features or modes are combined to form complex patterns. Then we discuss recent works either able to devise mean-field based learning algorithms; either able to reproduce generic aspects of the learning process from some {\it ensemble dynamics equations} or/and from linear stability arguments.

Disordered Systems and Neural Networks,Statistical Mechanics,Machine Learning
Machine learning force-field model for kinetic Monte Carlo simulations of itinerant Ising magnets

Alexa Tyberg,Yunhao Fan,Gia-Wei Chern

2024-11-29

Abstract:We present a scalable machine learning (ML) framework for large-scale kinetic Monte Carlo (kMC) simulations of itinerant electron Ising systems. As the effective interactions between Ising spins in such itinerant magnets are mediated by conducting electrons, the calculation of energy change due to a local spin update requires solving an electronic structure problem. Such repeated electronic structure calculations could be overwhelmingly prohibitive for large systems. Assuming the locality principle, a convolutional neural network (CNN) model is developed to directly predict the effective local field and the corresponding energy change associated with a given spin update based on Ising configuration in a finite neighborhood. As the kernel size of the CNN is fixed at a constant, the model can be directly scalable to kMC simulations of large lattices. Our approach is reminiscent of the ML force-field models widely used in first-principles molecular dynamics simulations. Applying our ML framework to a square-lattice double-exchange Ising model, we uncover unusual coarsening of ferromagnetic domains at low temperatures. Our work highlights the potential of ML methods for large-scale modeling of similar itinerant systems with discrete dynamical variables.

Statistical Mechanics,Strongly Correlated Electrons,Machine Learning
Machine Learning Renormalization Group for Statistical Physics

Wanda Hou,Yi-Zhuang You

2023-09-12

Abstract:We develop a Machine-Learning Renormalization Group (MLRG) algorithm to explore and analyze many-body lattice models in statistical physics. Using the representation learning capability of generative modeling, MLRG automatically learns the optimal renormalization group (RG) transformations from self-generated spin configurations and formulates RG equations without human supervision. The algorithm does not focus on simulating any particular lattice model but broadly explores all possible models compatible with the internal and lattice symmetries given the on-site symmetry representation. It can uncover the RG monotone that governs the RG flow, assuming a strong form of the $c$-theorem. This enables several downstream tasks, including unsupervised classification of phases, automatic location of phase transitions or critical points, controlled estimation of critical exponents and operator scaling dimensions. We demonstrate the MLRG method in two-dimensional lattice models with Ising symmetry and show that the algorithm correctly identifies and characterizes the Ising criticality.

Statistical Mechanics,Disordered Systems and Neural Networks,Strongly Correlated Electrons
Computing critical exponents in 3D Ising model via pattern recognition/deep learning approach

Timothy A. Burt

2024-11-05

Abstract:In this study, we computed three critical exponents ($\alpha, \beta, \gamma$) for the 3D Ising model with Metropolis Algorithm using Finite-Size Scaling Analysis on six cube length scales (L=20,30,40,60,80,90), and performed a supervised Deep Learning (DL) approach (3D Convolutional Neural Network or CNN) to train a neural network on specific conformations of spin states. We find one can effectively reduce the information in thermodynamic ensemble-averaged quantities vs. reduced temperature t (magnetization per spin $<m>(t)$, specific heat per spin $<c>(t)$, magnetic susceptibility per spin $<\chi>(t)$) to \textit{six} latent classes. We also demonstrate our CNN on a subset of L=20 conformations and achieve a train/test accuracy of 0.92 and 0.6875, respectively. However, more work remains to be done to quantify the feasibility of computing critical exponents from the output class labels (binned $m, c, \chi$) from this approach and interpreting the results from DL models trained on systems in Condensed Matter Physics in general.

Computational Physics,Statistical Mechanics,Artificial Intelligence,Computer Vision and Pattern Recognition,Machine Learning
Deep learning nonlocal and scalable energy functionals for quantum Ising models

Emanuele Costa,Rosario Fazio,Sebastiano Pilati

DOI: https://doi.org/10.1103/PhysRevB.108.125113

2023-09-29

Abstract:Density functional theory (DFT) is routinely employed in material science and in quantum chemistry to simulate weakly correlated electronic systems. Recently, deep learning (DL) techniques have been adopted to develop promising functionals for the strongly correlated regime. DFT can be applied to quantum spin models too, but functionals based on DL have not been developed yet. Here, we investigate DL-based DFTs for random quantum Ising chains, both with nearest-neighbor and up to next-nearest neighbor couplings. Our neural functionals are trained and tested on data produced via the Jordan-Wigner transformation, exact diagonalization, and tensor-network methods. An economical gradient-descent optimization is used to find the ground-state properties of previously unseen Hamiltonian instances. Notably, our non-local functionals drastically improve upon the common local density approximations, and they are designed to be scalable, allowing us to simulate chain sizes much larger than those used for training. The prediction accuracy is analyzed, paying attention to the spatial correlations of the learned functionals and to the role of quantum criticality. Our findings indicate a suitable strategy to extend the reach of other computational methods with a controllable accuracy.

Strongly Correlated Electrons,Disordered Systems and Neural Networks,Computational Physics
An exact mapping between the Variational Renormalization Group and Deep Learning

Pankaj Mehta,David J. Schwab

DOI: https://doi.org/10.48550/arXiv.1410.3831

IF: 5.414

2014-10-14

Machine Learning

Abstract:Deep learning is a broad set of techniques that uses multiple layers of representation to automatically learn relevant features directly from structured data. Recently, such techniques have yielded record-breaking results on a diverse set of difficult machine learning tasks in computer vision, speech recognition, and natural language processing. Despite the enormous success of deep learning, relatively little is understood theoretically about why these techniques are so successful at feature learning and compression. Here, we show that deep learning is intimately related to one of the most important and successful techniques in theoretical physics, the renormalization group (RG). RG is an iterative coarse-graining scheme that allows for the extraction of relevant features (i.e. operators) as a physical system is examined at different length scales. We construct an exact mapping from the variational renormalization group, first introduced by Kadanoff, and deep learning architectures based on Restricted Boltzmann Machines (RBMs). We illustrate these ideas using the nearest-neighbor Ising Model in one and two-dimensions. Our results suggests that deep learning algorithms may be employing a generalized RG-like scheme to learn relevant features from data.
Influence of anisotropy on the study of critical behavior of spin models by machine learning methods

Diana Sukhoverkhova,Lev Shchur

2024-10-18

Abstract:In this paper, we applied a deep neural network to study the issue of knowledge transferability between statistical mechanics models. The following computer experiment was conducted. A convolutional neural network was trained to solve the problem of binary classification of snapshots of the Ising model's spin configuration on a two-dimensional lattice. During testing, snapshots of the Ising model spins on a lattice with diagonal ferromagnetic and antiferromagnetic connections were fed to the input of the neural network. Estimates of the probability of samples belonging to the paramagnetic phase were obtained from the outputs of the tested network. The analysis of these probabilities allowed us to estimate the critical temperature and the critical correlation length exponent. It turned out that at weak anisotropy the neural network satisfactorily predicts the transition point and the value of the correlation length exponent. Strong anisotropy leads to a noticeable deviation of the predicted values from the precisely known ones. Qualitatively, strong anisotropy is associated with the presence of oscillations of the correlation function above the Stephenson disorder temperature and further approach to the point of the fully frustrated case.

Disordered Systems and Neural Networks
Neural Networks as Spin Models: From Glass to Hidden Order Through Training

Richard Barney,Michael Winer,Victor Galitksi

2024-08-13

Abstract:We explore a one-to-one correspondence between a neural network (NN) and a statistical mechanical spin model where neurons are mapped to Ising spins and weights to spin-spin couplings. The process of training an NN produces a family of spin Hamiltonians parameterized by training time. We study the magnetic phases and the melting transition temperature as training progresses. First, we prove analytically that the common initial state before training--an NN with independent random weights--maps to a layered version of the classical Sherrington-Kirkpatrick spin glass exhibiting a replica symmetry breaking. The spin-glass-to-paramagnet transition temperature is calculated. Further, we use the Thouless-Anderson-Palmer (TAP) equations--a theoretical technique to analyze the landscape of energy minima of random systems--to determine the evolution of the magnetic phases on two types of NNs (one with continuous and one with binarized activations) trained on the MNIST dataset. The two NN types give rise to similar results, showing a quick destruction of the spin glass and the appearance of a phase with a hidden order, whose melting transition temperature $T_c$ grows as a power law in training time. We also discuss the properties of the spectrum of the spin system's bond matrix in the context of rich vs. lazy learning. We suggest that this statistical mechanical view of NNs provides a useful unifying perspective on the training process, which can be viewed as selecting and strengthening a symmetry-broken state associated with the training task.

Disordered Systems and Neural Networks,Machine Learning,Adaptation and Self-Organizing Systems
Inferring effective couplings with Restricted Boltzmann Machines

Aurélien Decelle,Cyril Furtlehner,Alfonso De Jesus Navas Gómez,Beatriz Seoane

DOI: https://doi.org/10.21468/SciPostPhys.16.4.095

2024-01-24

Abstract:Generative models offer a direct way of modeling complex data. Energy-based models attempt to encode the statistical correlations observed in the data at the level of the Boltzmann weight associated with an energy function in the form of a neural network. We address here the challenge of understanding the physical interpretation of such models. In this study, we propose a simple solution by implementing a direct mapping between the Restricted Boltzmann Machine and an effective Ising spin Hamiltonian. This mapping includes interactions of all possible orders, going beyond the conventional pairwise interactions typically considered in the inverse Ising (or Boltzmann Machine) approach, and allowing the description of complex datasets. Earlier works attempted to achieve this goal, but the proposed mappings were inaccurate for inference applications, did not properly treat the complexity of the problem, or did not provide precise prescriptions for practical application. To validate our method, we performed several controlled inverse numerical experiments in which we trained the RBMs using equilibrium samples of predefined models with local external fields, 2-body and 3-body interactions in different sparse topologies. The results demonstrate the effectiveness of our proposed approach in learning the correct interaction network and pave the way for its application in modeling interesting binary variable datasets. We also evaluate the quality of the inferred model based on different training methods.

Disordered Systems and Neural Networks,Statistical Mechanics,Machine Learning
Deep learning and the renormalization group

Cédric Bény

DOI: https://doi.org/10.48550/arXiv.1301.3124

2013-03-14

Abstract:Renormalization group (RG) methods, which model the way in which the effective behavior of a system depends on the scale at which it is observed, are key to modern condensed-matter theory and particle physics. We compare the ideas behind the RG on the one hand and deep machine learning on the other, where depth and scale play a similar role. In order to illustrate this connection, we review a recent numerical method based on the RG---the multiscale entanglement renormalization ansatz (MERA)---and show how it can be converted into a learning algorithm based on a generative hierarchical Bayesian network model. Under the assumption---common in physics---that the distribution to be learned is fully characterized by local correlations, this algorithm involves only explicit evaluation of probabilities, hence doing away with sampling.

Quantum Physics
Few-shot Machine Learning in the Three-Dimensional Ising Model

Rui Zhang,Bin Wei,Dong Zhang,Jia-Ji Zhu,Kai Chang

DOI: https://doi.org/10.1103/physrevb.99.094427

2019-01-01

Abstract:We investigate theoretically the phase transition in a three-dimensional cubic Ising model utilizing state-of-the-art machine learning algorithms. Supervised machine learning models show high accuracies (99%) in phase classification and very small relative errors (<10(-4)) of the energies in different spin configurations. Unsupervised machine learning models are introduced to study the spin configuration reconstructions and reductions, and the phases of reconstructed spin configurations can be accurately classified by a linear logistic algorithm. Based on the comparison between various machine learning models, we develop a few-shot strategy to predict phase transitions in larger lattices from a trained sample in smaller lattices. The few-shot machine learning strategy for a three-dimensional (3D) Ising model enables us to study the 3D Ising model efficiently and provides an integrated and highly accurate approach to other spin models.
Phylogenetic relationships within the lizard clade Xantusiidae: using trees and divergence times to address evolutionary questions at multiple levels.

B. Noonan,J. Pramuk,R. L. Bezy,E. A. Sinclair,K. De Queiroz,J. Sites

DOI: https://doi.org/10.1016/j.ympev.2013.05.017

IF: 5.019

2013-10-01

Molecular Phylogenetics and Evolution

Abstract:
Generating configurations of increasing lattice size with machine learning and the inverse renormalization group

Dimitrios Bachtis

2024-05-26

Abstract:We review recent developments of machine learning algorithms pertinent to the inverse renormalization group, which was originally established as a generative numerical method by Ron-Swendsen-Brandt via the implementation of compatible Monte Carlo simulations. Inverse renormalization group methods enable the iterative generation of configurations for increasing lattice size without the critical slowing down effect. We discuss the construction of inverse renormalization group transformations with the use of convolutional neural networks and present applications in models of statistical mechanics, lattice field theory, and disordered systems. We highlight the case of the three-dimensional Edwards-Anderson spin glass, where the inverse renormalization group can be employed to construct configurations for lattice volumes that have not yet been accessed by dedicated supercomputers.

High Energy Physics - Lattice,Disordered Systems and Neural Networks,Statistical Mechanics,Machine Learning
Nearest-Neighbours Neural Network architecture for efficient sampling of statistical physics models

Luca Maria Del Bono,Federico Ricci-Tersenghi,Francesco Zamponi

2024-07-28

Abstract:The task of sampling efficiently the Gibbs-Boltzmann distribution of disordered systems is important both for the theoretical understanding of these models and for the solution of practical optimization problems. Unfortunately, this task is known to be hard, especially for spin glasses at low temperatures. Recently, many attempts have been made to tackle the problem by mixing classical Monte Carlo schemes with newly devised Neural Networks that learn to propose smart moves. In this article we introduce the Nearest-Neighbours Neural Network (4N) architecture, a physically-interpretable deep architecture whose number of parameters scales linearly with the size of the system and that can be applied to a large variety of topologies. We show that the 4N architecture can accurately learn the Gibbs-Boltzmann distribution for the two-dimensional Edwards-Anderson model, and specifically for some of its most difficult instances. In particular, it captures properties such as the energy, the correlation function and the overlap probability distribution. Finally, we show that the 4N performance increases with the number of layers, in a way that clearly connects to the correlation length of the system, thus providing a simple and interpretable criterion to choose the optimal depth.

Disordered Systems and Neural Networks,Statistical Mechanics,Computational Physics
Deep learning of spatial densities in inhomogeneous correlated quantum systems

Alex Blania,Sandro Herbig,Fabian Dechent,Evert van Nieuwenburg,Florian Marquardt

DOI: https://doi.org/10.48550/arXiv.2211.09050

2022-11-17

Abstract:Machine learning has made important headway in helping to improve the treatment of quantum many-body systems. A domain of particular relevance are correlated inhomogeneous systems. What has been missing so far is a general, scalable deep-learning approach that would enable the rapid prediction of spatial densities for strongly correlated systems in arbitrary potentials. In this work, we present a straightforward scheme, where we learn to predict densities using convolutional neural networks trained on random potentials. While we demonstrate this approach in 1D and 2D lattice models using data from numerical techniques like Quantum Monte Carlo, it is directly applicable as well to training data obtained from experimental quantum simulators. We train networks that can predict the densities of multiple observables simultaneously and that can predict for a whole class of many-body lattice models, for arbitrary system sizes. We show that our approach can handle well the interplay of interference and interactions and the behaviour of models with phase transitions in inhomogeneous situations, and we also illustrate the ability to solve inverse problems, finding a potential for a desired density.

Quantum Physics,Strongly Correlated Electrons

Adult multilineage mesenchymal progenitor cells isolated from the bursa subacromialis.

Learning Spin Liquids on a Honeycomb Lattice with Artificial Neural Networks.

Deep Learning the Ising Model Near Criticality

Is Deep Learning a Renormalization Group Flow?

A Deep Dive into the Connections Between the Renormalization Group and Deep Learning in the Ising Model

Deep neural networks for direct, featureless learning through observation: the case of 2d spin models

Restricted Boltzmann Machine, recent advances and mean-field theory

Machine learning force-field model for kinetic Monte Carlo simulations of itinerant Ising magnets

Machine Learning Renormalization Group for Statistical Physics

Computing critical exponents in 3D Ising model via pattern recognition/deep learning approach

Deep learning nonlocal and scalable energy functionals for quantum Ising models

An exact mapping between the Variational Renormalization Group and Deep Learning

Influence of anisotropy on the study of critical behavior of spin models by machine learning methods

Neural Networks as Spin Models: From Glass to Hidden Order Through Training

Inferring effective couplings with Restricted Boltzmann Machines

Deep learning and the renormalization group

Few-shot Machine Learning in the Three-Dimensional Ising Model

Phylogenetic relationships within the lizard clade Xantusiidae: using trees and divergence times to address evolutionary questions at multiple levels.

Generating configurations of increasing lattice size with machine learning and the inverse renormalization group

Nearest-Neighbours Neural Network architecture for efficient sampling of statistical physics models

Deep learning of spatial densities in inhomogeneous correlated quantum systems