Learning many-electron wavefunctions with deep neural networks

James Spencer

DOI: https://doi.org/10.1038/s42254-021-00330-5

IF: 36.273

2021-06-01

Nature Reviews Physics

Abstract:James Spencer explains how deep neural networks can approximate many-electron wavefunctions used in variational quantum Monte Carlo, introducing the Fermionic Neural Network or FermiNet.

physics, applied, multidisciplinary

What problem does this paper attempt to address?

Ab initio solution of the many-electron Schrödinger equation with deep neural networks

David Pfau,James S. Spencer,Alexander G. D. G. Matthews,W. M. C. Foulkes,and W. M. C. Foulkes

DOI: https://doi.org/10.1103/PhysRevResearch.2.033429

2020-09-17

Physical Review Research

Abstract:This paper presents a deep neural network that can be used as a wavefunction Ansatz for variational quantum Monte Carlo on many-electron systems. [Phys. Rev. Research 2, 033429] Published Wed Sep 16, 2020

English Else
Better, Faster Fermionic Neural Networks

James S. Spencer,David Pfau,Aleksandar Botev,W. M. C. Foulkes

DOI: https://doi.org/10.48550/arXiv.2011.07125

2020-11-13

Computational Physics

Abstract:The Fermionic Neural Network (FermiNet) is a recently-developed neural network architecture that can be used as a wavefunction Ansatz for many-electron systems, and has already demonstrated high accuracy on small systems. Here we present several improvements to the FermiNet that allow us to set new records for speed and accuracy on challenging systems. We find that increasing the size of the network is sufficient to reach chemical accuracy on atoms as large as argon. Through a combination of implementing FermiNet in JAX and simplifying several parts of the network, we are able to reduce the number of GPU hours needed to train the FermiNet on large systems by an order of magnitude. This enables us to run the FermiNet on the challenging transition of bicyclobutane to butadiene and compare against the PauliNet on the automerization of cyclobutadiene, and we achieve results near the state of the art for both.
Neural Wave Functions for Superfluids

Wan Tong Lou,Halvard Sutterud,Gino Cassella,W.M.C. Foulkes,Johannes Knolle,David Pfau,James S. Spencer

DOI: https://doi.org/10.1103/PhysRevX.14.021030

2024-06-10

Abstract:Understanding superfluidity remains a major goal of condensed matter physics. Here we tackle this challenge utilizing the recently developed Fermionic neural network (FermiNet) wave function Ansatz [D. Pfau et al., Phys. Rev. Res. 2, 033429 (2020).] for variational Monte Carlo calculations. We study the unitary Fermi gas, a system with strong, short-range, two-body interactions known to possess a superfluid ground state but difficult to describe quantitatively. We demonstrate key limitations of the FermiNet Ansatz in studying the unitary Fermi gas and propose a simple modification based on the idea of an antisymmetric geminal power singlet (AGPs) wave function. The new AGPs FermiNet outperforms the original FermiNet significantly in paired systems, giving results which are more accurate than fixed-node diffusion Monte Carlo and are consistent with experiment. We prove mathematically that the new Ansatz, which only differs from the original Ansatz by the method of antisymmetrization, is a strict generalization of the original FermiNet architecture, despite the use of fewer parameters. Our approach shares several advantages with the original FermiNet: the use of a neural network removes the need for an underlying basis set; and the flexibility of the network yields extremely accurate results within a variational quantum Monte Carlo framework that provides access to unbiased estimates of arbitrary ground-state expectation values. We discuss how the method can be extended to study other superfluids.

Quantum Gases,Superconductivity,Machine Learning,Computational Physics
Efficient solutions of fermionic systems using artificial neural networks

Even M. Nordhagen,Jane M. Kim,Bryce Fore,Alessandro Lovato,Morten Hjorth-Jensen

DOI: https://doi.org/10.3389/fphy.2023.1061580

IF: 3.718

2023-06-26

Frontiers in Physics

Abstract:In this study, we explore the similarities and differences between variational Monte Carlo techniques that employ conventional and artificial neural network representations of the ground-state wave function for fermionic systems. Our primary focus is on shallow neural network architectures, specifically the restricted Boltzmann machine, and we examine unsupervised learning algorithms that are appropriate for modeling complex many-body correlations. We assess the advantages and drawbacks of conventional and neural network wave functions by applying them to a range of circular quantum dot systems. Our findings, which include results for systems containing up to 90 electrons, emphasize the efficient implementation of these methods on both homogeneous and heterogeneous high-performance computing facilities.

physics, multidisciplinary
Solving many-electron Schrödinger equation using deep neural networks

Jiequn Han,Linfeng Zhang,E Weinan,Weinan E

DOI: https://doi.org/10.1016/j.jcp.2019.108929

IF: 4.645

2019-12-01

Journal of Computational Physics

Abstract:<p>We introduce a new family of trial wave-functions based on deep neural networks to solve the many-electron Schrödinger equation. The Pauli exclusion principle is dealt with explicitly to ensure that the trial wave-functions are physical. The optimal trial wave-function is obtained through variational Monte Carlo and the computational cost scales quadratically with the number of electrons. The algorithm does not make use of any prior knowledge such as atomic orbitals. Yet it is able to represent accurately the ground-states of the tested systems, including He, H<sub>2</sub>, Be, B, LiH, and a chain of 10 hydrogen atoms. This opens up new possibilities for solving large-scale many-electron Schrödinger equation.</p>

physics, mathematical,computer science, interdisciplinary applications
Machine-learned nodal structures of Fermion systems

William Freitas,B. Abreu,S. A. Vitiello

2024-11-05

Abstract:A major challenge in quantum physics is the accurate simulation of fermionic systems, particularly those involving strong correlations. While effective for bosonic systems, traditional quantum Monte Carlo methods encounter the notorious sign problem when applied to Fermions, often resulting in biased outcomes through the fixed-node approximation. This work demonstrates the potential of machine learning techniques to address these limitations by allowing nodal structures to be learned through gradient descent optimization iterations and the variational algorithm. Using a neural network to represent the wave function, we focus on quantum dots containing up to 30 electrons. The results show a significant reduction in the variational bias, achieving greater accuracy and a lower ground state energy than diffusion Monte Carlo with the fixed-node approximation. Our approach paves the way for precise and accurate property predictions in fermionic strongly correlated systems, advancing fundamental understanding and applications in quantum technologies.

Computational Physics,Mesoscale and Nanoscale Physics,Strongly Correlated Electrons
Unifying view of fermionic neural network quantum states: From neural network backflow to hidden fermion determinant states

Zejun Liu,Bryan K. Clark

2024-09-11

Abstract:Among the variational wave functions for Fermionic Hamiltonians, neural network backflow (NNBF) and hidden fermion determinant states (HFDS) are two prominent classes to provide accurate approximations to the ground state. Here we develop a unifying view of fermionic neural quantum states casting them all in the framework of NNBF. NNBF wave-functions have configuration-dependent single-particle orbitals (SPO) which are parameterized by a neural network. We show that HFDS with $r$ hidden fermions can be written as a NNBF with an $r \times r$ determinant Jastrow and a restricted low-rank $r$ additive correction to the SPO. Furthermore, we show that in NNBF wave-functions, such determinant Jastrow's can generically be removed at the cost of further complicating the additive SPO correction increasing its rank by $r$. We numerically and analytically compare additive SPO corrections generated by the product of two matrices with inner dimension $r$. We find that larger $r$ wave-functions span a larger space and give evidence that simpler and more direct updates to the SPO's tend to be more expressive and better energetically. These suggest the standard NNBF approach is preferred amongst other related choices. Finally, we uncover that the row-selection used to select single-particle orbitals allows significant sign and amplitude modulation between nearby configurations and is partially responsible for the quality of NNBF and HFDS wave-functions.

Disordered Systems and Neural Networks,Quantum Physics
Fermionic neural network with effective core potential

Xiang Li,Cunwei Fan,Weiluo Ren,Ji Chen

DOI: https://doi.org/10.1103/PhysRevResearch.4.013021

2022-01-11

Physical Review Research

Abstract:Deep learning techniques have opened a new venue for electronic structure theory in recent years. In contrast to traditional methods, deep neural networks provide much more expressive and flexible wave function Ansätze, resulting in better accuracy and timescale behavior. In order to study larger systems while retaining sufficient accuracy, we integrate a powerful neural-network-based model (FermiNet) with the effective core potential method, which helps to reduce the complexity of the problem by replacing inner core electrons with additional semilocal potential terms in the Hamiltonian. In this work, we calculate the ground-state energy of 3d transition metal atoms and their monoxides, which is quite challenging for the original FermiNet work, and the results are consistent with both experimental data and other state-of-the-art computational methods. Our work is an important step for a broader application of deep learning in the electronic structure calculation of molecules and materials. https://doi.org/10.1103/PhysRevResearch.4.013021 Published by the American Physical Society under the terms of the Creative Commons Attribution 4.0 International license. Further distribution of this work must maintain attribution to the author(s) and the published article's title, journal citation, and DOI. Published by the American Physical Society

English Else
Open-Source Fermionic Neural Networks with Ionic Charge Initialization

Shai Pranesh,Shang Zhu,Venkat Viswanathan,Bharath Ramsundar

2024-01-16

Abstract:Finding accurate solutions to the electronic Schrödinger equation plays an important role in discovering important molecular and material energies and characteristics. Consequently, solving systems with large numbers of electrons has become increasingly important. Variational Monte Carlo (VMC) methods, especially those approximated through deep neural networks, are promising in this regard. In this paper, we aim to integrate one such model called the FermiNet, a post-Hartree-Fock (HF) Deep Neural Network (DNN) model, into a standard and widely used open source library, DeepChem. We also propose novel initialization techniques to overcome the difficulties associated with the assignment of excess or lack of electrons for ions.

Machine Learning,Chemical Physics
Ab initio quantum chemistry with neural-network wavefunctions

Jan Hermann,James Spencer,Kenny Choo,Antonio Mezzacapo,W. M. C. Foulkes,David Pfau,Giuseppe Carleo,Frank Noé

DOI: https://doi.org/10.1038/s41570-023-00516-8

IF: 34.5695

2023-08-10

Nature Reviews Chemistry

Abstract:Deep learning methods outperform human capabilities in pattern recognition and data processing problems and now have an increasingly important role in scientific discovery. A key application of machine learning in molecular science is to learn potential energy surfaces or force fields from ab initio solutions of the electronic Schrödinger equation using data sets obtained with density functional theory, coupled cluster or other quantum chemistry (QC) methods. In this Review, we discuss a complementary approach using machine learning to aid the direct solution of QC problems from first principles. Specifically, we focus on quantum Monte Carlo methods that use neural-network ansatzes to solve the electronic Schrödinger equation, in first and second quantization, computing ground and excited states and generalizing over multiple nuclear configurations. Although still at their infancy, these methods can already generate virtually exact solutions of the electronic Schrödinger equation for small systems and rival advanced conventional QC methods for systems with up to a few dozen electrons.

chemistry, multidisciplinary
Unsupervised Deep Neural Network Approach To Solve Fermionic Systems

Avishek Singh,Nirmal Ganguli

2024-05-24

Abstract:Solving the Schrödinger equation for interacting many-body quantum systems faces computational challenges due to exponential scaling with system size. This complexity limits the study of important phenomena in materials science and physics. We develop an Artificial Neural Network (ANN)-driven algorithm to simulate fermionic systems on lattices. Our method uses Pauli matrices to represent quantum states, incorporates Markov Chain Monte Carlo sampling, and leverages an adaptive momentum optimizer. We demonstrate the algorithm's accuracy by simulating the Heisenberg Hamiltonian on a one-dimensional lattice, achieving results with an error in the order of $10^{-4}$ compared to exact diagonalization. Furthermore, we successfully model a magnetic phase transition in a two-dimensional lattice under an applied magnetic field. Importantly, our approach avoids the sign problem common to traditional Fermionic Monte Carlo methods, enabling the investigation of frustrated systems. This work demonstrates the potential of ANN-based algorithms for efficient simulation of complex quantum systems, opening avenues for discoveries in condensed matter physics and materials science.

Materials Science,Strongly Correlated Electrons
Towards a transferable fermionic neural wavefunction for molecules

Michael Scherbela,Leon Gerard,Philipp Grohs

DOI: https://doi.org/10.1038/s41467-023-44216-9

IF: 16.6

2024-01-02

Nature Communications

Abstract:Abstract Deep neural networks have become a highly accurate and powerful wavefunction ansatz in combination with variational Monte Carlo methods for solving the electronic Schrödinger equation. However, despite their success and favorable scaling, these methods are still computationally too costly for wide adoption. A significant obstacle is the requirement to optimize the wavefunction from scratch for each new system, thus requiring long optimization. In this work, we propose a neural network ansatz, which effectively maps uncorrelated, computationally cheap Hartree-Fock orbitals, to correlated, high-accuracy neural network orbitals. This ansatz is inherently capable of learning a single wavefunction across multiple compounds and geometries, as we demonstrate by successfully transferring a wavefunction model pre-trained on smaller fragments to larger compounds. Furthermore, we provide ample experimental evidence to support the idea that extensive pre-training of such a generalized wavefunction model across different compounds and geometries could lead to a foundation wavefunction model. Such a model could yield high-accuracy ab-initio energies using only minimal computational effort for fine-tuning and evaluation of observables.

multidisciplinary sciences
Ju l 2 01 8 Solving Many-Electron Schrödinger Equation Using Deep Neural Networks

Jiequn Han,Linfeng Zhang

2018-01-01

Abstract:We introduce a new family of trial wave-functions based on deep neural networks to solve the many-electron Schr̈ondinger equation. The Pauli exclusion principle is dealt with explicitly to ensure that the trial wave-functions are physical. The optimal trial wave-function is obtained through variational Monte Carlo and the computational cost scales quadratically with the number of electrons. The algorithm does not make use of any prior knowledge such as atomic orbitals. Yet it is able to represent accurately the ground-states of the tested systems, including He, H2, Be, B, LiH, and a chain of 10 hydrogen atoms. This opens up new possibilities for solving large-scale many-electron Schröndinger equation.
Artificial Neural Networks as Trial Wave Functions for Quantum Monte Carlo

Jan Kessler,Francesco Calcavecchia,Thomas D. Kühne

DOI: https://doi.org/10.1002/adts.202000269

2021-01-25

Advanced Theory and Simulations

Abstract:Inspired by the universal approximation theorem and widespread adoption of artificial neural network techniques in a diversity of fields, feed‐forward neural networks are proposed as a general purpose trial wave function for quantum Monte Carlo simulations of continuous many‐body systems. Whereas for simple model systems the whole many‐body wave function can be represented by a neural network, the antisymmetry condition of non‐trivial fermionic systems is incorporated by means of a Slater determinant. To demonstrate the accuracy of the trial wave functions, an exactly solvable model system of two trapped interacting particles, as well as the hydrogen dimer, is studied. A number of general purpose trial wave functions based on feed‐forward neural networks are proposed for quantum Monte Carlo simulations of bosons and fermions. The behavior and accuracy of the trial wave functions are investigated for an exactly solvable model system of two trapped interacting particles and the hydrogen dimer.

multidisciplinary sciences
Symmetries and many-body excited states with neural-network quantum states

Kenny Choo,Giuseppe Carleo,Nicolas Regnault,Titus Neupert

DOI: https://doi.org/10.48550/arXiv.1807.03325

2018-07-09

Strongly Correlated Electrons

Abstract:Artificial neural networks have been recently introduced as a general ansatz to compactly represent many- body wave functions. In conjunction with Variational Monte Carlo, this ansatz has been applied to find Hamil- tonian ground states and their energies. Here we provide extensions of this method to study properties of ex- cited states, a central task in several many-body quantum calculations. First, we give a prescription that allows to target eigenstates of a (nonlocal) symmetry of the Hamiltonian. Second, we give an algorithm that allows to compute low-lying excited states without symmetries. We demonstrate our approach with both Restricted Boltzmann machines states and feedforward neural networks as variational wave-functions. Results are shown for the one-dimensional spin-1/2 Heisenberg model, and for the one-dimensional Bose-Hubbard model. When comparing to available exact results, we obtain good agreement for a large range of excited-states energies. Interestingly, we also find that deep networks typically outperform shallow architectures for high-energy states.
Solving and visualizing fractional quantum Hall wavefunctions with neural network

Yi Teng,David D. Dai,Liang Fu

2024-12-01

Abstract:We introduce an attention-based fermionic neural network (FNN) to variationally solve the problem of two-dimensional Coulomb electron gas in magnetic fields, a canonical platform for fractional quantum Hall (FQH) liquids, Wigner crystals and other unconventional electron states. Working directly with the full Hilbert space of $N$ electrons confined to a disk, our FNN consistently attains energies lower than LL-projected exact diagonalization (ED) and learns the ground state wavefunction to high accuracy. In low LL mixing regime, our FNN reveals microscopic features in the short-distance behavior of FQH wavefunction beyond the Laughlin ansatz. For moderate and strong LL mixing parameters, the FNN outperforms ED significantly. Moreover, a phase transition from FQH liquid to a crystal state is found at strong LL mixing. Our study demonstrates unprecedented power and universality of FNN based variational method for solving strong-coupling many-body problems with topological order and electron fractionalization.

Strongly Correlated Electrons,Disordered Systems and Neural Networks,Quantum Physics
Deep-neural-network approach to solving the ab initio nuclear structure problem

Yilong Yang,Pengwei Zhao

DOI: https://doi.org/10.1103/PhysRevC.107.034320

2023-04-04

Abstract:Predicting the structure of quantum many-body systems from the first principles of quantum mechanics is a common challenge in physics, chemistry, and material science. Deep machine learning has proven to be a powerful tool for solving condensed matter and chemistry problems, while for atomic nuclei it is still quite challenging because of the complicated nucleon-nucleon interactions, which strongly couple the spatial, spin, and isospin degrees of freedom. By combining essential physics of the nuclear wave functions and the strong expressive power of artificial neural networks, we develop FeynmanNet, a deep-learning variational quantum Monte Carlo approach for \emph{ab initio} nuclear structure. We show that FeynmanNet can provide very accurate solutions of ground-state energies and wave functions for $^4$He, $^6$Li, and even up to $^{16}$O as emerging from the leading-order and next-to-leading-order Hamiltonians of pionless effective field theory. Compared to the conventional diffusion Monte Carlo approaches, which suffer from the severe inherent fermion-sign problem, FeynmanNet reaches such a high accuracy in a variational way and scales polynomially with the number of nucleons. Therefore, it paves the way to a highly accurate and efficient \emph{ab initio} method for predicting nuclear properties based on the realistic interactions between nucleons.

Nuclear Theory,Disordered Systems and Neural Networks,Quantum Physics

Learning many-electron wavefunctions with deep neural networks

Ab initio solution of the many-electron Schrödinger equation with deep neural networks

Better, Faster Fermionic Neural Networks

Neural Wave Functions for Superfluids

Efficient solutions of fermionic systems using artificial neural networks

Solving many-electron Schrödinger equation using deep neural networks

Machine-learned nodal structures of Fermion systems

Unifying view of fermionic neural network quantum states: From neural network backflow to hidden fermion determinant states

Fermionic neural network with effective core potential

Open-Source Fermionic Neural Networks with Ionic Charge Initialization

Ab initio quantum chemistry with neural-network wavefunctions

Unsupervised Deep Neural Network Approach To Solve Fermionic Systems

Towards a transferable fermionic neural wavefunction for molecules

Ju l 2 01 8 Solving Many-Electron Schrödinger Equation Using Deep Neural Networks

Artificial Neural Networks as Trial Wave Functions for Quantum Monte Carlo

Symmetries and many-body excited states with neural-network quantum states

Solving and visualizing fractional quantum Hall wavefunctions with neural network

Deep-neural-network approach to solving the ab initio nuclear structure problem