Abstract:Neural-network quantum states (NQS) has emerged as a powerful application of quantum-inspired deep learning for variational Monte Carlo methods, offering a competitive alternative to existing techniques for identifying ground states of quantum problems. A significant advancement toward improving the practical scalability of NQS has been the incorporation of autoregressive models, most recently transformers, as variational ansatze. Transformers learn sequence information with greater expressiveness than recurrent models, but at the cost of increased time complexity with respect to sequence length. We explore the use of the retentive network (RetNet), a recurrent alternative to transformers, as an ansatz for solving electronic ground state problems in $\textit{ab initio}$ quantum chemistry. Unlike transformers, RetNets overcome this time complexity bottleneck by processing data in parallel during training, and recurrently during inference. We give a simple computational cost estimate of the RetNet and directly compare it with similar estimates for transformers, establishing a clear threshold ratio of problem-to-model size past which the RetNet's time complexity outperforms that of the transformer. Though this efficiency can comes at the expense of decreased expressiveness relative to the transformer, we overcome this gap through training strategies that leverage the autoregressive structure of the model -- namely, variational neural annealing. Our findings support the RetNet as a means of improving the time complexity of NQS without sacrificing accuracy. We provide further evidence that the ablative improvements of neural annealing extend beyond the RetNet architecture, suggesting it would serve as an effective general training strategy for autoregressive NQS.

Efficiency of neural quantum states in light of the quantum geometric tensor

Neural-Network Quantum States for Spin-1 Systems: Spin-Basis and Parameterization Effects on Compactness of Representations

Scalable Imaginary Time Evolution with Neural Network Quantum States

Polynomially efficient quantum enabled variational Monte Carlo for training neural-network quantum states for physico-chemical applications

Variational Quantum-Neural Hybrid Eigensolver.

Fine-tuning Neural Network Quantum States

Comment on "Can Neural Quantum States Learn Volume-Law Ground States?"

Approximating Ground States by Neural Network Quantum States

Universal Performance Gap of Neural Quantum States Applied to the Hofstadter-Bose-Hubbard Model

Empowering deep neural quantum states through efficient optimization

Geometry of learning neural quantum states

Entanglement transition in deep neural quantum states

Optimizing Design Choices for Neural Quantum States

Retentive Neural Quantum States: Efficient Ansätze for Ab Initio Quantum Chemistry

Neural Quantum States of frustrated magnets: generalization and sign structure

A Non-stochastic Optimization Algorithm for Neural-network Quantum States

A Nonstochastic Optimization Algorithm for Neural-Network Quantum States

Neural-network quantum states for many-body physics

Neural Network Representations of Quantum Many-Body States