Abstract:We propose both a theoretical and a methodological framework to address a critical challenge in applying deep learning to physical systems: the reconciliation of non-linear expressiveness with SO(3)-equivariance in predictions of SO(3)-equivariant quantities. Inspired by covariant theory in physics, we present a solution by exploring the mathematical relationships between SO(3)-invariant and SO(3)-equivariant quantities and their representations. We first construct theoretical SO(3)-invariant quantities derived from the SO(3)-equivariant regression targets, and use these invariant quantities as supervisory labels to guide the learning of high-quality SO(3)-invariant features. Given that SO(3)-invariance is preserved under non-linear operations, the encoding process for invariant features can extensively utilize non-linear mappings, thereby fully capturing the non-linear patterns inherent in physical systems. Building on this, we propose a gradient-based mechanism to induce SO(3)-equivariant encodings of various degrees from the learned SO(3)-invariant features. This mechanism can incorporate non-linear expressive capabilities into SO(3)-equivariant representations, while theoretically preserving their equivariant properties as we prove, establishing a strong foundation for regressing complex SO(3)-equivariant targets. We apply our theory and method to the electronic-structure Hamiltonian prediction tasks, experimental results on eight benchmark databases covering multiple types of systems and challenging scenarios show substantial improvements on the state-of-the-art prediction accuracy of deep learning paradigm. Our method boosts Hamiltonian prediction accuracy by up to 40% and enhances downstream physical quantities, such as occupied orbital energy, by a maximum of 76%.

General framework for E(3)-equivariant neural network representation of density functional theory Hamiltonian

General framework for E(3)-equivariant neural network representation of density functional theory Hamiltonian

Deep-learning density functional theory Hamiltonian for efficient ab initio electronic-structure calculation

Deep Neural Network Representation of Density Functional Theory Hamiltonian

Deep learning density functional theory Hamiltonian in real space

A deep equivariant neural network approach for efficient hybrid density functional calculations

DeepH-2: Enhancing deep-learning electronic structure via an equivariant local-coordinate transformer

Improving density matrix electronic structure method by deep learning

Transferable E(3) equivariant parameterization for Hamiltonian of molecules and solids

Efficient hybrid density functional calculation by deep learning

Neural-network Density Functional Theory Based on Variational Energy Minimization

Generalizing deep learning electronic structure calculation to the plane-wave basis

A Framework of SO(3)-equivariant Non-linear Representation Learning and its Application to Electronic-Structure Hamiltonian Prediction

Improving the efficiency of ab initio electronic-structure calculations by deep learning

Deepks: A Comprehensive Data-Driven Approach Toward Chemically Accurate Density Functional Theory

Transferable E(3) Equivariant Parameterization for Hamiltonian of Molecules and Solids

Equivariant neural network for Green's functions of molecules and materials

Deep Density: Circumventing the Kohn-Sham equations via symmetry preserving neural networks

Higher-Order Equivariant Neural Networks for Charge Density Prediction in Materials

Universal Machine Learning Kohn-Sham Hamiltonian for Materials