Abstract:We propose both a theoretical and a methodological framework to address a critical challenge in applying deep learning to physical systems: the reconciliation of non-linear expressiveness with SO(3)-equivariance in predictions of SO(3)-equivariant quantities. Inspired by covariant theory in physics, we present a solution by exploring the mathematical relationships between SO(3)-invariant and SO(3)-equivariant quantities and their representations. We first construct theoretical SO(3)-invariant quantities derived from the SO(3)-equivariant regression targets, and use these invariant quantities as supervisory labels to guide the learning of high-quality SO(3)-invariant features. Given that SO(3)-invariance is preserved under non-linear operations, the encoding process for invariant features can extensively utilize non-linear mappings, thereby fully capturing the non-linear patterns inherent in physical systems. Building on this, we propose a gradient-based mechanism to induce SO(3)-equivariant encodings of various degrees from the learned SO(3)-invariant features. This mechanism can incorporate non-linear expressive capabilities into SO(3)-equivariant representations, while theoretically preserving their equivariant properties as we prove, establishing a strong foundation for regressing complex SO(3)-equivariant targets. We apply our theory and method to the electronic-structure Hamiltonian prediction tasks, experimental results on eight benchmark databases covering multiple types of systems and challenging scenarios show substantial improvements on the state-of-the-art prediction accuracy of deep learning paradigm. Our method boosts Hamiltonian prediction accuracy by up to 40% and enhances downstream physical quantities, such as occupied orbital energy, by a maximum of 76%.

Transferable E(3) Equivariant Parameterization for Hamiltonian of Molecules and Solids

Transferable E(3) equivariant parameterization for Hamiltonian of molecules and solids

Transferable equivariant graph neural networks for the Hamiltonians of molecules and solids

General framework for E(3)-equivariant neural network representation of density functional theory Hamiltonian

Universal Machine Learning Kohn-Sham Hamiltonian for Materials

Transferability in Machine Learning for Electronic Structure via the Molecular Orbital Basis

Self-Consistency Training for Density-Functional-Theory Hamiltonian Prediction

A Framework of SO(3)-equivariant Non-linear Representation Learning and its Application to Electronic-Structure Hamiltonian Prediction

A deep equivariant neural network approach for efficient hybrid density functional calculations

Deep-learning density functional theory Hamiltonian for efficient ab initio electronic-structure calculation

Bridging electronic and classical density-functional theory using universal machine-learned functional approximations

Active learning of effective Hamiltonian for super-large-scale atomic structures

Neural network representation of electronic structure from $ab$ $initio$ molecular dynamics

Efficient hybrid density functional calculation by deep learning

Deep learning density functional theory Hamiltonian in real space

Constructing Accurate and Efficient General-Purpose Atomistic Machine Learning Model with Transferable Accuracy for Quantum Chemistry

MLTB: Enhancing Transferability and Extensibility of Density Functional Tight Binding Theory with Multi-body Interaction Corrections

Machine-learned approximations to Density Functional Theory Hamiltonians