Abstract:Machine learning potentials (MLPs) trained on accurate quantum chemical data can retain the high accuracy, while inflicting little computational demands. On the downside, they need to be trained for each individual system. In recent years, a vast number of MLPs has been trained from scratch because learning additional data typically requires to train again on all data to not forget previously acquired knowledge. Additionally, most common structural descriptors of MLPs cannot represent efficiently a large number of different chemical elements. In this work, we tackle these problems by introducing element-embracing atom-centered symmetry functions (eeACSFs) which combine structural properties and element information from the periodic table. These eeACSFs are a key for our development of a lifelong machine learning potential (lMLP). Uncertainty quantification can be exploited to transgress a fixed, pre-trained MLP to arrive at a continuously adapting lMLP, because a predefined level of accuracy can be ensured. To extend the applicability of an lMLP to new systems, we apply continual learning strategies to enable autonomous and on-the-fly training on a continuous stream of new data. For the training of deep neural networks, we propose the continual resilient (CoRe) optimizer and incremental learning strategies relying on rehearsal of data, regularization of parameters, and the architecture of the model.

What problem does this paper attempt to address?

The paper primarily aims to address several key issues in the application of machine learning potentials (MLPs) in the fields of chemistry and materials science: 1. **Adaptability and Scalability Issues**: Existing MLPs typically need to be trained from scratch for each specific system, which means that whenever a new system is encountered, the model needs to be retrained to avoid forgetting previously learned knowledge. This has led to the development of numerous single-purpose MLPs. 2. **Efficiency of Element Descriptors**: Most common MLP structural descriptors cannot effectively represent a large number of different chemical elements, and as the number of elements increases, the computational cost rises sharply. To address these issues, the paper proposes the following methods: - **Introduction of Element-Inclusive Atomic Centered Symmetry Functions (eeACSFs)**: This new structural descriptor combines the structural characteristics around atoms with the information from the periodic table, effectively handling multiple chemical elements without significantly increasing computational complexity as the number of elements grows. - **Development of Lifelong Machine Learning Potentials (lMLP)**: By introducing uncertainty quantification and continuous learning strategies, the MLP can continuously adapt to new data streams, achieving ongoing model improvement and knowledge accumulation, thereby avoiding the phenomenon of "catastrophic forgetting." - **Proposing the Continuous Resilience (CoRe) Optimizer**: This is a new optimization algorithm for training deep neural networks, aimed at improving the efficiency of the training process and the accuracy of the final model. The CoRe optimizer combines the performance advantages of the Adam optimizer and the robustness of RPROP, and introduces mechanisms such as adaptive decay rate, weight plasticity factor, and weight decay to further accelerate convergence speed and enhance model accuracy. - **Incremental Learning Strategies**: Including data replay, parameter regularization, and model architecture adjustments, these strategies help maintain the model's memory of old data while continuously receiving new data. In summary, the core contribution of the paper lies in proposing a machine learning potential model and its associated training methods that can efficiently handle multi-element systems and possess continuous learning capabilities, providing a more general and powerful tool for atomic-scale simulations in the fields of chemistry and materials science.

Lifelong Machine Learning Potentials

Developing General Reactive Element-Based Machine Learning Potentials as the Main Computational Engine for Heterogeneous Catalysis

Machine Learning Nonadiabatic Dynamics: Eliminating Phase Freedom of Nonadiabatic Couplings with the State-Intraction State-Averaged Spin-Restricted Ensemble-Referenced Kohn-Sham Approach

Machine learning potentials with Iterative Boltzmann Inversion: training to experiment

Learning Together: Towards foundational models for machine learning interatomic potentials with meta-learning

Stable and Accurate Atomistic Simulations of Flexible Molecules using Conformationally Generalisable Machine Learned Potentials

General-purpose Machine-Learned Potential for 16 Elemental Metals and Their Alloys

Learning together: Towards foundation models for machine learning interatomic potentials with meta-learning

Machine Learning Potentials with the Iterative Boltzmann Inversion: Training to Experiment

Towards a Generalizable Machine-Learned Potential for Metal-Organic Frameworks

Efficient Training of Neural Network Potentials for Chemical and Enzymatic Reactions by Continual Learning

The Potential of Neural Network Potentials

Discrepancies and Error Evaluation Metrics for Machine Learning Interatomic Potentials

Machine Learning in QM/MM Molecular Dynamics Simulations of Condensed-Phase Systems

Accelerating fourth-generation machine learning potentials by quasi-linear scaling particle mesh charge equilibration

Large-Scale Atomic Simulation via Machine Learning Potentials Constructed by Global Potential Energy Surface Exploration

Quantum Extreme Learning of molecular potential energy surfaces and force fields

Learning Interatomic Potentials at Multiple Scales

Considerations in the use of ML interaction potentials for free energy calculations

Combining Machine Learning Potential and Structure Prediction for Accelerated Materials Design and Discovery

Machine Learning of Reactive Potentials