Multi-scale approach for the prediction of atomic scale properties

Andrea Grisafi,Jigyasa Nigam,Michele Ceriotti

DOI: https://doi.org/10.1039/d0sc04934d

IF: 8.4

2021-01-01

Chemical Science

Abstract:Multi-scale equivariant representations overcome the nearsightedness of local machine-learning approaches.

What problem does this paper attempt to address?

Predicting the properties of molecular materials: multiscale simulation workflows meet machine learning

Fabio Le Piane,Matteo Baldoni,Francesco Mercuri

DOI: https://doi.org/10.48550/arXiv.2007.14832

2020-07-29

Materials Science

Abstract:Machine Learning tools are nowadays widely applied extensively to the prediction of the properties of molecular materials, using datasets extracted from high-throughput computational models. In several cases of scientific and technological relevance, the properties of molecular materials are related to the link between molecular structure and phenomena occurring across a wide set of spatial scales, from the nanoscale to the macroscale. Here, we describe an approach for predicting the properties of molecular aggregates based on multiscale simulations and machine learning.
Deep Learning for Multi-Scale Molecular Modeling

Jun Zhang,Yaokun Lei,Yi Isaac Yang,Yi Qin Gao

DOI: https://doi.org/10.26434/chemrxiv.9640814.v3

2020-01-01

Abstract:Molecular simulations are widely applied in the study of chemical and bio-physical systems. However, theaccessible timescales of atomistic simulations are limited, and extracting equilibrium properties of systemscontaining rare events remains challenging. Two distinct strategies are usually adopted in this regard: eithersticking to the atomistic level and performing enhanced sampling, or trading details for speed by leveragingcoarse-grained models. Although both strategies are promising, either of them, if adopted individually,exhibits severe limitations. In this paper we propose a machine-learning approach to ally both strategies sothat simulations on different scales can benefit mutually from their cross-talks: Accurate coarse-grained (CG)models can be inferred from the fine-grained (FG) simulations through deep generative learning; In turn, FGsimulations can be boosted by the guidance of CG models via deep reinforcement learning. Our methoddefines a variational and adaptive training objective which allows end-to-end training of parametricmolecular models using deep neural networks. Through multiple experiments, we show that our method isefficient and flexible, and performs well on challenging chemical and bio-molecular systems.
Reinforcement Learning for Multi-Scale Molecular Modeling

Jun Zhang,Yaokun Lei,Yi Isaac Yang,Yi Qin Gao

DOI: https://doi.org/10.26434/chemrxiv.9640814.v2

2020-01-01

Abstract:Molecular simulations are widely applied in the study of chemical and bio-physical systems of interest. However, the accessible timescales of atomistic simulations are limited, and extracting equilibrium properties of systems containing rare events remains challenging. Two distinct strategies are usually adopted in this regard: either sticking to the atomistic level and performing enhanced sampling, or trading details for speed by leveraging coarse-grained models. Although both strategies are promising, either of them, if adopted individually, exhibits severe limitations. In this paper we propose a machine-learning approach to take advantage of both strategies. In this approach, simulations on different scales are executed simultaneously and benefit mutually from their cross-talks: Accurate coarse-grained (CG) models can be inferred from the fine-grained (FG) simulations; In turn, FG simulations can be boosted by the guidance of CG models. Our method grounds on unsupervised and reinforcement learning, defined by a variational and adaptive training objective, and allows end-to-end training of parametric models. Through multiple experiments, we show that our method is efficient and flexible, and performs well on challenging chemical and bio-molecular systems.
From Molecules to Materials: Pre-training Large Generalizable Models for Atomic Property Prediction

Nima Shoghi,Adeesh Kolluru,John R. Kitchin,Zachary W. Ulissi,C. Lawrence Zitnick,Brandon M. Wood

2024-05-07

Abstract:Foundation models have been transformational in machine learning fields such as natural language processing and computer vision. Similar success in atomic property prediction has been limited due to the challenges of training effective models across multiple chemical domains. To address this, we introduce Joint Multi-domain Pre-training (JMP), a supervised pre-training strategy that simultaneously trains on multiple datasets from different chemical domains, treating each dataset as a unique pre-training task within a multi-task framework. Our combined training dataset consists of $\sim$120M systems from OC20, OC22, ANI-1x, and Transition-1x. We evaluate performance and generalization by fine-tuning over a diverse set of downstream tasks and datasets including: QM9, rMD17, MatBench, QMOF, SPICE, and MD22. JMP demonstrates an average improvement of 59% over training from scratch, and matches or sets state-of-the-art on 34 out of 40 tasks. Our work highlights the potential of pre-training strategies that utilize diverse data to advance property prediction across chemical domains, especially for low-data tasks. Please visit

Machine Learning
The confluence of machine learning and multiscale simulations

Harsh Bhatia,Fikret Aydin,Timothy S Carpenter,Felice C Lightstone,Peer-Timo Bremer,Helgi I Ingólfsson,Dwight V Nissley,Frederick H Streitz

DOI: https://doi.org/10.1016/j.sbi.2023.102569

Abstract:Multiscale modeling has a long history of use in structural biology, as computational biologists strive to overcome the time- and length-scale limits of atomistic molecular dynamics. Contemporary machine learning techniques, such as deep learning, have promoted advances in virtually every field of science and engineering and are revitalizing the traditional notions of multiscale modeling. Deep learning has found success in various approaches for distilling information from fine-scale models, such as building surrogate models and guiding the development of coarse-grained potentials. However, perhaps its most powerful use in multiscale modeling is in defining latent spaces that enable efficient exploration of conformational space. This confluence of machine learning and multiscale simulation with modern high-performance computing promises a new era of discovery and innovation in structural biology.
Molecular Property Prediction: A Multilevel Quantum Interactions Modeling Perspective

Chengqiang Lu,Qi Liu,Chao Wang,Zhenya Huang,Peize Lin,Lixin He

DOI: https://doi.org/10.1609/aaai.v33i01.33011052

2019-07-17

Proceedings of the AAAI Conference on Artificial Intelligence

Abstract:Predicting molecular properties (e.g., atomization energy) is an essential issue in quantum chemistry, which could speed up much research progress, such as drug designing and substance discovery. Traditional studies based on density functional theory (DFT) in physics are proved to be time-consuming for predicting large number of molecules. Recently, the machine learning methods, which consider much rule-based information, have also shown potentials for this issue. However, the complex inherent quantum interactions of molecules are still largely underexplored by existing solutions. In this paper, we propose a generalizable and transferable Multilevel Graph Convolutional neural Network (MGCN) for molecular property prediction. Specifically, we represent each molecule as a graph to preserve its internal structure. Moreover, the well-designed hierarchical graph neural network directly extracts features from the conformation and spatial information followed by the multilevel interactions. As a consequence, the multilevel overall representations can be utilized to make the prediction. Extensive experiments on both datasets of equilibrium and off-equilibrium molecules demonstrate the effectiveness of our model. Furthermore, the detailed results also prove that MGCN is generalizable and transferable for the prediction.

English Else
A machine-learning-enhanced hierarchical multiscale method for bridging from molecular dynamics to continua

Shaoping Xiao,Renjie Hu,Zhen Li,Siamak Attarian,Kaj-Mikael Björk,Amaury Lendasse

DOI: https://doi.org/10.1007/s00521-019-04480-7

2019-09-18

Neural Computing and Applications

Abstract:In the community of computational materials science, one of the challenges in hierarchical multiscale modeling is information-passing from one scale to another, especially from the molecular model to the continuum model. A machine-learning-enhanced approach, proposed in this paper, provides an alternative solution. In the developed hierarchical multiscale method, molecular dynamics simulations in the molecular model are conducted first to generate a dataset, which represents physical phenomena at the nanoscale. The dataset is then used to train a material failure/defect classification model and stress regression models. Finally, the well-trained models are implemented in the continuum model to study the mechanical behaviors of materials at the macroscale. Multiscale modeling and simulation of a molecule chain and an aluminum crystalline solid are presented as the applications of the proposed method. In addition to support vector machines, extreme learning machines with single-layer neural networks are employed due to their computational efficiency.

computer science, artificial intelligence
Multi-scale Modeling and Computation

e weinan,bjorn engquist

2003-01-01

Notices of the American Mathematical Society

Abstract:Many applications involve phenomena that act at scales of more than one order of magnitude. This article discusses developments in numerical and analytical techniques for handling such multiscale d ...
Machine learning-assisted multi-scale modeling

Weinan E,Huan Lei,Pinchen Xie,Linfeng Zhang

DOI: https://doi.org/10.1063/5.0149861

IF: 1.469

2023-07-01

Journal of Mathematical Physics

Abstract:Neural network-based machine learning is capable of approximating functions in very high dimension with unprecedented efficiency and accuracy. This has opened up many exciting new possibilities, one of which is to use machine learning algorithms to assist multi-scale modeling. In this review, we use three examples to illustrate the process involved in using machine learning in multi-scale modeling: ab initio molecular dynamics, ab initio meso-scale models, such as Landau models and generalized Langevin equation, and hydrodynamic models for non-Newtonian flows.

physics, mathematical
Cross-Modal Learning for Chemistry Property Prediction: Large Language Models Meet Graph Machine Learning

Sakhinana Sagar Srinivas,Venkataramana Runkana

2024-08-27

Abstract:In the field of chemistry, the objective is to create novel molecules with desired properties, facilitating accurate property predictions for applications such as material design and drug screening. However, existing graph deep learning methods face limitations that curb their expressive power. To address this, we explore the integration of vast molecular domain knowledge from Large Language Models (LLMs) with the complementary strengths of Graph Neural Networks (GNNs) to enhance performance in property prediction tasks. We introduce a Multi-Modal Fusion (MMF) framework that synergistically harnesses the analytical prowess of GNNs and the linguistic generative and predictive abilities of LLMs, thereby improving accuracy and robustness in predicting molecular properties. Our framework combines the effectiveness of GNNs in modeling graph-structured data with the zero-shot and few-shot learning capabilities of LLMs, enabling improved predictions while reducing the risk of overfitting. Furthermore, our approach effectively addresses distributional shifts, a common challenge in real-world applications, and showcases the efficacy of learning cross-modal representations, surpassing state-of-the-art baselines on benchmark datasets for property prediction tasks.

Machine Learning
Equivariant representations for molecular Hamiltonians and N-center atomic-scale properties

Jigyasa Nigam,Michael Willatt,Michele Ceriotti

DOI: https://doi.org/10.1063/5.0072784

2021-12-21

Abstract:Symmetry considerations are at the core of the major frameworks used to provide an effective mathematical representation of atomic configurations that is then used in machine-learning models to predict the properties associated with each structure. In most cases, the models rely on a description of atom-centered environments, and are suitable to learn atomic properties, or global observables that can be decomposed into atomic contributions. Many quantities that are relevant for quantum mechanical calculations, however -- most notably the single-particle Hamiltonian matrix when written in an atomic-orbital basis -- are not associated with a single center, but with two (or more) atoms in the structure. We discuss a family of structural descriptors that generalize the very successful atom-centered density correlation features to the N-centers case, and show in particular how this construction can be applied to efficiently learn the matrix elements of the (effective) single-particle Hamiltonian written in an atom-centered orbital basis. These N-centers features are fully equivariant -- not only in terms of translations and rotations, but also in terms of permutations of the indices associated with the atoms -- and are suitable to construct symmetry-adapted machine-learning models of new classes of properties of molecules and materials.

Chemical Physics,Machine Learning
Unified theory of atom-centered representations and message-passing machine-learning schemes

Jigyasa Nigam,Sergey Pozdnyakov,Guillaume Fraux,Michele Ceriotti

DOI: https://doi.org/10.1063/5.0087042

2022-04-01

Abstract:Data-driven schemes that associate molecular and crystal structures with their microscopic properties share the need for a concise, effective description of the arrangement of their atomic constituents. Many types of models rely on descriptions of atom-centered environments, that are associated with an atomic property or with an atomic contribution to an extensive macroscopic quantity. Frameworks in this class can be understood in terms of atom-centered density correlations (ACDC), that are used as a basis for a body-ordered, symmetry-adapted expansion of the targets. Several other schemes, that gather information on the relationship between neighboring atoms using "message-passing" ideas, cannot be directly mapped to correlations centered around a single atom. We generalize the ACDC framework to include multi-centered information, generating representations that provide a complete linear basis to regress symmetric functions of atomic coordinates, and provides a coherent foundation to systematize our understanding of both atom-centered and message-passing, invariant and equivariant machine-learning schemes.

Machine Learning,Chemical Physics
A Vision for the Future of Multiscale Modeling

Matteo Capone,Marco Romanelli,Davide Castaldo,Giovanni Parolin,Alessandro Bello,Gabriel Gil,Mirko Vanzan

DOI: https://doi.org/10.1021/acsphyschemau.3c00080

2024-03-04

Abstract:The rise of modern computer science enabled physical chemistry to make enormous progresses in understanding and harnessing natural and artificial phenomena. Nevertheless, despite the advances achieved over past decades, computational resources are still insufficient to thoroughly simulate extended systems from first principles. Indeed, countless biological, catalytic and photophysical processes require ab initio treatments to be properly described, but the breadth of length and time scales involved makes it practically unfeasible. A way to address these issues is to couple theories and algorithms working at different scales by dividing the system into domains treated at different levels of approximation, ranging from quantum mechanics to classical molecular dynamics, even including continuum electrodynamics. This approach is known as multiscale modeling and its use over the past 60 years has led to remarkable results. Considering the rapid advances in theory, algorithm design, and computing power, we believe multiscale modeling will massively grow into a dominant research methodology in the forthcoming years. Hereby we describe the main approaches developed within its realm, highlighting their achievements and current drawbacks, eventually proposing a plausible direction for future developments considering also the emergence of new computational techniques such as machine learning and quantum computing. We then discuss how advanced multiscale modeling methods could be exploited to address critical scientific challenges, focusing on the simulation of complex light-harvesting processes, such as natural photosynthesis. While doing so, we suggest a cutting-edge computational paradigm consisting in performing simultaneous multiscale calculations on a system allowing the various domains, treated with appropriate accuracy, to move and extend while they properly interact with each other. Although this vision is very ambitious, we believe the quick development of computer science will lead to both massive improvements and widespread use of these techniques, resulting in enormous progresses in physical chemistry and, eventually, in our society.
Machine learning for structure-property relationships: Scalability and limitations

Zhongzheng Tian,Sheng Zhang,Gia-Wei Chern

DOI: https://doi.org/10.1103/PhysRevE.108.065304

2023-04-12

Abstract:We present a scalable machine learning (ML) framework for predicting intensive properties and particularly classifying phases of many-body systems. Scalability and transferability are central to the unprecedented computational efficiency of ML methods. In general, linear-scaling computation can be achieved through the divide and conquer approach, and the locality of physical properties is key to partitioning the system into sub-domains that can be solved separately. Based on the locality assumption, ML model is developed for the prediction of intensive properties of a finite-size block. Predictions of large-scale systems can then be obtained by averaging results of the ML model from randomly sampled blocks of the system. We show that the applicability of this approach depends on whether the block-size of the ML model is greater than the characteristic length scale of the system. In particular, in the case of phase identification across a critical point, the accuracy of the ML prediction is limited by the diverging correlation length. The two-dimensional Ising model is used to demonstrate the proposed framework. We obtain an intriguing scaling relation between the prediction accuracy and the ratio of ML block size over the spin-spin correlation length. Implications for practical applications are also discussed.

Statistical Mechanics,Materials Science,Machine Learning
Materials Representation and Transfer Learning for Multi-Property Prediction

Shufeng Kong,Dan Guevarra,Carla P. Gomes,John M. Gregoire

DOI: https://doi.org/10.1063/5.0047066

2021-06-18

Abstract:The adoption of machine learning in materials science has rapidly transformed materials property prediction. Hurdles limiting full capitalization of recent advancements in machine learning include the limited development of methods to learn the underlying interactions of multiple elements, as well as the relationships among multiple properties, to facilitate property prediction in new composition spaces. To address these issues, we introduce the Hierarchical Correlation Learning for Multi-property Prediction (H-CLMP) framework that seamlessly integrates (i) prediction using only a material's composition, (ii) learning and exploitation of correlations among target properties in multi-target regression, and (iii) leveraging training data from tangential domains via generative transfer learning. The model is demonstrated for prediction of spectral optical absorption of complex metal oxides spanning 69 3-cation metal oxide composition spaces. H-CLMP accurately predicts non-linear composition-property relationships in composition spaces for which no training data is available, which broadens the purview of machine learning to the discovery of materials with exceptional properties. This achievement results from the principled integration of latent embedding learning, property correlation learning, generative transfer learning, and attention models. The best performance is obtained using H-CLMP with Transfer learning (H-CLMP(T)) wherein a generative adversarial network is trained on computational density of states data and deployed in the target domain to augment prediction of optical absorption from composition. H-CLMP(T) aggregates multiple knowledge sources with a framework that is well-suited for multi-target regression across the physical sciences.

Machine Learning,Artificial Intelligence
Deep learning for variational multiscale molecular modeling

Jun Zhang,Yao-Kun Lei,Yi Isaac Yang,Yi Qin Gao

DOI: https://doi.org/10.1063/5.0026836

2020-11-07

Abstract:Molecular simulations are widely applied in the study of chemical and bio-physical problems. However, the accessible timescales of atomistic simulations are limited, and extracting equilibrium properties of systems containing rare events remains challenging. Two distinct strategies are usually adopted in this regard: either sticking to the atomistic level and performing enhanced sampling or trading details for speed by leveraging coarse-grained models. Although both strategies are promising, either of them, if adopted individually, exhibits severe limitations. In this paper, we propose a machine-learning approach to ally both strategies so that simulations on different scales can benefit mutually from their crosstalks: Accurate coarse-grained (CG) models can be inferred from the fine-grained (FG) simulations through deep generative learning; in turn, FG simulations can be boosted by the guidance of CG models via deep reinforcement learning. Our method defines a variational and adaptive training objective, which allows end-to-end training of parametric molecular models using deep neural networks. Through multiple experiments, we show that our method is efficient and flexible and performs well on challenging chemical and bio-molecular systems.
Deep Learning for Variational Multi-Scale Molecular Modeling

Jun Zhang,Yao-Kun Lei,Yi Isaac Yang,Yi Qin Gao

DOI: https://doi.org/10.1063/5.0026836

2020-01-01

Abstract:Molecular simulations are widely applied in the study of chemical and bio-physical problems. However, the accessible timescales of atomistic simulations are limited, and extracting equilibrium properties of systems containing rare events remains challenging. Two distinct strategies are usually adopted in this regard: either sticking to the atomistic level and performing enhanced sampling or trading details for speed by leveraging coarse-grained models. Although both strategies are promising, either of them, if adopted individually, exhibits severe limitations. In this paper, we propose a machine-learning approach to ally both strategies so that simulations on different scales can benefit mutually from their crosstalks: Accurate coarse-grained (CG) models can be inferred from the fine-grained (FG) simulations through deep generative learning; in turn, FG simulations can be boosted by the guidance of CG models via deep reinforcement learning. Our method defines a variational and adaptive training objective, which allows end-to-end training of parametric molecular models using deep neural networks. Through multiple experiments, we show that our method is efficient and flexible and performs well on challenging chemical and bio-molecular systems.
Distributed Representations of Atoms and Materials for Machine Learning

Luis M. Antunes,Ricardo Grau-Crespo,Keith T. Butler

DOI: https://doi.org/10.48550/arXiv.2107.14664

2021-07-30

Abstract:The use of machine learning is becoming increasingly common in computational materials science. To build effective models of the chemistry of materials, useful machine-based representations of atoms and their compounds are required. We derive distributed representations of compounds from their chemical formulas only, via pooling operations of distributed representations of atoms. These compound representations are evaluated on ten different tasks, such as the prediction of formation energy and band gap, and are found to be competitive with existing benchmarks that make use of structure, and even superior in cases where only composition is available. Finally, we introduce a new approach for learning distributed representations of atoms, named SkipAtom, which makes use of the growing information in materials structure databases.

Materials Science,Machine Learning
MultiModal-Learning for Predicting Molecular Properties: A Framework Based on Image and Graph Structures

Zhuoyuan Wang,Jiacong Mi,Shan Lu,Jieyue He

2024-04-19

Abstract:The quest for accurate prediction of drug molecule properties poses a fundamental challenge in the realm of Artificial Intelligence Drug Discovery (AIDD). An effective representation of drug molecules emerges as a pivotal component in this pursuit. Contemporary leading-edge research predominantly resorts to self-supervised learning (SSL) techniques to extract meaningful structural representations from large-scale, unlabeled molecular data, subsequently fine-tuning these representations for an array of downstream tasks. However, an inherent shortcoming of these studies lies in their singular reliance on one modality of molecular information, such as molecule image or SMILES representations, thus neglecting the potential complementarity of various molecular modalities. In response to this limitation, we propose MolIG, a novel MultiModaL molecular pre-training framework for predicting molecular properties based on Image and Graph structures. MolIG model innovatively leverages the coherence and correlation between molecule graph and molecule image to execute self-supervised tasks, effectively amalgamating the strengths of both molecular representation forms. This holistic approach allows for the capture of pivotal molecular structural characteristics and high-level semantic information. Upon completion of pre-training, Graph Neural Network (GNN) Encoder is used for the prediction of downstream tasks. In comparison to advanced baseline models, MolIG exhibits enhanced performance in downstream tasks pertaining to molecular property prediction within benchmark groups such as MoleculeNet Benchmark Group and ADMET Benchmark Group.

Machine Learning,Artificial Intelligence,Chemical Physics,Biomolecules
Graph attention neural networks for mapping materials and molecules beyond short-range interatomic correlations

Yuanbin Liu,Xin Liu,Bing-Yang Cao

DOI: https://doi.org/10.1088/1361-648x/ad2584

2024-02-04

Journal of Physics Condensed Matter

Abstract:Bringing advances in machine learning to chemical science is leading to a revolutionary change in the way of accelerating materials discovery and atomic-scale simulations. Currently, most successful machine learning schemes can be largely traced to the use of localized atomic environments in the structural representation of materials and molecules. However, this may undermine the reliability of machine learning models for mapping complex systems and describing long-range physical effects because of the lack of non-local correlations between atoms. To overcome such limitations, here we report a graph attention neural network as a unified framework to map materials and molecules into a generalizable and interpretable representation that combines local and non-local information of atomic environments from multiple scales. As an exemplary study, our model is applied to predict electronic structure properties of metal-organic frameworks (MOFs) which have notable diversity in compositions and structures. The results show that our model achieves the state-of-the-art performance. The clustering analysis further demonstrates that our model leads to a high-level identification of MOFs with spatial and chemical resolution, which would facilitate the rational design of promising reticular materials. Furthermore, the application of our model in predicting the heat capacity of complex nanoporous materials, a critical property in a carbon capture process, showcases its versatility and accuracy in handling diverse physical properties beyond electronic structures.

physics, condensed matter

Multi-scale approach for the prediction of atomic scale properties

Predicting the properties of molecular materials: multiscale simulation workflows meet machine learning

Deep Learning for Multi-Scale Molecular Modeling

Reinforcement Learning for Multi-Scale Molecular Modeling

From Molecules to Materials: Pre-training Large Generalizable Models for Atomic Property Prediction

The confluence of machine learning and multiscale simulations

Molecular Property Prediction: A Multilevel Quantum Interactions Modeling Perspective

A machine-learning-enhanced hierarchical multiscale method for bridging from molecular dynamics to continua

Multi-scale Modeling and Computation

Machine learning-assisted multi-scale modeling

Cross-Modal Learning for Chemistry Property Prediction: Large Language Models Meet Graph Machine Learning

Equivariant representations for molecular Hamiltonians and N-center atomic-scale properties

Unified theory of atom-centered representations and message-passing machine-learning schemes

A Vision for the Future of Multiscale Modeling

Machine learning for structure-property relationships: Scalability and limitations

Materials Representation and Transfer Learning for Multi-Property Prediction

Deep learning for variational multiscale molecular modeling

Deep Learning for Variational Multi-Scale Molecular Modeling

Distributed Representations of Atoms and Materials for Machine Learning

MultiModal-Learning for Predicting Molecular Properties: A Framework Based on Image and Graph Structures

Graph attention neural networks for mapping materials and molecules beyond short-range interatomic correlations