Molecular Autonomous Pathfinder using Deep Reinforcement Learning

Ken-ichi Nomura,Ankit Mishra,Tian Sang,Rajiv K. Kalia,Aiichiro Nakano,Priya Vashishta

2023-12-09

Abstract:Diffusion in solids is a slow process that dictates rate-limiting processes in key chemical reactions. Unlike crystalline solids that offer well-defined diffusion pathways, the lack of similar structural motifs in amorphous or glassy materials poses a great scientific challenge in estimating slow diffusion time. To tackle this problem, we have developed an AI-guided long-time atomistic simulation approach: Molecular Autonomous Pathfinder (MAP) framework based on Deep Reinforcement Learning (RL), where RL agent is trained to uncover energy efficient diffusion pathways. We employ Deep Q-Network architecture with distributed prioritized replay buffer enabling fully online agent training with accelerated experience sampling by an ensemble of asynchronous agents. After training, the agents provide atomistic configurations of diffusion pathways with their energy profile. We use a piecewise Nudged Elastic Band to refine the energy profile of the obtained pathway and corresponding diffusion time on the basis of transition state theory. With MAP, we have successfully identified atomistic mechanisms along molecular diffusion pathways in amorphous silica, with time scales comparable to experiments.

Materials Science

What problem does this paper attempt to address?

The problem that this paper attempts to solve is: in amorphous or glassy materials, due to the lack of crystal - like structural features, estimating the time scale of slow diffusion processes poses a great challenge. Specifically, the diffusion of water molecules in silicate glass is a very slow process, which plays the role of a rate - limiting step in many chemical reactions. For example, during the stress - corrosion cracking (SCC) process, water molecules react with the stretched Si - O bonds at the crack tip, leading to crack propagation. However, traditional molecular dynamics simulation methods cannot directly simulate such long - time - scale diffusion processes because of high computational costs. To solve this problem, the author has developed an atomic - level simulation framework based on deep reinforcement learning (DRL) - Molecular Autonomous Pathfinder (MAP). This framework trains agents to automatically explore energy - efficient diffusion paths and combines transition state theory (TST) to accurately calculate the diffusion time. Specifically, the MAP framework utilizes the Deep Q - Network (DQN) architecture and the distributed prioritized replay buffer to achieve fully online agent training, thereby accelerating empirical sampling and improving the efficiency of diffusion path discovery. In summary, this research aims to provide an efficient and general method for studying molecular diffusion mechanisms in amorphous materials, especially on experimental - relevant time scales, by combining AI technology and physical simulation. This not only helps to understand the mechanical responses of silicate materials but also has important significance for fields such as Earth and planetary science.

Molecular Autonomous Pathfinder using Deep Reinforcement Learning

Molecular Autonomous Pathfinder Using Deep Reinforcement Learning

Learning to Cooperate: Application of Deep Reinforcement Learning for Online AGV Path Finding.

Predictive Hierarchical Reinforcement Learning for Path-Efficient Mapless Navigation with Moving Target.

Learning Mean First Passage Time: Chemical Short-Range Order and Kinetics of Diffusive Relaxation

Ab-Initio Molecular Dynamics Acceleration Scheme with an Adaptive Machine Learning Framework

Scalable Fragment-Based 3D Molecular Design with Reinforcement Learning

Autonomous artificial intelligence discovers mechanisms of molecular self-organization in virtual experiments

Computing Transition Pathways for the Study of Rare Events Using Deep Reinforcement Learning

Reinforcement learning of optimal active particle navigation

Diffused Task-Agnostic Milestone Planner

Enabling high throughput deep reinforcement learning with first principles to investigate catalytic reaction mechanisms

Simulations of Disordered Matter in 3D with the Morphological Autoregressive Protocol (MAP) and Convolutional Neural Networks

Single-Particle Diffusion Characterization by Deep Learning

DiffMD: A Geometric Diffusion Model for Molecular Dynamics Simulations

Efficient Navigation of Colloidal Robots in an Unknown Environment via Deep Reinforcement Learning

Deep reinforcement learning for microstructural optimisation of silica aerogels

Automated Multiscale Approach To Predict Self-Diffusion from a Potential Energy Field

Reinforcement Learning for Multi-Scale Molecular Modeling

Path Planning of Autonomous Mobile Robot in Comprehensive Unknown Environment Using Deep Reinforcement Learning