Reinforcement learning with thermal fluctuations at the nanoscale

Francesco Boccardo and Olivier Pierre-Louis
DOI: https://doi.org/10.1103/physreve.110.l023301
IF: 2.707
2024-09-02
Physical Review E
Abstract:Reinforcement Learning offers a framework to learn to choose actions in order to control a system. However, at small scales Brownian fluctuations limit the control of nanomachine actuation or nanonavigation and of the molecular machinery of life. We analyze this regime using the general framework of Markov decision processes. We show that at the nanoscale, while optimal control actions should bring an improvement proportional to the small ratio of the applied force times a length scale over the temperature, the learned improvement is smaller and proportional to the square of this small ratio. Consequently, the efficiency of learning, which compares the learning improvement to the theoretical optimal improvement, drops to zero. Nevertheless, these limitations can be circumvented by using actions learned at a lower temperature. These results are illustrated with simulations of the control of the shape of small particle clusters. https://doi.org/10.1103/PhysRevE.110.L023301 ©2024 American Physical Society
physics, fluids & plasmas, mathematical
What problem does this paper attempt to address?