Abstract:(Objective). Vagus nerve stimulation (VNS) is being investigated as a potential therapy for cardiovascular diseases including heart failure, cardiac arrhythmia, and hypertension. The lack of a systematic approach for controlling and tuning the VNS parameters poses a significant challenge. Closed-loop VNS strategies combined with artificial intelligence (AI) approaches offer a framework for systematically learning and adapting the optimal stimulation parameters. In this study, we presented an interactive AI framework using reinforcement learning (RL) for automated data-driven design of closed-loop VNS control systems in a computational study. (Approach). Multiple simulation environments with a standard application programming interface were developed to facilitate the design and evaluation of the automated data-driven closed-loop VNS control systems. These environments simulate the hemodynamic response to multi-location VNS using biophysics-based computational models of healthy and hypertensive rat cardiovascular systems in resting and exercise states. We designed and implemented the RL-based closed-loop VNS control frameworks in the context of controlling the heart rate (HR) and the mean arterial pressure (MAP) for a set point tracking task. Our experimental design included two approaches; a general policy using deep RL algorithms and a sample-efficient adaptive policy using probabilistic inference for learning and control (PILCO). (Main results). Our simulation results demonstrated the capabilities of the closed-loop RL-based approaches to learn optimal VNS control policies and to adapt to variations in the target set points and the underlying dynamics of the cardiovascular system. Our findings highlighted the trade-off between sample-efficiency and generalizability, providing insights for proper algorithm selection. Finally, we demonstrated that transfer learning improves the sample efficiency of Deep RL algorithms allowing the development of more efficient and personalized closed-loop VNS systems. (Significance). We demonstrated the capability of RL-based closed-loop VNS systems. Our approach provided a systematic adaptable framework for learning control strategies without requiring prior knowledge about the underlying dynamics.

Neuron model-free control of drug infusion rate

Double neuron model-free control for pH processes

Neuron Controller With Adaptive Regulated Gain For Paper Machine

Nonlinear adaptive control based on RBF networks and multi-model method

A Pid Controller With Neuron Tuning Parameters For Multi-Model Plants

Neural network model based predictive control for multivariable nonlinear systems

Adaptive nonlinear model predictive control for a class of multivariable chemical processes

Model-free Neural Control of a Class of Nonlinear Plants

Design Of Model Predictive Controller Based On Neural Systems

Delay-Dependent Output-Feedback Control for Blood Pressure Regulation Using LPV Techniques

Continuous Drug Infusion for Diabetes Therapy: A Closed-Loop Control System Design

A Model-Based Architecture for Automatic Anesthesia Infusion

An Adaptive Neuro-fuzzy Strategy in Closed-loop Control of Anesthesia

Neuron Model-Free Pid Control

Support Vector Regression Based Nonlinear Model Reference Adaptive Control

Online Reinforcement Learning Neural Network Controller Design for Nanomanipulation

Developing Physics-Informed Neural Networks for Model Predictive Control of Periodic Counter-Current Chromatography

Automated Multi-Drugs Administration During Total Intravenous Anesthesia Using Multi-Model Predictive Control

Model-free Control Based on Neural Networks

Reinforcement Learning for Closed-loop Regulation of Cardiovascular System with Vagus Nerve Stimulation: A Computational Study

A Robust and Adaptive Control Algorithm for Closed-Loop Brain Stimulation