Abstract:Objective: The artificial pancreas (AP) has shown promising potential in achieving closed-loop glucose control for individuals with type 1 diabetes mellitus (T1DM). However, designing an effective control policy for the AP remains challenging due to the complex physiological processes, delayed insulin response, and inaccurate glucose measurements. While model predictive control (MPC) offers safety and stability through the dynamic model and safety constraints, it lacks individualization and is adversely affected by unannounced meals. Conversely, deep reinforcement learning (DRL) provides personalized and adaptive strategies but faces challenges with distribution shifts and substantial data requirements. Methods: We propose a hybrid control policy for the artificial pancreas (HyCPAP) to address the above challenges. HyCPAP combines an MPC policy with an ensemble DRL policy, leveraging the strengths of both policies while compensating for their respective limitations. To facilitate faster deployment of AP systems in real-world settings, we further incorporate meta-learning techniques into HyCPAP, leveraging previous experience and patient-shared knowledge to enable fast adaptation to new patients with limited available data. Results: We conduct extensive experiments using the FDA-accepted UVA/Padova T1DM simulator across three scenarios. Our approaches achieve the highest percentage of time spent in the desired euglycemic range and the lowest occurrences of hypoglycemia. Conclusion: The results clearly demonstrate the superiority of our methods for closed-loop glucose management in individuals with T1DM. Significance: The study presents novel control policies for AP systems, affirming the great potential of proposed methods for efficient closed-loop glucose control.

ARLPE: A Meta Reinforcement Learning Framework for Glucose Regulation in Type 1 Diabetics.

An Improved Strategy for Blood Glucose Control Using Multi-Step Deep Reinforcement Learning

Reinforcement Learning: A Paradigm Shift in Personalized Blood Glucose Management for Diabetes

Offline Deep Reinforcement Learning and Off-Policy Evaluation for Personalized Basal Insulin Control in Type 1 Diabetes

Basal-Bolus Advisor for Type 1 Diabetes (T1D) Patients Using Multi-Agent Reinforcement Learning (RL) Methodology

Optimized Glycemic Control of Type 2 Diabetes with Reinforcement Learning: a Proof-of-concept Trial.

Hybrid Control Policy for Artificial Pancreas Via Ensemble Deep Reinforcement Learning

Basal Glucose Control in Type 1 Diabetes Using Deep Reinforcement Learning: An In Silico Validation

An Insulin Bolus Advisor for Type 1 Diabetes Using Deep Reinforcement Learning

End-to-end Offline Reinforcement Learning for Glycemia Control

A Dual Mode Adaptive Basal-Bolus Advisor Based on Reinforcement Learning

Reinforcement Learning for Diabetes Blood Glucose Control with Meal Information

Offline reinforcement learning for safer blood glucose control in people with type 1 diabetes

A Deep Learning Framework for Automatic Meal Detection and Estimation in Artificial Pancreas Systems

Multivariate Learning Framework for Long-Term Adaptation in the Artificial Pancreas

Model-Based Reinforcement Learning for Type 1Diabetes Blood Glucose Control

Evaluating Deep Q-Learning Algorithms for Controlling Blood Glucose in In Silico Type 1 Diabetes

Personalized Multimorbidity Management for Patients with Type 2 Diabetes Using Reinforcement Learning of Electronic Health Records

Auto adaptation of closed-loop insulin delivery system using continuous reward functions and incremental discretization

A Multivariate Bayesian Optimization Framework for Long-Term Controller Adaptation in Artificial Pancreas

Data-enabled Learning and Control Algorithms for Intelligent Glucose Management: the State of the Art