Policy Optimization in Dynamic Bayesian Network Hybrid Models of Biomanufacturing Processes

Hua Zheng,Wei Xie,Ilya O. Ryzhov,Dongming Xie

DOI: https://doi.org/10.48550/arXiv.2105.06543

2022-07-25

Abstract:Biopharmaceutical manufacturing is a rapidly growing industry with impact in virtually all branches of medicines. Biomanufacturing processes require close monitoring and control, in the presence of complex bioprocess dynamics with many interdependent factors, as well as extremely limited data due to the high cost of experiments as well as the novelty of personalized bio-drugs. We develop a novel model-based reinforcement learning framework that can achieve human-level control in low-data environments. The model uses a dynamic Bayesian network to capture causal interdependencies between factors and predict how the effects of different inputs propagate through the pathways of the bioprocess mechanisms. This enables the design of process control policies that are both interpretable and robust against model risk. We present a computationally efficient, provably convergence stochastic gradient method for optimizing such policies. Validation is conducted on a realistic application with a multi-dimensional, continuous state variable.

Artificial Intelligence,Machine Learning

What problem does this paper attempt to address?

The problem that this paper attempts to solve is how to achieve effective process control in the biopharmaceutical manufacturing process with limited data. Specifically, biopharmaceutical manufacturing is a rapidly developing industry, involving complex biological process dynamics and numerous interdependent factors. At the same time, due to the high cost of experiments and the novelty of personalized biopharmaceuticals, the available data is very limited. These problems pose challenges to process control, especially in ensuring drug quality and improving production efficiency. To address these challenges, the paper proposes a reinforcement learning framework based on Dynamic Bayesian Network (DBN). This framework can capture the causal dependencies between factors and predict how the effects of different inputs propagate through the biological process mechanism paths. This makes the designed process control strategy not only interpretable but also robust to model risks. The paper also introduces a computationally efficient stochastic gradient method to optimize such a strategy and proves the convergence of this method. In addition, this research is verified in a practical application with multi - dimensional continuous state variables, demonstrating its effectiveness in a small - data environment.

Policy Optimization in Dynamic Bayesian Network Hybrid Models of Biomanufacturing Processes

Dynamic Modeling And Nonlinear Predictive Control Based On Partitioned Model And Nonlinear Optimization

Opportunities of Hybrid Model-based Reinforcement Learning for Cell Therapy Manufacturing Process Control

Production Optimization for Concentration and Volume-Limited Fed-Batch Reactors in Biochemical Processes

Bayesian prediction for physical models with application to the optimization of the synthesis of pharmaceutical products using chemical kinetics

Model-free Adaptive Dynamic Programming for Optimal Control of Discrete-time Affine Nonlinear System

Interpretable Biomanufacturing Process Risk and Sensitivity Analyses for Quality-by-Design and Stability Control

Human-algorithm collaborative Bayesian optimization for engineering systems

Process Quality Control Through Bayesian Optimization with Adaptive Local Convergence

Biomanufacturing Harvest Optimization with Small Data

Reinforcement learning for batch bioprocess optimization

Towards a modeling, optimization and predictive control framework for fed-batch metabolic cybergenetics

Hybrid Gaussian Process Modeling Applied to Economic Stochastic Model Predictive Control of Batch Processes

Multi-fidelity Gaussian Process for Biomanufacturing Process Modeling with Small Data

Neural Network-Based Hybrid Modeling Approach Incorporating Bayesian Optimization with Industrial Soft Sensor Application

Advances in Bayesian Probabilistic Modeling for Industrial Applications

Safe contextual Bayesian optimization integrated in industrial control for self-learning machines

A computational framework for optimal and Model Predictive Control of stochastic gene regulatory networks

Transition Constrained Bayesian Optimization via Markov Decision Processes

MFRL-BI: Design of a Model-free Reinforcement Learning Process Control Scheme by Using Bayesian Inference