Abstract:This article proposes model-free reinforcement learning methods for minimum-cost state-flipped control in Boolean control networks (BCNs). We tackle two questions: 1) finding the flipping kernel, namely, the flip set with the smallest cardinality ensuring reachability and 2) deriving optimal policies to minimize the number of flipping actions for reachability based on the obtained flipping kernel. For Question 1), Q-learning's capability in determining reachability is demonstrated. To expedite convergence, we incorporate two improvements: 1) demonstrating that previously reachable states remain reachable after adding elements to the flip set, followed by employing transfer learning and 2) initiating each episode with special initial states whose reachability to the target state set are currently unknown. For Question 2), it is challenging to encapsulate the objective of simultaneously reducing control costs and satisfying terminal constraints exclusively through the reward function employed in the Q-learning framework. To bridge the gap, we propose a BCN-characteristics-based reward scheme and prove its optimality. Questions 1) and 2) with large-scale BCNs are addressed by employing small memory Q-learning, which reduces memory usage by only recording visited action-values. An upper bound on memory usage is provided to assess the algorithm's feasibility. To expedite convergence for Question 2) in large-scale BCNs, we introduce adaptive variable rewards based on the known maximum steps needed to reach the target state set without cycles. Finally, the effectiveness of the proposed methods is validated on both small- and large-scale BCNs.

State-flipped control and Q-learning algorithm for the stabilization of Boolean control networks

Weak Stabilization of Boolean Networks under State-Flipped Control

Stabilization and Set Stabilization of Switched Boolean Control Networks Via Flipping Mechanism

State-flipped Control and Q-learning for Finite Horizon Output Tracking of Boolean Control Networks

Stabilization and Finite-Time Stabilization of Probabilistic Boolean Control Networks

Robust Flipping Stabilization of Boolean Networks: A Q-learning Approach.

Asynchronous Stabilization of Boolean Control Networks with Stochastic Switched Signals.

Stabilization of Boolean Control Networks with Stochastic Impulses.

Output tracking of Boolean control networks based on event-triggered and state-flipped

Delayed Feedback Control for Stabilization of Boolean Control Networks With State Delay.

Higher Order Boolean Networks under State-Flipped Control

Partial Stabilization for Boolean Network with State Feedback Control

Stabilizing Boolean Networks by Optimal Event-Triggered Feedback Control

State Estimation via Designing Controller and State Estimation-Based Stabilization for Boolean Control Networks

Set Stabilization for Switched Boolean Control Networks.

Time-optimal State Feedback Stabilization of Switched Boolean Control Networks.

Set Stabilization and Optimal Control of Switched Multi-Valued Logical Control Networks with State-Dependent Switching Signals

Minimum-Cost State-Flipped Control for Reachability of Boolean Control Networks Using Reinforcement Learning

Reinforcement Learning Based Minimum State-flipped Control for the Reachability of Boolean Control Networks

On Algorithms for State Feedback Stabilization of Boolean Control Networks

Asymptotical Stabilization for Probabilistic Boolean Control Networks under Operators and Inputs Constraints