Abstract:This article proposes model-free reinforcement learning methods for minimum-cost state-flipped control in Boolean control networks (BCNs). We tackle two questions: 1) finding the flipping kernel, namely, the flip set with the smallest cardinality ensuring reachability and 2) deriving optimal policies to minimize the number of flipping actions for reachability based on the obtained flipping kernel. For Question 1), Q-learning's capability in determining reachability is demonstrated. To expedite convergence, we incorporate two improvements: 1) demonstrating that previously reachable states remain reachable after adding elements to the flip set, followed by employing transfer learning and 2) initiating each episode with special initial states whose reachability to the target state set are currently unknown. For Question 2), it is challenging to encapsulate the objective of simultaneously reducing control costs and satisfying terminal constraints exclusively through the reward function employed in the Q-learning framework. To bridge the gap, we propose a BCN-characteristics-based reward scheme and prove its optimality. Questions 1) and 2) with large-scale BCNs are addressed by employing small memory Q-learning, which reduces memory usage by only recording visited action-values. An upper bound on memory usage is provided to assess the algorithm's feasibility. To expedite convergence for Question 2) in large-scale BCNs, we introduce adaptive variable rewards based on the known maximum steps needed to reach the target state set without cycles. Finally, the effectiveness of the proposed methods is validated on both small- and large-scale BCNs.

Cluster Synchronization of Boolean Control Networks with Reinforcement Learning

Cluster Synchronization of Boolean Network

Minimum-Cost State-Flipped Control for Reachability of Boolean Control Networks Using Reinforcement Learning

Generalized cluster synchronization of Boolean control networks with delays in both the states and the inputs

Reinforcement Learning Based Minimum State-flipped Control for the Reachability of Boolean Control Networks

State-flipped Control and Q-learning for Finite Horizon Output Tracking of Boolean Control Networks

State-flipped control and Q-learning algorithm for the stabilization of Boolean control networks

Cluster Synchronization of Boolean Networks under Probabilistic Function Perturbation

A state-flipped approach to complete synchronization of Boolean networks

Reachability and Synchronization of Hierarchical Boolean Control Networks

Pinning Synchronization of Large-Scale Boolean Networks

State feedback controller design for synchronization of master-slave Boolean networks based on core input-state cycles.

Feedback Controller Design for the Synchronization of Boolean Control Networks.

Stabilization and Finite-Time Stabilization of Probabilistic Boolean Control Networks

Enhanced Q-Learning Approach to Finite-Time Reachability with Maximum Probability for Probabilistic Boolean Control Networks

Rewiring Complex Networks to Achieve Cluster Synchronization Using Graph Convolution Networks With Reinforcement Learning

Weak Stabilization of Boolean Networks under State-Flipped Control

Aggregation Method to Reachability and Optimal Control of Large-Size Boolean Control Networks

Event-triggered control for the synchronization of Boolean control networks

State Feedback Controller Design for the Synchronization of Master-Slave Probabilistic Boolean Networks

Stabilizing Boolean Networks by Optimal Event-Triggered Feedback Control