Abstract:The nuclear fuel loading pattern optimization problem belongs to the class of large-scale combinatorial optimization. It is also characterized by multiple objectives and constraints, which makes it impossible to solve explicitly. Stochastic optimization methodologies including Genetic Algorithms and Simulated Annealing are used by different nuclear utilities and vendors but hand-designed solutions continue to be the prevalent method in the industry. To improve the state-of-the-art, Deep Reinforcement Learning (RL), in particular, Proximal Policy Optimization is leveraged. This work presents a first-of-a-kind approach to utilize deep RL to solve the loading pattern problem and could be leveraged for any engineering design optimization. This paper is also to our knowledge the first to propose a study of the behavior of several hyper-parameters that influence the RL algorithm. The algorithm is highly dependent on multiple factors such as the shape of the objective function derived for the core design that behaves as a fudge factor that affects the stability of the learning. But also an exploration/exploitation trade-off that manifests through different parameters such as the number of loading patterns seen by the agents per episode, the number of samples collected before a policy update , and an entropy factor that increases the randomness of the policy during training. We found that RL must be applied similarly to a Gaussian Process in which the acquisition function is replaced by a parametrized policy. Then, once an initial set of hyper-parameters is found, reducing and until no more learning is observed will result in the highest sample efficiency robustly and stably. This resulted in an economic benefit of 535,000 - 642,000 $/year/plant. Future work must extend this research to multi-objective settings and comparing them to state-of-the-art implementation of stochastic optimization methods.

Reinforcement learning-trained optimisers and Bayesian optimisation for online particle accelerator tuning

Learning to Do or Learning While Doing: Reinforcement Learning and Bayesian Optimisation for Online Continuous Tuning

Injection Optimization at Particle Accelerators via Reinforcement Learning: From Simulation to Real-World Application

Large Language Models for Human-Machine Collaborative Particle Accelerator Tuning through Natural Language

Microsecond-Latency Feedback at a Particle Accelerator by Online Reinforcement Learning on Hardware

Assessment of reinforcement learning algorithms for nuclear power plant fuel optimization

Harnessing the Power of Gradient-Based Simulations for Multi-Objective Optimization in Particle Accelerators

Machine-learning approach for operating electron beam at KEK electron/positron injector linac

Bayesian Optimization Algorithms for Accelerator Physics

Efficient beam commissioning in HIPI accelerator based on reinforcement learning

Non-myopic Bayesian optimization using model-free reinforcement learning and its application to optimization in electrochemistry

Machine learning for online control of particle accelerators

Action-Attentive Deep Reinforcement Learning for Autonomous Alignment of Beamlines

CBOL-Tuner: Classifier-pruned Bayesian optimization to explore temporally structured latent spaces for particle accelerator tuning

Reactor Optimization Benchmark by Reinforcement Learning

Reinforcement Learning for Load-balanced Parallel Particle Tracing

Automation and control of laser wakefield accelerators using Bayesian optimisation

Automatic tuning of hyper-parameters of reinforcement learning algorithms using Bayesian optimization with behavioral cloning

One-shot, Offline and Production-Scalable PID Optimisation with Deep Reinforcement Learning

Prior-mean-assisted Bayesian optimization application on FRIB Front-End tunning