Abstract:This paper describes a real-time beam tuning method with an improved asynchronous advantage actor–critic (A3C) algorithm for accelerator systems. The operating parameters of devices are usually inconsistent with the predictions of physical designs because of errors in mechanical matching and installation. Therefore, parameter optimization methods such as pointwise scanning, evolutionary algorithms (EAs), and robust conjugate direction search are widely used in beam tuning to compensate for this inconsistency. However, it is difficult for them to deal with a large number of discrete local optima. The A3C algorithm, which has been applied in the automated control field, provides an approach for improving multi-dimensional optimization. The A3C algorithm is introduced and improved for the real-time beam tuning code for accelerators. Experiments in which optimization is achieved by using pointwise scanning, the genetic algorithm (one kind of EAs), and the A3C-algorithm are conducted and compared to optimize the currents of four steering magnets and two solenoids in the low-energy beam transport section (LEBT) of the Xi’an Proton Application Facility. Optimal currents are determined when the highest transmission of a radio frequency quadrupole (RFQ) accelerator downstream of the LEBT is achieved. The optimal work points of the tuned accelerator were obtained with currents of 0 A, 0 A, 0 A, and 0.1 A, for the four steering magnets, and 107 A and 96 A for the two solenoids. Furthermore, the highest transmission of the RFQ was 91.2%. Meanwhile, the lower time required for the optimization with the A3C algorithm was successfully verified. Optimization with the A3C algorithm consumed 42% and 78% less time than pointwise scanning with random initialization and pre-trained initialization of weights, respectively.

Efficient beam commissioning in HIPI accelerator based on reinforcement learning

Machine learning for online control of particle accelerators

Beam matching adaptive control via extremum seeking

Injection Optimization at Particle Accelerators via Reinforcement Learning: From Simulation to Real-World Application

Extremum Seeking Adaptive Control of Beam Envelope in Particle Accelerators

Application of the asynchronous advantage actor–critic machine learning algorithm to real-time accelerator tuning

Machine-learning approach for operating electron beam at KEK electron/positron injector linac

Action-Attentive Deep Reinforcement Learning for Autonomous Alignment of Beamlines

Reinforcement learning-trained optimisers and Bayesian optimisation for online particle accelerator tuning

Initial Application of Machine Learning for Beam Parameter Optimization at the Hefei Light Source II

Trend-Based SAC Beam Control Method with Zero-Shot in Superconducting Linear Accelerator

Bridging the gap between machine learning and particle accelerator physics with high-speed, differentiable simulations

A versatile framework for attitude tuning of beamlines at advanced light sources

Machine-learning approach for operating electron beam at KEK $e^-/e^+$ injector Linac

Application of Deep Reinforcement Learning in Beam Offset Calibration of MEBT at C-ADS Injector-II

Beyond PID Controllers: PPO with Neuralized PID Policy for Proton Beam Intensity Control in Mu2e

More Sample-Efficient Tuning of Particle Accelerators with Bayesian Optimization and Prior Mean Models

Real-time Beam Transport Simulator for Wide Energy-Spread Proton Beam Generated by Laser Acceleration

Synthesizing Particle-in-Cell Simulations Through Learning and GPU Computing for Hybrid Particle Accelerator Beamlines

Research on the simulation environment of hard X-ray nanoprobe beamline station

Start-to-end Modeling and Transmission Efficiency Optimization for a Cyclotron-Based Proton Therapy Beamline