Model-free Distortion Canceling and Control of Quantum Devices

Ahmed F. Fouad,Akram Youssry,Ahmed El-Rafei,Sherif Hammad
2024-07-13
Abstract:Quantum devices need precise control to achieve their full capability. In this work, we address the problem of controlling closed quantum systems, tackling two main issues. First, in practice the control signals are usually subject to unknown classical distortions that could arise from the device fabrication, material properties and/or instruments generating those signals. Second, in most cases modeling the system is very difficult or not even viable due to uncertainties in the relations between some variables and inaccessibility to some measurements inside the system. In this paper, we introduce a general model-free control approach based on deep reinforcement learning (DRL), that can work for any closed quantum system. We train a deep neural network (NN), using the REINFORCE policy gradient algorithm to control the state probability distribution of a closed quantum system as it evolves, and drive it to different target distributions. We present a novel controller architecture that comprises multiple NNs. This enables accommodating as many different target state distributions as desired, without increasing the complexity of the NN or its training process. The used DRL algorithm works whether the control problem can be modeled as a Markov decision process (MDP) or a partially observed MDP. Our method is valid whether the control signals are discrete- or continuous-valued. We verified our method through numerical simulations based on a photonic waveguide array chip. We trained a controller to generate sequences of different target output distributions of the chip with fidelity higher than 99%, where the controller showed superior performance in canceling the classical signal distortions.
Quantum Physics,Machine Learning,Systems and Control
What problem does this paper attempt to address?
The problems that this paper attempts to solve mainly focus on two major challenges encountered when controlling closed - quantum systems: 1. **The control signal is subject to unknown classical distortion**: In practical applications, due to factors such as the device manufacturing process, material properties, or the instruments generating these signals, the control signal is usually subject to unknown classical distortion. These distortions will deform the control signal before it acts on the quantum system, thus affecting the evolution of the system. 2. **Difficulties in system modeling**: In most cases, due to the uncertainty in the relationships between internal variables of the system or the inability to access some internal measurement values of the system, it is very difficult or even impossible to model the quantum system. This makes traditional model - based control methods difficult to implement. To address the above challenges, the paper proposes a model - free control method based on deep reinforcement learning (DRL). This method does not rely on an accurate model of the system but learns the optimal control strategy through interaction with the environment (i.e., the quantum system). Specifically, the main contributions of the paper include: - **Proposing a novel controller architecture**: This architecture consists of multiple neural networks, and each neural network corresponds to a target - state probability distribution. This design allows the controller to handle any number of target - state distributions without increasing the complexity or training difficulty of the network. - **Applicable to continuous and discrete action spaces**: The proposed control method is applicable not only to discrete action spaces but also to continuous action spaces, which makes this method more flexible in practical applications. - **Capable of handling partially observable Markov decision processes (POMDP)**: Even if the next state of the system depends not only on the current state and action but also on historical actions, the proposed controller can still work effectively. Verified by numerical simulation, this method performs excellently when controlling a voltage - controlled optical waveguide array chip. It can drive the output of the system to different target probability distributions and effectively compensate for the classical distortion in the control signal.