Abstract:Quantum machine learning (QML) has been identified as one of the key fields that could reap advantages from near-term quantum devices, next to optimization and quantum chemistry. Research in this area has focused primarily on variational quantum algorithms (VQAs), and several proposals to enhance supervised, unsupervised and reinforcement learning (RL) algorithms with VQAs have been put forward. Out of the three, RL is the least studied and it is still an open question whether VQAs can be competitive with state-of-the-art classical algorithms based on neural networks (NNs) even on simple benchmark tasks. In this work, we introduce a training method for parametrized quantum circuits (PQCs) that can be used to solve RL tasks for discrete and continuous state spaces based on the deep Q-learning algorithm. We investigate which architectural choices for quantum Q-learning agents are most important for successfully solving certain types of environments by performing ablation studies for a number of different data encoding and readout strategies. We provide insight into why the performance of a VQA-based Q-learning algorithm crucially depends on the observables of the quantum model and show how to choose suitable observables based on the learning task at hand. To compare our model against the classical DQN algorithm, we perform an extensive hyperparameter search of PQCs and NNs with varying numbers of parameters. We confirm that similar to results in classical literature, the architectural choices and hyperparameters contribute more to the agents' success in a RL setting than the number of parameters used in the model. Finally, we show when recent separation results between classical and quantum agents for policy gradient RL can be extended to inferring optimal Q-values in restricted families of environments.

Quantum Reinforcement Learning Method and Application Based on Value Function

Quantum Reinforcement Learning for Multi-Armed Bandits

Quantum Multi-Armed Bandits and Stochastic Linear Bandits Enjoy Logarithmic Regrets

Safe reinforcement learning under temporal logic with reward design and quantum action selection

Control Design Of Two-Level Quantum Systems With Reinforcement Learning

A Quantum Reinforcement Learning Method For Repeated Game Theory

Bandit Algorithm Driven by a Classical Random Walk and a Quantum Walk

Quantum Computation For Action Selection Using Reinforcement Learning

Optimizing Variational Quantum Circuits Using Metaheuristic Strategies in Reinforcement Learning

Auxiliary Task-based Deep Reinforcement Learning for Quantum Control

Reinforcement Learning with Quantum Variational Circuits

Efficient quantum recurrent reinforcement learning via quantum reservoir computing

Variational Quantum Circuit Design for Quantum Reinforcement Learning on Continuous Environments

QTRL: Toward Practical Quantum Reinforcement Learning via Quantum-Train

When does reinforcement learning stand out in quantum control? A comparative study on state preparation

Provably Efficient Exploration in Quantum Reinforcement Learning with Logarithmic Worst-Case Regret

Quantum Reinforcement Learning in Non-Abelian Environments: Unveiling Novel Formulations and Quantum Advantage Exploration

An Introduction to Quantum Reinforcement Learning (QRL)

Quantum agents in the Gym: a variational quantum algorithm for deep Q-learning

Trends in quantum reinforcement learning: State‐of‐the‐arts and the road ahead

A reinforcement learning approach for quantum state engineering