Abstract:The key challenge in the noisy intermediate-scale quantum era is finding useful circuits compatible with current device limitations. Variational quantum algorithms (VQAs) offer a potential solution by fixing the circuit architecture and optimizing individual gate parameters in an external loop. However, parameter optimization can become intractable, and the overall performance of the algorithm depends heavily on the initially chosen circuit architecture. Several quantum architecture search (QAS) algorithms have been developed to design useful circuit architectures automatically. In the case of parameter optimization alone, noise effects have been observed to dramatically influence the performance of the optimizer and final outcomes, which is a key line of study. However, the effects of noise on the architecture search, which could be just as critical, are poorly understood. This work addresses this gap by introducing a curriculum-based reinforcement learning QAS (CRLQAS) algorithm designed to tackle challenges in realistic VQA deployment. The algorithm incorporates (i) a 3D architecture encoding and restrictions on environment dynamics to explore the search space of possible circuits efficiently, (ii) an episode halting scheme to steer the agent to find shorter circuits, and (iii) a novel variant of simultaneous perturbation stochastic approximation as an optimizer for faster convergence. To facilitate studies, we developed an optimized simulator for our algorithm, significantly improving computational efficiency in simulating noisy quantum circuits by employing the Pauli-transfer matrix formalism in the Pauli-Liouville basis. Numerical experiments focusing on quantum chemistry tasks demonstrate that CRLQAS outperforms existing QAS algorithms across several metrics in both noiseless and noisy environments.

What problem does this paper attempt to address?

The main problem that this paper attempts to solve is how to effectively search for and design circuit architectures suitable for variational quantum algorithms (VQAs) in the presence of noise in current quantum hardware. Specifically, the paper focuses on the challenges of quantum architecture search (QAS) in practical deployment, especially the impact of parameter optimization and architecture search in a noisy environment. The authors point out that although previous studies have explored the impact of noise on parameter optimization, the impact of noise on architecture search is still unclear. Therefore, this paper aims to fill this gap by introducing a QAS method based on curriculum - reinforced learning (CRLQAS) to address the limitations and noise problems of quantum hardware in the real world. ### Main contributions: 1. **Proposing the CRLQAS method**: This is a reinforcement - learning method combined with a curriculum - learning strategy for automatically designing efficient quantum - circuit architectures. 2. **Optimizing the simulator**: An optimized simulator has been developed, which uses the Pauli transfer matrix (PTM) form to calculate in the Pauli - Liouville basis, significantly improving the computational efficiency of simulating noisy quantum circuits. 3. **Introducing multiple key techniques**: - **Illegal - action mechanism**: Prevent the generation of invalid gate sequences and reduce the search space. - **Random - stop mechanism**: Guide the agent to learn more compact circuits. - **Tensor - based binary - circuit encoding**: Capture the circuit depth and the fully - connected qubit structure. - **SPSA algorithm with adaptive momentum**: Use adaptive momentum and a variable measurement sample budget to improve the convergence speed and robustness. ### Research background: - **Variational quantum algorithms (VQAs)**: Composed of parameterized quantum circuits (PQC), quantum observables, and classical optimization routines, used to minimize the cost function. - **Quantum architecture search (QAS)**: Aims to find the optimal circuit architecture from a limited pool of quantum gates to minimize the cost function. - **Impact of noise**: Noise can cause the gradient to disappear, an increase in local optimal solutions, and changes in optimal parameters, thus affecting the performance of VQAs. ### Experimental results: - **Chemical accuracy**: CRLQAS shows better performance than existing QAS methods in the ground - state energy estimation tasks of multiple molecules (such as H2, LiH, and H2O) in both noise - free and noisy environments. - **Impact of noise level**: As the noise level increases, the number of required circuit gates and parameters also increases accordingly. ### Conclusion: The CRLQAS method proposed in this paper has made significant progress in dealing with the quantum architecture search problem in a noisy environment, providing strong support for future applications on actual quantum hardware.

Curriculum reinforcement learning for quantum architecture search under hardware errors

Curriculum reinforcement learning for quantum architecture search under hardware errors

Reinforcement learning-assisted quantum architecture search for variational quantum algorithms

Quantum circuit architecture search: error mitigation and trainability enhancement for variational quantum solvers.

Quantum Circuit Architecture Search for Variational Quantum Algorithms

Neural Predictor Based Quantum Architecture Search

Gradient-based optimization for quantum architecture search

Quantum Architecture Search with Unsupervised Representation Learning

Quantum Circuit Architecture Search on a Superconducting Processor

Reinforcement learning-based architecture search for quantum machine learning

Differentiable Quantum Architecture Search for Quantum Reinforcement Learning

Continuous evolution for efficient quantum architecture search

An RNN-policy gradient approach for quantum architecture search

Differentiable Quantum Architecture Search in Asynchronous Quantum Reinforcement Learning

A quantum information theoretic analysis of reinforcement learning-assisted quantum architecture search

An RNN–policy gradient approach for quantum architecture search

Trainability maximization using estimation of distribution algorithms assisted by surrogate modelling for quantum architecture search

Quantum Architecture Search: A Survey

Élivágar: Efficient Quantum Circuit Search for Classification