Abstract:Very recently, studies have shown that quantum neural networks surpass classical neural networks in tasks like image classification when a similar number of learnable parameters are used. However, the development and optimization of quantum models are currently hindered by issues such as qubit instability and limited qubit availability, leading to error-prone systems with weak performance. In contrast, classical models can exhibit high-performance owing to substantial resource availability. As a result, more studies have been focusing on hybrid classical-quantum integration. A line of research particularly focuses on transfer learning through classical-quantum integration or quantum-quantum approaches. Unlike previous studies, this paper introduces a new method to transfer knowledge from classical to quantum neural networks using knowledge distillation, effectively bridging the gap between classical machine learning and emergent quantum computing techniques. We adapt classical convolutional neural network (CNN) architectures like LeNet and AlexNet to serve as teacher networks, facilitating the training of student quantum models by sending supervisory signals during backpropagation through KL-divergence. The approach yields significant performance improvements for the quantum models by solely depending on classical CNNs, with quantum models achieving an average accuracy improvement of 0.80% on the MNIST dataset and 5.40% on the more complex Fashion MNIST dataset. Applying this technique eliminates the cumbersome training of huge quantum models for transfer learning in resource-constrained settings and enables re-using existing pre-trained classical models to improve performance.Thus, this study paves the way for future research in quantum machine learning (QML) by positioning knowledge distillation as a core technique for advancing QML applications.

Transfer learning for scalability of neural-network quantum states

Quantum Algorithms for Feedforward Neural Networks

Bridging Classical and Quantum Machine Learning: Knowledge Transfer From Classical to Quantum Neural Networks Using Knowledge Distillation

Transfer learning in predicting quantum many-body dynamics: from physical observables to entanglement entropy

Classical-to-quantum convolutional neural network transfer learning

A simple linear algebra identity to optimize Large-Scale Neural Network Quantum States

Predicting Quantum Many-Body Dynamics with Transferable Neural Networks

Quantum Transfer Learning for Real-World, Small, and High-Dimensional Datasets

Neural networks in quantum many-body physics: a hands-on tutorial

Transformer neural networks and quantum simulators: a hybrid approach for simulating strongly correlated systems

Quantum neural networks force fields generation

Scalable quantum measurement error mitigation via conditional independence and transfer learning

Splitting and Parallelizing of Quantum Convolutional Neural Networks for Learning Translationally Symmetric Data

Quantum enhancements for deep reinforcement learning in large spaces

An Invitation to Distributed Quantum Neural Networks

How To Use Neural Networks To Investigate Quantum Many-Body Physics

Solving Quantum Master Equations with Deep Quantum Neural Networks

Completely quantum neural networks

On quantum backpropagation, information reuse, and cheating measurement collapse

Neural-Network Quantum States: from Condensed Matter to Quantum Computing

ShadowNet for Data-Centric Quantum System Learning