QNMF: A quantum neural network based multimodal fusion system for intelligent diagnosis

Zhiguo Qu,Yang Li,Prayag Tiwari
DOI: https://doi.org/10.1016/j.inffus.2023.101913
IF: 18.6
2023-07-18
Information Fusion
Abstract:The Internet of Medical Things (IoMT) has emerged as a significant research area in the medical field, enabling the transmission of various types of data to the cloud for analysis and diagnosis. Fusing data from multiple modalities can enhance accuracy but necessitates substantial computing power. Theoretically, quantum computers can rapidly process large volumes of high-dimensional medical data. Despite the accelerated development of quantum computing, research on quantum machine learning (QML) for multimodal data processing remains relatively limited. Considering these factors, this paper presents a quantum neural network-based multimodal fusion system for intelligent diagnosis (QNMF), which can process multimodal medical data transmitted by IoMT devices, fuse data from different modalities, and improve the performance of the intelligent diagnosis. This system employs a quantum convolutional neural network (QCNN) to extract features from medical images efficiently. These QCNN-based features are then fused with other modalities features (such as blood test results or breast cell slices), and utilized to train an effective variational quantum classifier (VQC) for intelligent diagnosis. The experimental results demonstrate that using a QCNN can effectively extract image data features. Furthermore, QNMF achieved an accuracy of 97.07% and 97.61% on breast cancer diagnosis and Covid-19 diagnosis experiments, respectively. In addition, QNMF exhibits strong quantum noise robustness.
computer science, artificial intelligence, theory & methods
What problem does this paper attempt to address?