Abstract:Introduction Because of persistent airflow limitation in chronic obstructive pulmonary disease (COPD), patients with COPD often have complications of dyspnea. However, as a leading symptom of COPD, dyspnea in COPD deserves special consideration regarding treatment in this fragile population for pre-clinical health management in COPD. Methods: Based on the above, this paper proposes a multi-modal data combination strategy by combining the local and global features for dyspnea identification in COPD based on the multi-layer perceptron (MLP) classifier. Methods First, lung region images are automatically segmented from chest HRCT images for extracting the original 1,316 lung radiomics (OLR, 1,316) and 13,824 3D CNN features (O3C, 13,824). Second, the local features, including five selected pulmonary function test (PFT) parameters (SLF, 5), 28 selected lung radiomics (SLR, 28), and 22 selected 3D CNN features (S3C, 22), are respectively selected from the original 11 PFT parameters (OLF, 11), 1,316 OLR, and 13,824 O3C by the least absolute shrinkage and selection operator (Lasso) algorithm. Meantime, the global features, including two fused PFT parameters (FLF, 2), six fused lung radiomics (FLR, 6), and 34 fused 3D CNN features (F3C, 34), are respectively fused by 11 OLF, 1,316 OLR, and 13,824 O3C using the principal component analysis (PCA) algorithm. Finally, we combine all the local and global features (SLF + FLF + SLR + FLR + S3C + F3C, 5+ 2 + 28 + 6 + 22 + 34) for dyspnea identification in COPD based on the MLP classifier. Results Our proposed method comprehensively improves classification performance. The MLP classifier with all the local and global features achieves the best classification performance at 87.7% of accuracy, 87.7% of precision, 87.7% of recall, 87.7% of F1-scorel, and 89.3% of AUC, respectively. Discussion Compared with single-modal data, the proposed strategy effectively improves the classification performance for dyspnea identification in COPD, providing an objective and effective tool for COPD management.

Transformer-Based Multi-Modal Data Fusion Method for COPD Classification and Physiological and Biochemical Indicators Identification

Multimodal Fusion with Cross-attention Transformer for HCC Early Recurrence Prediction from Multi-Phase CT and Clinical Data

Multi-modal data combination strategy based on chest HRCT images and PFT parameters for intelligent dyspnea identification in COPD

Enhancing Heart Failure Diagnosis Through Multi-Modal Data Integration and Deep Learning

Missing-modality Enabled Multi-modal Fusion Architecture for Medical Data

CMACF: Transformer-based Cross-Modal Attention Cross-Fusion Model for Systemic Lupus Erythematosus Diagnosis Combining Raman Spectroscopy, FTIR Spectroscopy, and Metabolomics

MACTFusion: Lightweight Cross Transformer for Adaptive Multimodal Medical Image Fusion

Application of Multimodal Fusion Deep Learning Model in Disease Recognition

Disease Classification Model Based on Multi-Modal Feature Fusion

A Multi-modal Fusion Framework Based on Multi-task Correlation Learning for Cancer Prognosis Prediction

MFCSA-CAT: a Multimodal Fusion Method for Cancer Survival Analysis Based on Cross-Attention Transformer

Deep Multi-modal Fusion of Image and Non-image Data in Disease Diagnosis and Prognosis: A Review

Interpretation on Deep Multimodal Fusion for Diagnostic Classification

TAMM: A Task-Adaptive Multi-Modal Fusion Network for Facial-Related Health Assessments on 3D Facial Images

Multi-modal data Alzheimer’s disease detection based on 3D convolution

MBFusion: Multi-modal balanced fusion and multi-task learning for cancer diagnosis and prognosis

MDC-RHT: Multi-Modal Medical Image Fusion via Multi-Dimensional Dynamic Convolution and Residual Hybrid Transformer

Multimodal fusion with deep neural networks for leveraging CT imaging and electronic health record: a case-study in pulmonary embolism detection

MultiFusionNet: Multilayer Multimodal Fusion of Deep Neural Networks for Chest X-Ray Image Classification

Multimodal Audio-based Disease Prediction with Transformer-based Hierarchical Fusion Network

COVID-19 Pneumonia Classification with Transformer from Incomplete Modalities