Abstract:Background and objective: COVID-19 outbreak has become one of the most challenging problems for human being. It is a communicable disease caused by a new coronavirus strain, which infected over 375 million people already and caused almost 6 million deaths. This paper aims to develop and design a framework for early diagnosis and fast classification of COVID-19 symptoms using multimodal Deep Learning techniques. Methods: we collected chest X-ray and cough sample data from open source datasets, Cohen and datasets and local hospitals. The features are extracted from the chest X-ray images are extracted from chest X-ray datasets. We also used cough audio datasets from Coswara project and local hospitals. The publicly available Coughvid DetectNow and Virufy datasets are used to evaluate COVID-19 detection based on speech sounds, respiratory, and cough. The collected audio data comprises slow and fast breathing, shallow and deep coughing, spoken digits, and phonation of sustained vowels. Gender, geographical location, age, preexisting medical conditions, and current health status (COVID-19 and Non-COVID-19) are recorded. Results: The proposed framework uses the selection algorithm of the pre-trained network to determine the best fusion model characterized by the pre-trained chest X-ray and cough models. Third, deep chest X-ray fusion by discriminant correlation analysis is used to fuse discriminatory features from the two models. The proposed framework achieved recognition accuracy, specificity, and sensitivity of 98.91%, 96.25%, and 97.69%, respectively. With the fusion method we obtained 94.99% accuracy. Conclusion: This paper examines the effectiveness of well-known ML architectures on a joint collection of chest-X-rays and cough samples for early classification of COVID-19. It shows that existing methods can effectively used for diagnosis and suggesting that the fusion learning paradigm could be a crucial asset in diagnosing future unknown illnesses. The proposed framework supports health informatics basis on early diagnosis, clinical decision support, and accurate prediction.

Research on Classification of COVID-19 Chest X-Ray Image Modal Feature Fusion Based on Deep Learning

Deep Learning-based Multi-Class COVID-19 Classification with X-ray Images

Deep feature fusion classification network (DFFCNet): Towards accurate diagnosis of COVID-19 using chest X-rays images

Classification of COVID-19 in Chest X-ray Images Using Fusion of Deep Features and LightGBM

COVID-19 classification using chest X-ray images based on fusion-assisted deep Bayesian optimization and Grad-CAM visualization

Multi-modal fusion of deep transfer learning based COVID-19 diagnosis and classification using chest x-ray images

Artificial Neural Network-Based Deep Learning Model for COVID-19 Patient Detection Using X-Ray Chest Images

A novel hand-crafted with deep learning features based fusion model for COVID-19 diagnosis and classification using chest X-ray images

Chest X-ray Classification for the Detection of COVID-19 Using Deep Learning Techniques

Deep Learning-Based System Combining Chest X-Ray and Computerized Tomography Images for COVID-19 Diagnosis

An efficient mixture of deep and machine learning models for COVID-19 diagnosis in chest X-ray images

A novel multimodal fusion framework for early diagnosis and accurate classification of COVID-19 patients using X-ray images and speech signal processing techniques

A Deep Learning Network for Detecting COVID-19 from Chest X-Ray Images

COVID‐19 detection from chest X-ray images using transfer learning

Detecting COVID-19 in Chest X-Ray Images via MCFF-Net

A New Method for Detecting COVID-19 Combining Deep Learning with Improved Histogram Equalization and Median Filter

COVID-19 Case Recognition from Chest CT Images by Deep Learning, Entropy-Controlled Firefly Optimization, and Parallel Feature Fusion

Comparison of Classification Models Based on Deep learning on COVID-19 Chest X-Rays

COVID-19 classification by CCSHNet with deep fusion using transfer learning and discriminant correlation analysis

DeepCOVID-Fuse: A Multi-Modality Deep Learning Model Fusing Chest X-rays and Clinical Variables to Predict COVID-19 Risk Levels

A deep learning-based COVID-19 classification from chest X-ray image: case study