Abstract:Background and objective: COVID-19 outbreak has become one of the most challenging problems for human being. It is a communicable disease caused by a new coronavirus strain, which infected over 375 million people already and caused almost 6 million deaths. This paper aims to develop and design a framework for early diagnosis and fast classification of COVID-19 symptoms using multimodal Deep Learning techniques. Methods: we collected chest X-ray and cough sample data from open source datasets, Cohen and datasets and local hospitals. The features are extracted from the chest X-ray images are extracted from chest X-ray datasets. We also used cough audio datasets from Coswara project and local hospitals. The publicly available Coughvid DetectNow and Virufy datasets are used to evaluate COVID-19 detection based on speech sounds, respiratory, and cough. The collected audio data comprises slow and fast breathing, shallow and deep coughing, spoken digits, and phonation of sustained vowels. Gender, geographical location, age, preexisting medical conditions, and current health status (COVID-19 and Non-COVID-19) are recorded. Results: The proposed framework uses the selection algorithm of the pre-trained network to determine the best fusion model characterized by the pre-trained chest X-ray and cough models. Third, deep chest X-ray fusion by discriminant correlation analysis is used to fuse discriminatory features from the two models. The proposed framework achieved recognition accuracy, specificity, and sensitivity of 98.91%, 96.25%, and 97.69%, respectively. With the fusion method we obtained 94.99% accuracy. Conclusion: This paper examines the effectiveness of well-known ML architectures on a joint collection of chest-X-rays and cough samples for early classification of COVID-19. It shows that existing methods can effectively used for diagnosis and suggesting that the fusion learning paradigm could be a crucial asset in diagnosing future unknown illnesses. The proposed framework supports health informatics basis on early diagnosis, clinical decision support, and accurate prediction.

The EIHW-GLAM Deep Attentive Multi-model Fusion System for Cough-based COVID-19 Recognition in the DiCOVA 2021 Challenge

EIHW-MTG: Second DiCOVA Challenge System Report

EIHW-MTG DiCOVA 2021 Challenge System Report

A Residual Network based Deep Learning Model for Detection of COVID-19 from Cough Sounds

PANACEA Cough Sound-Based Diagnosis of COVID-19 for the DiCOVA 2021 Challenge.

Attention-based hybrid CNN-LSTM and spectral data augmentation for COVID-19 diagnosis from cough sound

Acoustic COVID-19 Detection Using Multiple Instance Learning

Fused Audio Instance and Representation for Respiratory Disease Detection

A novel multimodal fusion framework for early diagnosis and accurate classification of COVID-19 patients using X-ray images and speech signal processing techniques

SRIB Submission to Interspeech 2021 DiCOVA Challenge

Detection of COVID-19 from Deep Breathing Sounds Using Sound Spectrum with Image Augmentation and Deep Learning Techniques

Supervised and Self-supervised Pretraining Based COVID-19 Detection Using Acoustic Breathing/Cough/Speech Signals

Deep learning based cough detection camera using enhanced features

Cough2COVID-19 detection using an enhanced multi layer ensemble deep learning framework and CoughFeatureRanker

Towards reliable respiratory disease diagnosis based on cough sounds and vision transformers

Pay Attention to the cough: Early Diagnosis of COVID-19 using Interpretable Symptoms Embeddings with Cough Sound Signal Processing

Exploring Self-Supervised Representation Ensembles for COVID-19 Cough Classification

CMC-COV19D: Contrastive Mixup Classification for COVID-19 Diagnosis

DML‐OFA: Deep mutual learning with online feature alignment for the detection of COVID‐19 from chest x‐ray images

COVID-19 Detection System: A Comparative Analysis of System Performance Based on Acoustic Features of Cough Audio Signals