Abstract:In this day and age, depression is still one of the biggest problems in the world. If left untreated, it can lead to suicidal thoughts and attempts. There is a need for proper diagnoses of Major Depressive Disorder (MDD) and evaluation of the early stages to stop the side effects. Early detection is critical to identify a variety of serious conditions. In order to provide safe and effective protection to MDD patients, it is crucial to automate diagnoses and make decision-making tools widely available. Although there are various classification systems for the diagnosis of MDD, no reliable, secure method that meets these requirements has been established to date. In this paper, a federated deep learning-based multimodal system for MDD classification using electroencephalography (EEG) and audio datasets is presented while meeting data privacy requirements. The performance of the federated learning (FL) model was tested on independent and identically distributed (IID) and non-IID data. The study began by extracting features from several pre-trained models and ultimately decided to use bidirectional short-term memory (Bi-LSTM) as the base model, as it had the highest validation accuracy of 91% compared to a convolutional neural network and LSTM with 85% and 89% validation accuracy on audio data, respectively. The Bi-LSTM model also achieved a validation accuracy of 98.9% for EEG data. The FL method was then used to perform experiments on IID and non-IID datasets. The FL-based multimodal model achieved an exceptional training and validation accuracy of 99.9% when trained and evaluated on both IID and non-IIID datasets. These results show that the FL multimodal system performs almost as well as the Bi-LSTM multimodal system and emphasize its suitability for processing IID and non-IIID data. Several clients were found to perform better than conventional pre-trained models in a multimodal framework for federated learning using EEG and audio datasets. The proposed framework stands out from other classification techniques for MDD due to its special features, such as multimodality and data privacy for edge machines with limited resources. Due to these additional features, the framework concept is the most suitable alternative approach for the early classification of MDD patients.

A Novel Audio-Visual Information Fusion System for Mental Disorders Detection

An Intra- and Inter-Emotion Transformer-Based Fusion Model with Homogeneous and Diverse Constraints Using Multi-Emotional Audiovisual Features for Depression Detection.

Automatic Assessment of Depression from Speech Via a Hierarchical Attention Transfer Network and Attention Autoencoders

DISTINGUISHING BIPOLAR DEPRESSION FROM MAJOR DEPRESSIVE DISORDER USING FNIRS AND DEEP NEURAL NETWORK

Attention-Like Multimodality Fusion With Data Augmentation for Diagnosis of Mental Disorders Using MRI

A Hybrid Learning-Architecture for Mental Disorder Detection Using Emotion Recognition

Multimodal Sensing for Depression Risk Detection: Integrating Audio, Video, and Text Data

Multimodal temporal machine learning for Bipolar Disorder and Depression Recognition

Multimodal Deep Learning for Mental Disorders Prediction from Audio Speech Samples

Attention-Based Acoustic Feature Fusion Network for Depression Detection

ADHD Intelligent Auxiliary Diagnosis System Based on Multimodal Information Fusion.

Cross-Silo, Privacy-Preserving, and Lightweight Federated Multimodal System for the Identification of Major Depressive Disorder Using Audio and Electroencephalogram

Multimodal Depression Detection: Fusion of Electroencephalography and Paralinguistic Behaviors Using a Novel Strategy for Classifier Ensemble.

A Multimodal Approach for Detection and Assessment of Depression Using Text, Audio and Video

Audio Visual Multimodal Classification of Bipolar Disorder Episodes

Deep Learning System for Brain Image-Aided Diagnosis of Multiple Major Mental Disorders

The Verbal and Non Verbal Signals of Depression -- Combining Acoustics, Text and Visuals for Estimating Depression Level

End-to-end multimodal system for depression detection from online recordings

Depression Detection and Analysis using Large Language Models on Textual and Audio-Visual Modalities

Multimodal Spatiotemporal Representation for Automatic Depression Level Detection

A novel Image-Data-Driven and Frequency-Based method for depression detection