DREAMS: A python framework to train deep learning models with model card reporting for medical and health applications

Rabindra Khadka,Pedro G Lind,Anis Yazidi,Asma Belhadi
2024-09-26
Abstract:Electroencephalography (EEG) data provides a non-invasive method for researchers and clinicians to observe brain activity in real time. The integration of deep learning techniques with EEG data has significantly improved the ability to identify meaningful patterns, leading to valuable insights for both clinical and research purposes. However, most of the frameworks so far, designed for EEG data analysis, are either too focused on pre-processing or in deep learning methods per, making their use for both clinician and developer communities problematic. Moreover, critical issues such as ethical considerations, biases, uncertainties, and the limitations inherent in AI models for EEG data analysis are frequently overlooked, posing challenges to the responsible implementation of these technologies. In this paper, we introduce a comprehensive deep learning framework tailored for EEG data processing, model training and report generation. While constructed in way to be adapted and developed further by AI developers, it enables to report, through model cards, the outcome and specific information of use for both developers and clinicians. In this way, we discuss how this framework can, in the future, provide clinical researchers and developers with the tools needed to create transparent and accountable AI models for EEG data analysis and diagnosis.
Artificial Intelligence
What problem does this paper attempt to address?
The main problems that this paper attempts to solve include: 1. **Limitations of existing frameworks**: - Current frameworks for EEG data analysis are either too focused on pre - processing or on deep - learning methods, lacking comprehensiveness and flexibility. This makes it difficult for these frameworks to meet the needs of clinicians and developers simultaneously. 2. **Ethical and bias issues**: - In EEG data analysis, ethical considerations are crucial, including ensuring informed consent and privacy protection during the data collection and analysis process. In addition, unresolved biases (such as selection bias and interpretation bias) may lead to distorted results, affecting the accuracy and reliability of the research. 3. **Uncertainty and model limitations**: - The complexity of EEG signals brings uncertainty and challenges that need to be recognized and quantified in model development and application. Existing AI models have interpretability problems in EEG data analysis, which may mislead conclusions and affect responsible research and clinical practice. 4. **Lack of transparency and traceability**: - The lack of proper documentation makes it difficult to assess the reliability, effectiveness, and potential biases of the model, which poses a challenge to safe and ethical use. To solve these problems, the author introduced **DREAMS** (Deep REport for AI ModelS), a Python framework aimed at providing a comprehensive solution for EEG data processing, model training, and report generation. DREAMS enhances transparency and promotes the ethical use of AI technology by recording the performance characteristics, usage guidelines, and limitations of the model in the form of model cards (Model Cards). Specifically, DREAMS can: - Provide detailed model cards covering key information such as dataset description, model architecture, and performance metrics. - Support multiple visualization tools to help users better understand the model's performance. - Adapt to different machine - learning and deep - learning models to ensure the possibility of its wide application. - Achieve content customization through YAML configuration files to meet the needs of different projects. Through these functions, DREAMS aims to provide clinical researchers and developers with the tools needed to create transparent and responsible AI models, especially in the healthcare application field.