Abstract:Those experiencing strokes, traumatic brain injuries, and drug complications can often end up hospitalized and diagnosed with coma or locked-in syndrome. Such mental impediments can permanently alter the neurological pathways in work and significantly decrease the quality of life (QoL). It is critical to translate brain signals into images to gain a deeper understanding of the thoughts of a comatose patient. Traditionally, brain signals collected by an EEG could only be translated into text, but with the novel method of an open-source model available on GitHub, DreamDiffusion can be used to convert brain waves into images directly. DreamDiffusion works by extracting features from EEG signals and then using the features to create images through StableDiffusion. Upon this, we made further improvements that could make StableDiffusion the forerunner technology in waves to media translation. In our study, we begin by modifying the existing DreamDiffusion codebase so that it does not require any prior setup, avoiding any confusing steps needed to run the model from GitHub. For many researchers, the incomplete setup process, errors in the existing code, and a lack of directions made it nearly impossible to run, not even considering the model's performance. We brought the code into Google Colab so users could run and evaluate problems cell-by-cell, eliminating the specific file and repository dependencies. We also provided the original training data file so users do not need to purchase the necessary computing power to train the model from the given dataset. The second change is utilizing the mutability of the code and optimizing the model so it can be used to generate images from other given inputs, such as sleep data. Additionally, the affordability of EEG technology allows for global dissemination and creates the opportunity for those who want to work on the shared DreamDiffusion model.

AudioDiffusion: Generating High-Quality Audios from EEG Signals : Reconstructing Audio from EEG Signals

DreamDiffusion: Generating High-Quality Images from Brain EEG Signals

DreamDiffusion: High-Quality EEG-to-Image Generation with Temporal Masked Signal Modeling and CLIP Alignment

Enhancing EEG Signal Generation through a Hybrid Approach Integrating Reinforcement Learning and Diffusion Models

EzAudio: Enhancing Text-to-Audio Generation with Efficient Diffusion Transformer

Multimodal Speech Recognition Using EEG and Audio Signals: A Novel Approach for Enhancing ASR Systems

Diff-ETS: Learning a Diffusion Probabilistic Model for Electromyography-to-Speech Conversion

AudioEditor: A Training-Free Diffusion-Based Audio Editing Framework

EEG Synthetic Data Generation Using Probabilistic Diffusion Models

EEGDfus: A Conditional Diffusion Model for Fine-Grained EEG Denoising

BrainECHO: Semantic Brain Signal Decoding through Vector-Quantized Spectrogram Reconstruction for Whisper-Enhanced Text Generation

EDMSound: Spectrogram Based Diffusion Models for Efficient and High-Quality Audio Synthesis

E3 TTS: Easy End-to-End Diffusion-based Text to Speech

Neurodm: Decoding and Visualizing Human Brain Activity with Eeg-Guided Diffusion Model

Diff-E: Diffusion-based Learning for Decoding Imagined Speech EEG

EEG-Based Speech Decoding: A Novel Approach Using Multi-Kernel Ensemble Diffusion Models

A Novel EEG Based Directed Transfer Function for Investigating Human Perception to Audio Noise.

A Novel Approach to Image EEG Sleep Data for Improving Quality of Life in Patients Suffering From Brain Injuries Using DreamDiffusion

Enhancing EEG-to-Text Decoding through Transferable Representations from Pre-trained Contrastive EEG-Text Masked Autoencoder

Predicting Different Acoustic Features from EEG and towards direct synthesis of Audio Waveform from EEG

Generative AI Enables EEG Super-Resolution via Spatio-Temporal Adaptive Diffusion Learning