Abstract:Accurate grading of brain tumors plays a crucial role in the diagnosis and treatment of glioma. While convolutional neural networks (CNNs) have shown promising performance in this task, their clinical applicability is still constrained by the interpretability and robustness of the models. In the conventional framework, the classification model is trained first, and then visual explanations are generated. However, this approach often leads to models that prioritize classification performance or complexity, making it difficult to achieve a precise visual explanation.Motivated by these challenges, we propose the Unified Visualization and Classification Network (UniVisNet), a novel framework that aims to improve both the classification performance and the generation of high-resolution visual explanations. UniVisNet addresses attention misalignment by introducing a subregion based attention mechanism, which replaces traditional down-sampling operations. Additionally, multiscale feature maps are fused to achieve higher resolution, enabling the generation of detailed visual explanations. To streamline the process, we introduce the Unified Visualization and Classification head (UniVisHead), which directly generates visual explanations without the need for additional separation steps.Through extensive experiments, our proposed UniVisNet consistently outperforms strong baseline classification models and prevalent visualization methods. Notably, UniVisNet achieves remarkable results on the glioma grading task, including an AUC of 94.7%, an accuracy of 89.3%, a sensitivity of 90.4%, and a specificity of 85.3%. Moreover, UniVisNet provides visually interpretable explanations that surpass existing approaches.In conclusion, UniVisNet innovatively generates visual explanations in brain tumor grading by simultaneously improving the classification performance and generating high-resolution visual explanations. This work contributes to the clinical application of deep learning, empowering clinicians with comprehensive insights into the spatial heterogeneity of glioma.

UniBrain: Unify Image Reconstruction and Captioning All in One Diffusion Model from Human Brain Activity

Brain Captioning: Decoding human brain activity into images and text

UniVisNet: A Unified Visualization and Classification Network for accurate of from MRI

MindDiffuser: Controlled Image Reconstruction from Human Brain Activity with Semantic and Structural Diffusion

MindSemantix: Deciphering Brain Visual Experiences with a Brain-Language Model

Mind-bridge: Reconstructing Visual Images Based on Diffusion Model from Human Brain Activity

Natural scene reconstruction from fMRI signals using generative latent diffusion

BrainNetDiff: Generative AI Empowers Brain Network Generation via Multimodal Diffusion Model

BrainChat: Decoding Semantic Information from fMRI using Vision-language Pretrained Models

NeuroPictor: Refining fMRI-to-Image Reconstruction via Multi-individual Pretraining and Multi-level Modulation

Neuro-Vision to Language: Enhancing Brain Recording-based Visual Reconstruction and Language Interaction

NeuroCine: Decoding Vivid Video Sequences from Human Brain Activties

UniBrain: Universal Brain MRI Diagnosis with Hierarchical Knowledge-enhanced Pre-training

Improving visual image reconstruction from human brain activity using latent diffusion models via multiple decoded inputs

DreamCatcher: Revealing the Language of the Brain with fMRI using GPT Embedding

Decoding Realistic Images from Brain Activity with Contrastive Self-supervision and Latent Diffusion

Unidirectional brain-computer interface: Artificial neural network encoding natural images to fMRI response in the visual cortex

VTVBrain: A Two-stage Brain Encoding Model for Decoding Key Neural Responses in Multimodal Contexts

UniCoRN: Unified Cognitive Signal ReconstructioN bridging cognitive signals and human language

BrainVis: Exploring the Bridge between Brain and Visual Signals via Image Reconstruction

Brain3D: Generating 3D Objects from fMRI