U-RISC: an Ultra-high Resolution EM 1 Dataset Challenging Existing Deep 2 Learning Algorithms 3

Ruohua Shi,Wenyao Wang,Zhixuan Li,Liuyuan He,Kaiwen Sheng,Lei Ma,Kai Du,Tingting Jiang,Tiejun Huang

2021-01-01

Abstract:Connectomics is a developing ﬁled aiming at reconstructing the connection of neural

What problem does this paper attempt to address?

U-RISC: An Annotated Ultra-High-Resolution Electron Microscopy Dataset Challenging the Existing Deep Learning Algorithms

Ruohua Shi,Wenyao Wang,Zhixuan Li,Liuyuan He,Kaiwen Sheng,Lei Ma,Kai Du,Tingting Jiang,Tiejun Huang

DOI: https://doi.org/10.3389/fncom.2022.842760

IF: 3.387

2022-04-11

Frontiers in Computational Neuroscience

Abstract:Connectomics is a developing field aiming at reconstructing the connection of the neural system at the nanometer scale. Computer vision technology, especially deep learning methods used in image processing, has promoted connectomic data analysis to a new era. However, the performance of the state-of-the-art (SOTA) methods still falls behind the demand of scientific research. Inspired by the success of ImageNet, we present an annotated ultra-high resolution image segmentation dataset for cell membrane (U-RISC), which is the largest cell membrane-annotated electron microscopy (EM) dataset with a resolution of 2.18 nm/pixel. Multiple iterative annotations ensured the quality of the dataset. Through an open competition, we reveal that the performance of current deep learning methods still has a considerable gap from the human level, different from ISBI 2012, on which the performance of deep learning is closer to the human level. To explore the causes of this discrepancy, we analyze the neural networks with a visualization method, which is an attribution analysis. We find that the U-RISC requires a larger area around a pixel to predict whether the pixel belongs to the cell membrane or not. Finally, we integrate the currently available methods to provide a new benchmark (0.67, 10% higher than the leader of the competition, 0.61) for cell membrane segmentation on the U-RISC and propose some suggestions in developing deep learning algorithms. The U-RISC dataset and the deep learning codes used in this study are publicly available.

neurosciences,mathematical & computational biology
Fast and Lightweight Network Improves Serial Brain Section Stitching

Lianchao Wang,Jiajia Chen,Wei Gong,Ke Si

DOI: https://doi.org/10.1117/12.3005396

2023-01-01

Abstract:High-resolution three-dimensional brain image reconstruction is crucial for understanding the brain. Light sheet microscopy combined with tissue clearing imaging plays a pivotal role in analyzing the micro-level structure of mammalian brains. However, the complex multi-level stitching process poses challenges such as non-overlapping areas, surface deformation, and tissue loss, resulting in incomplete or discontinuous tissue structures at the junctions. These issues not only impact the precision of the atlas but also complicate subsequent analyses like cell counting and neuron tracing. To address these issues, we propose a rapid deep learning-based image inpainting approach for accurate neuron reconstruction and analysis. Our approach involves initially employing conventional registration algorithms to preliminarily stitch brain sections together, followed by utilizing a neural network to predict and restore missing tissue with a thickness exceeding 10 µm. This process enhances the structural continuity and integrity between adjacent brain slices. Compared to the original 3D U-Net and ResNet models, our approach performs better and has a processing speed that is five times faster than the original 3D U-Net. Moreover, our method enables more accurate cell counting by repairing incomplete cell bodies, leading to an average improvement of 37.37% in the number of cell bodies accurately counted near the slice junction. By integrating this novel 3D image inpainting network into brain reconstruction processes, our research opens new avenues for a more detailed and accurate investigation of neural circuitry and neurological disorders.
TF-Unet:An Automatic Cardiac MRI Image Segmentation Method

Zhenyin Fu,Jin Zhang,Ruyi Luo,Yutong Sun,Dongdong Deng,Ling Xia

DOI: https://doi.org/10.3934/mbe.2022244

2022-01-01

Mathematical Biosciences and Engineering

Abstract:Personalized heart models are widely used to study the mechanisms of cardiac arrhythmias and have been used to guide clinical ablation of different types of arrhythmias in recent years. MRI images are now mostly used for model building. In cardiac modeling studies, the degree of segmentation of the heart image determines the success of subsequent 3D reconstructions. Therefore, a fully automated segmentation is needed. In this paper, we combine U-Net and Transformer as an alternative approach to perform powerful and fully automated segmentation of medical images. On the one hand, we use convolutional neural networks for feature extraction and spatial encoding of inputs to fully exploit the advantages of convolution in detail grasping; on the other hand, we use Transformer to add remote dependencies to high-level features and model features at different scales to fully exploit the advantages of Transformer. The results show that, the average dice coefficients for ACDC and Synapse datasets are 91.72 and 85.46%, respectively, and compared with Swin-Unet, the segmentation accuracy are improved by 1.72% for ACDC dataset and 6.33% for Synapse dataset.
DeU-Net 2.0: Enhanced deformable U-Net for 3D cardiac cine MRI segmentation

Shunjie Dong,Zixuan Pan,Yu Fu,Qianqian Yang,Yuanxue Gao,Tianbai Yu,Yiyu Shi,Cheng Zhuo

DOI: https://doi.org/10.1016/j.media.2022.102389

IF: 10.9

2022-01-01

Medical Image Analysis

Abstract:Automatic segmentation of cardiac magnetic resonance imaging (MRI) facilitates efficient and accurate volume measurement in clinical applications. However, due to anisotropic resolution, ambiguous borders and complicated shapes, existing methods suffer from the degradation of accuracy and robustness in cardiac MRI segmentation. In this paper, we propose an enhanced Deformable U-Net (DeU-Net) for 3D cardiac cine MRI segmentation, composed of three modules, namely Temporal Deformable Aggregation Module (TDAM), Enhanced Deformable Attention Network (EDAN), and Probabilistic Noise Correction Module (PNCM). TDAM first takes consecutive cardiac MR slices (including a target slice and its neighboring reference slices) as input, and extracts spatio-temporal information by an offset prediction network to generate fused features of the target slice. Then the fused features are also fed into EDAN that exploits several flexible deformable convolutional layers and generates clear borders of every segmentation map. A Multi-Scale Attention Module (MSAM) in EDAN is proposed to capture long range dependencies between features of different scales. Meanwhile, PNCM treats the fused features as a distribution to quantify uncertainty. Experimental results show that our DeU-Net achieves the state-of-the-art performance in terms of the commonly used evaluation metrics on the Extended ACDC dataset and competitive performance on other two datasets, validating the robustness and generalization of DeU-Net. (c) 2022 Elsevier B.V. All rights reserved.
High-Speed Automated Reconstruction of Drosophila Larval Brain from Volumetric EM Data

Ziao Liu,Yi Zhang,Yifei Yu,Yixuan Sun,Yijun Wang,Yinhui He,Qianhui Zhao,Nenggan Zheng,Zhefeng Gong,Linqing Feng

DOI: https://doi.org/10.1109/embc40787.2023.10340599

2023-01-01

Abstract:To uncover the relationship between neural activity and behavior, it is essential to reconstruct neural circuits. However, methods typically used for neuron reconstruction from volumetric electron microscopy (EM) dataset are often time-consuming and require extensive manual proofreading, making it difficult to reproduce in a typical laboratory setting. To address this challenge, we have developed a set of acceleration techniques that build upon the Flood Filling Network (FFN), significantly reducing the time required for this task. These techniques can be easily adapted to other similar datasets and laboratory settings. To validate our approach, we tested our pipeline on a dataset of Drosophila larval brain serial section EM images at synaptic-resolution level. Our results demonstrate that our pipeline significantly reduces the inference time compared to the FFN baseline method and greatly reduces the time required for reconstructing the 3D morphology of neurons.
Deep residual contextual and subpixel convolution network for automated neuronal structure segmentation in micro-connectomics

Chi Xiao,Bei Hong,Jing Liu,Yuanyan Tang,Qiwei Xie,Hua Han

DOI: https://doi.org/10.1016/j.cmpb.2022.106759

IF: 6.1

2022-06-01

Computer Methods and Programs in Biomedicine

Abstract:Background and Objective: The goal of micro-connectomics research is to reconstruct the connectome and elucidate the mechanisms and functions of the nervous system via electron microscopy (EM). Due to the enormous variety of neuronal structures, neuron segmentation is among most difficult tasks in connectome reconstruction, and neuroanatomists desperately need a reliable neuronal structure segmentation method to reduce the burden of manual labeling and validation. Methods: In this article, we proposed an effective deep learning method based on a deep residual contextual and subpixel convolution network to obtain the neuronal structure segmentation in anisotropic EM image stacks. Furthermore, lifted multicut is used for post-processing to optimize the prediction and obtain the reconstruction results. Results: On the ISBI EM segmentation challenge, the proposed method ranks among the top of the leader board and yields a Rand score of 0.98788. On the public data set of mouse piriform cortex, it achieves a Rand score of 0.9562 and 0.9318 in the different testing stacks. The evaluation scores of our method are significantly improved when compared with those of state-of-the-art methods. Conclusions: The proposed automatic method contributes to the development of micro-connectomics, which improves the accuracy of neuronal structure segmentation and provides neuroanatomists with an effective approach to obtain the segmentation and reconstruction of neurons.

engineering, biomedical,computer science, interdisciplinary applications,medical informatics, theory & methods
Large Scale Image Segmentation with Structured Loss Based Deep Learning for Connectome Reconstruction

Jan Funke,Fabian Tschopp,William Grisaitis,Arlo Sheridan,Chandan Singh,Stephan Saalfeld,Srinivas C Turaga

DOI: https://doi.org/10.1109/TPAMI.2018.2835450

Abstract:We present a method combining affinity prediction with region agglomeration, which improves significantly upon the state of the art of neuron segmentation from electron microscopy (EM) in accuracy and scalability. Our method consists of a 3D U-Net, trained to predict affinities between voxels, followed by iterative region agglomeration. We train using a structured loss based on Malis, encouraging topologically correct segmentations obtained from affinity thresholding. Our extension consists of two parts: First, we present a quasi-linear method to compute the loss gradient, improving over the original quadratic algorithm. Second, we compute the gradient in two separate passes to avoid spurious gradient contributions in early training stages. Our predictions are accurate enough that simple learning-free percentile-based agglomeration outperforms more involved methods used earlier on inferior predictions. We present results on three diverse EM datasets, achieving relative improvements over previous results of 27, 15, and 250 percent. Our findings suggest that a single method can be applied to both nearly isotropic block-face EM data and anisotropic serial sectioned EM data. The runtime of our method scales linearly with the size of the volume and achieves a throughput of $\sim$∼ 2.6 seconds per megavoxel, qualifying our method for the processing of very large datasets.
AxonEM Dataset: 3D Axon Instance Segmentation of Brain Cortical Regions

Donglai Wei,Kisuk Lee,Hanyu Li,Ran Lu,J. Alexander Bae,Zequan Liu,Lifu Zhang,Márcia dos Santos,Zudi Lin,Thomas Uram,Xueying Wang,Ignacio Arganda-Carreras,Brian Matejek,Narayanan Kasthuri,Jeff Lichtman,Hanspeter Pfister

DOI: https://doi.org/10.48550/arXiv.2107.05451

2021-07-12

Computer Vision and Pattern Recognition

Abstract:Electron microscopy (EM) enables the reconstruction of neural circuits at the level of individual synapses, which has been transformative for scientific discoveries. However, due to the complex morphology, an accurate reconstruction of cortical axons has become a major challenge. Worse still, there is no publicly available large-scale EM dataset from the cortex that provides dense ground truth segmentation for axons, making it difficult to develop and evaluate large-scale axon reconstruction methods. To address this, we introduce the AxonEM dataset, which consists of two 30x30x30 um^3 EM image volumes from the human and mouse cortex, respectively. We thoroughly proofread over 18,000 axon instances to provide dense 3D axon instance segmentation, enabling large-scale evaluation of axon reconstruction methods. In addition, we densely annotate nine ground truth subvolumes for training, per each data volume. With this, we reproduce two published state-of-the-art methods and provide their evaluation results as a baseline. We publicly release our code and data at https://connectomics-bazaar.github.io/proj/AxonEM/index.html to foster the development of advanced methods.
A hierarchically annotated dataset drives tangled filament recognition in digital neuron reconstruction

Wu Chen,Mingwei Liao,Shengda Bao,Sile An,Wenwei Li,Xin Liu,Ganghua Huang,Hui Gong,Qingming Luo,Chi Xiao,Anan Li

DOI: https://doi.org/10.1016/j.patter.2024.101007

2024-06-21

Abstract:Reconstructing neuronal morphology is vital for classifying neurons and mapping brain connectivity. However, it remains a significant challenge due to its complex structure, dense distribution, and low image contrast. In particular, AI-assisted methods often yield numerous errors that require extensive manual intervention. Therefore, reconstructing hundreds of neurons is already a daunting task for general research projects. A key issue is the lack of specialized training for challenging regions due to inadequate data and training methods. This study extracted 2,800 challenging neuronal blocks and categorized them into multiple density levels. Furthermore, we enhanced images using an axial continuity-based network that improved three-dimensional voxel resolution while reducing the difficulty of neuron recognition. Comparing the pre- and post-enhancement results in automatic algorithms using fluorescence micro-optical sectioning tomography (fMOST) data, we observed a significant increase in the recall rate. Our study not only enhances the throughput of reconstruction but also provides a fundamental dataset for tangled neuron reconstruction.
Advanced Deep Networks for 3D Mitochondria Instance Segmentation

Mingxing Li,Chang Chen,Xiaoyu Liu,Wei Huang,Yueyi Zhang,Zhiwei Xiong

DOI: https://doi.org/10.48550/arXiv.2104.07961

2022-05-08

Abstract:Mitochondria instance segmentation from electron microscopy (EM) images has seen notable progress since the introduction of deep learning methods. In this paper, we propose two advanced deep networks, named Res-UNet-R and Res-UNet-H, for 3D mitochondria instance segmentation from Rat and Human samples. Specifically, we design a simple yet effective anisotropic convolution block and deploy a multi-scale training strategy, which together boost the segmentation performance. Moreover, we enhance the generalizability of the trained models on the test set by adding a denoising operation as pre-processing. In the Large-scale 3D Mitochondria Instance Segmentation Challenge at ISBI 2021, our method ranks the 1st place. Code is available at <a class="link-external link-https" href="https://github.com/Limingxing00/MitoEM2021-Challenge" rel="external noopener nofollow">this https URL</a>.

Computer Vision and Pattern Recognition
SmartEM: machine-learning guided electron microscopy

Yaron Meirovitch,Core Francisco Park,Lu Mi,Pavel Potocek,Shashata Sawmya,Yicong Li,Ishaan Singh Chandok,Thomas L. Athey,Neha Karlupia,Yuelong Wu,Daniel R. Berger,Richard Schalek,Hanspeter Pfister,Remco Schoenmakers,Maurice Peemen,Jeff W Lichtman,Aravinthan Samuel,Nir Shavit

DOI: https://doi.org/10.1101/2023.10.05.561103

2024-06-15

Abstract:Connectomics provides essential nanometer-resolution, synapse-level maps of neural circuits to understand brain activity and behavior. However, few researchers have access to the high-throughput electron microscopes necessary to generate enough data for whole circuit or brain reconstruction. To date, machine-learning methods have been used after the collection of images by electron microscopy (EM) to accelerate and improve neuronal segmentation, synapse reconstruction and other data analysis. With the computational improvements in processing EM images, acquiring EM images has now become the rate-limiting step. Here, in order to speed up EM imaging, we integrate machine-learning into real-time image acquisition in a singlebeam scanning electron microscope. This SmartEM approach allows an electron microscope to perform intelligent, data-aware imaging of specimens. SmartEM allocates the proper imaging time for each region of interest -- scanning all pixels equally rapidly, then re-scanning small subareas more slowly where a higher quality signal is required to achieve accurate segmentability, in significantly less time. We demonstrate that this pipeline achieves a 7-fold acceleration of image acquisition time for connectomics using a commercial single-beam SEM. We apply SmartEM to reconstruct a portion of mouse cortex with the same accuracy as traditional microscopy but in less time.

Neuroscience
RoboEM: automated 3D flight tracing for synaptic-resolution connectomics

Martin Schmidt,Alessandro Motta,Meike Sievers,Moritz Helmstaedter

DOI: https://doi.org/10.1038/s41592-024-02226-5

IF: 48

2024-03-22

Nature Methods

Abstract:Mapping neuronal networks from three-dimensional electron microscopy (3D-EM) data still poses substantial reconstruction challenges, in particular for thin axons. Currently available automated image segmentation methods require manual proofreading for many types of connectomic analysis. Here we introduce RoboEM, an artificial intelligence-based self-steering 3D 'flight' system trained to navigate along neurites using only 3D-EM data as input. Applied to 3D-EM data from mouse and human cortex, RoboEM substantially improves automated state-of-the-art segmentations and can replace manual proofreading for more complex connectomic analysis problems, yielding computational annotation cost for cortical connectomes about 400-fold lower than the cost of manual error correction.

biochemical research methods
Automatic Reconstruction of Mitochondria and Endoplasmic Reticulum in Electron Microscopy Volumes by Deep Learning

Jing Liu,Linlin Li,Yang Yang,Bei Hong,Xi Chen,Qiwei Xie,Hua Han

DOI: https://doi.org/10.3389/fnins.2020.00599

IF: 4.3

2020-07-21

Frontiers in Neuroscience

Abstract:Together, mitochondria and the endoplasmic reticulum (ER) occupy more than 20% of a cell's volume, and morphological abnormality may lead to cellular function disorders. With the rapid development of large-scale electron microscopy (EM), manual contouring and three-dimensional (3D) reconstruction of these organelles has previously been accomplished in biological studies. However, manual segmentation of mitochondria and ER from EM images is time consuming and thus unable to meet the demands of large data analysis. Here, we propose an automated pipeline for mitochondrial and ER reconstruction, including the mitochondrial and ER contact sites (MAMs). We propose a novel recurrent neural network to detect and segment mitochondria and a fully residual convolutional network to reconstruct the ER. Based on the sparse distribution of synapses, we use mitochondrial context information to rectify the local misleading results and obtain 3D mitochondrial reconstructions. The experimental results demonstrate that the proposed method achieves state-of-the-art performance.

neurosciences
Rapid and Extreme Low-light Superresolution Imaging Via Artificial Intelligence

Bei Liu,Luhong Jin,Bowei Dong,Ruiyan Song,Fenqiang Zhao,Stephen Hahn,Timothy C. Elston,Yingke Xu,Klaus M. Hahn

DOI: https://doi.org/10.1016/j.bpj.2019.11.1028

IF: 3.4

2020-01-01

Biophysical Journal

Abstract:Supreresolution (SR) fluorescence microscopy breaks the optical diffraction limit and achieves unprecedented nanoscale resolution, thus producing a huge impact on biological research. Structured illumination microscopy (SIM) is one of the most broadly used SR techniques. SIM applies varying, nonuniform illumination to samples. A dedicated computational algorithm is then used to retrieve SR information from nine or fifteen sequentially acquired images, for 2D or 2D/3D reconstruction respectively. Super-resolution radial fluctuation (SRRF) is a purely computational SR approach, that can retrieve high frequency information by spatio-temporally analyzing the stochastically fluctuating signals from image time series (typically >200 frames). We utilized deep neural networks to directly reconstruct super-resolved images out of a series of images taken with either structured or conventional wide-field illumination microscopy. This substantially reduced the number of images compared to state-of-the-art methods (as few as 3 images for SIM and 5 images for SRRF). Further, leveraging skip-layer connected U-Nets, we could restore high quality, high-resolution images from raw data with over 100 times less photons. We validated our results for different cellular structures, including microtubules, mitochondria, adhesions and actin filaments. We demonstrated its applications in ultra-fast, multi-color superresolution imaging of living cells. A ready-to-use open source package is available.
CMRxRecon: A publicly available k -space dataset and benchmark to advance deep learning for cardiac MRI

Chengyan Wang,Jun Lyu,Shuo Wang,Chen Qin,Kunyuan Guo,Xinyu Zhang,Xiaotong Yu,Yan Li,Fanwen Wang,Jianhua Jin,Zhang Shi,Ziqiang Xu,Yapeng Tian,Sha Hua,Zhensen Chen,Meng Liu,Mengting Sun,Xutong Kuang,Kang Wang,Haoran Wang,Hao Li,Yinghua Chu,Guang Yang,Wenjia Bai,Xiahai Zhuang,He Wang,Jing Qin,Xiaobo Qu

DOI: https://doi.org/10.1038/s41597-024-03525-4

2024-06-26

Scientific Data

Abstract:Cardiac magnetic resonance imaging (CMR) has emerged as a valuable diagnostic tool for cardiac diseases. However, a significant drawback of CMR is its slow imaging speed, resulting in low patient throughput and compromised clinical diagnostic quality. The limited temporal resolution also causes patient discomfort and introduces artifacts in the images, further diminishing their overall quality and diagnostic value. There has been growing interest in deep learning-based CMR imaging algorithms that can reconstruct high-quality images from highly under-sampled k-space data. However, the development of deep learning methods requires large training datasets, which have so far not been made publicly available for CMR. To address this gap, we released a dataset that includes multi-contrast, multi-view, multi-slice and multi-coil CMR imaging data from 300 subjects. Imaging studies include cardiac cine and mapping sequences. The 'CMRxRecon' dataset contains raw k-space data and auto-calibration lines. Our aim is to facilitate the advancement of state-of-the-art CMR image reconstruction by introducing standardized evaluation criteria and making the dataset freely accessible to the research community.

multidisciplinary sciences
Reconstructing Circuit Connectivity from in vivo Spike Trains Using Deep Domain-Adaptive Matching

Kaiwen Sheng,Shanghang Zhang,Maxime Beau,Peng Qu,Le Yang,Xiaofei Liu,Liuyuan He,Youhui Zhang,Lei Ma,Kai Du

DOI: https://doi.org/10.1101/2022.10.03.510694

2024-12-07

Abstract:Inferring the connectivity of neural circuits from in vivo experimental data is essential for understanding the neural architectures that underpin behavior and cognition. However, advanced machine learning approaches in this domain, particularly deep learning, are significantly constrained by incomplete in vivo observations. This limitation restricts the ability to assign 'ground-truth' labels to in vivo data, compelling researchers to rely on synthetic data generated by biophysical neural models for initial training. Yet, this strategy introduces a widely perceived insurmountable challenge: 'model mismatch', where synthetic model dynamics fail to accurately match true neural dynamics. To tackle these interrelated challenges, we introduce DeepDAM (Deep Domain-Adaptive Matching), a robust, adaptive, and versatile framework that fundamentally transforms the training of deep neural networks (DNNs) for inference tasks using both synthetic data and unlabeled in vivo data. DeepDAM fine-tunes DNNs on a fusion of synthetic and unlabeled in vivo data, adaptively aligning the DNN's feature space with in vivo neural dynamics, thereby effectively mitigating the model mismatch challenge. Impressively, we validated the framework with extracellular recordings in the hippocampal CA1 region of freely behaving mice, which achieves an exemplary Matthews correlation coefficient of 0.97-1.0 in connectivity inference accuracy, significantly outperforming existing methods (~0.6-0.7). Importantly, our framework can well adapt to diverse experimental conditions and a broad spectrum of neural properties and scales, demonstrating its high generalizability across various in vivo and ex vivo scenarios. This marks a significant step towards the accurate and comprehensive reconstruction of functional mammalian brains using data-driven methodologies.

Neuroscience
AxonCallosumEM Dataset: Axon Semantic Segmentation of Whole Corpus Callosum cross section from EM Images

Ao Cheng,Guoqiang Zhao,Lirong Wang,Ruobing Zhang

2023-07-06

Abstract:The electron microscope (EM) remains the predominant technique for elucidating intricate details of the animal nervous system at the nanometer scale. However, accurately reconstructing the complex morphology of axons and myelin sheaths poses a significant challenge. Furthermore, the absence of publicly available, large-scale EM datasets encompassing complete cross sections of the corpus callosum, with dense ground truth segmentation for axons and myelin sheaths, hinders the advancement and evaluation of holistic corpus callosum reconstructions. To surmount these obstacles, we introduce the AxonCallosumEM dataset, comprising a 1.83 times 5.76mm EM image captured from the corpus callosum of the Rett Syndrome (RTT) mouse model, which entail extensive axon bundles. We meticulously proofread over 600,000 patches at a resolution of 1024 times 1024, thus providing a comprehensive ground truth for myelinated axons and myelin sheaths. Additionally, we extensively annotated three distinct regions within the dataset for the purposes of training, testing, and validation. Utilizing this dataset, we develop a fine-tuning methodology that adapts Segment Anything Model (SAM) to EM images segmentation tasks, called EM-SAM, enabling outperforms other state-of-the-art methods. Furthermore, we present the evaluation results of EM-SAM as a baseline.

Image and Video Processing,Computer Vision and Pattern Recognition
EASE: EM-Assisted Source Extraction from calcium imaging data

Pengcheng Zhou,Jacob Reimer,Ding Zhou,Amol Pasarkar,Ian Kinsella,Emmanouil Froudarakis,Dimitri V Yatsenko,Paul G Fahey,Agnes Bodor,JoAnn Buchanan,Dan Bumbarger,Gayathri Mahalingam,Russel Torres,Sven Dorkenwald,Dodam Ih,Kisuk Lee,Ran Lu,Thomas Macrina,Jingpeng Wu,Nuno da Costa,R. Clay Reid,Andreas S Tolias,Liam Paninski

DOI: https://doi.org/10.1101/2020.03.25.007468

2020-01-01

Abstract:In this work we develop a pipeline for fusing CI and densely-reconstructed EM data. We model the observed CI data using a constrained nonnegative matrix factorization (CNMF) framework, in which segments extracted from the EM reconstruction serve to initialize and constrain the spatial components of the matrix factorization. We develop an efficient iterative procedure for solving the resulting combined matching and matrix factorization problem and apply this procedure to joint CI-EM data from mouse visual cortex. The method recovers hundreds of dendritic components from the CI data, visible across multiple functional scans at different depths, matched with densely-reconstructed three-dimensional neural segments recovered from the EM volume. We publicly release the output of this analysis as a new gold standard dataset that can be used to score algorithms for demixing signals from 2PCI data. Finally, we show that this database can be exploited to (1) learn a mapping from 3d EM segmentations to predict the corresponding 2d spatial components estimated from CI data, and (2) train a neural network to denoise these estimated spatial components. This neural network denoiser is a stand-alone module that can be dropped in to enhance any existing 2PCI analysis pipeline.
3D U-Net Improves Automatic Brain Extraction for Isotropic Rat Brain Magnetic Resonance Imaging Data

Li-Ming Hsu,Shuai Wang,Lindsay Walton,Tzu-Wen Winnie Wang,Sung-Ho Lee,Yen-Yu Ian Shih

DOI: https://doi.org/10.3389/fnins.2021.801008

IF: 4.3

2021-12-16

Frontiers in Neuroscience

Abstract:Brain extraction is a critical pre-processing step in brain magnetic resonance imaging (MRI) analytical pipelines. In rodents, this is often achieved by manually editing brain masks slice-by-slice, a time-consuming task where workloads increase with higher spatial resolution datasets. We recently demonstrated successful automatic brain extraction via a deep-learning-based framework, U-Net, using 2D convolutions. However, such an approach cannot make use of the rich 3D spatial-context information from volumetric MRI data. In this study, we advanced our previously proposed U-Net architecture by replacing all 2D operations with their 3D counterparts and created a 3D U-Net framework. We trained and validated our model using a recently released CAMRI rat brain database acquired at isotropic spatial resolution, including T2-weighted turbo-spin-echo structural MRI and T2*-weighted echo-planar-imaging functional MRI. The performance of our 3D U-Net model was compared with existing rodent brain extraction tools, including Rapid Automatic Tissue Segmentation, Pulse-Coupled Neural Network, SHape descriptor selected External Regions after Morphologically filtering, and our previously proposed 2D U-Net model. 3D U-Net demonstrated superior performance in Dice, Jaccard, center-of-mass distance, Hausdorff distance, and sensitivity. Additionally, we demonstrated the reliability of 3D U-Net under various noise levels, evaluated the optimal training sample sizes, and disseminated all source codes publicly, with a hope that this approach will benefit rodent MRI research community. Significant Methodological Contribution: We proposed a deep-learning-based framework to automatically identify the rodent brain boundaries in MRI. With a fully 3D convolutional network model, 3D U-Net, our proposed method demonstrated improved performance compared to current automatic brain extraction methods, as shown in several qualitative metrics (Dice, Jaccard, PPV, SEN, and Hausdorff). We trust that this tool will avoid human bias and streamline pre-processing steps during 3D high resolution rodent brain MRI data analysis. The software developed herein has been disseminated freely to the community.

neurosciences
Exploiting Large Neuroimaging Datasets to Create Connectome-Constrained Approaches for more Robust, Efficient, and Adaptable Artificial Intelligence

Erik C. Johnson,Brian S. Robinson,Gautam K. Vallabha,Justin Joyce,Jordan K. Matelsky,Raphael Norman-Tenazas,Isaac Western,Marisel Villafañe-Delgado,Martha Cervantes,Michael S. Robinette,Arun V. Reddy,Lindsey Kitchell,Patricia K. Rivlin,Elizabeth P. Reilly,Nathan Drenkow,Matthew J. Roos,I-Jeng Wang,Brock A. Wester,William R. Gray-Roncal,Joan A. Hoffmann

DOI: https://doi.org/10.48550/arXiv.2305.17300

2023-05-26

Neural and Evolutionary Computing

Abstract:Despite the progress in deep learning networks, efficient learning at the edge (enabling adaptable, low-complexity machine learning solutions) remains a critical need for defense and commercial applications. We envision a pipeline to utilize large neuroimaging datasets, including maps of the brain which capture neuron and synapse connectivity, to improve machine learning approaches. We have pursued different approaches within this pipeline structure. First, as a demonstration of data-driven discovery, the team has developed a technique for discovery of repeated subcircuits, or motifs. These were incorporated into a neural architecture search approach to evolve network architectures. Second, we have conducted analysis of the heading direction circuit in the fruit fly, which performs fusion of visual and angular velocity features, to explore augmenting existing computational models with new insight. Our team discovered a novel pattern of connectivity, implemented a new model, and demonstrated sensor fusion on a robotic platform. Third, the team analyzed circuitry for memory formation in the fruit fly connectome, enabling the design of a novel generative replay approach. Finally, the team has begun analysis of connectivity in mammalian cortex to explore potential improvements to transformer networks. These constraints increased network robustness on the most challenging examples in the CIFAR-10-C computer vision robustness benchmark task, while reducing learnable attention parameters by over an order of magnitude. Taken together, these results demonstrate multiple potential approaches to utilize insight from neural systems for developing robust and efficient machine learning techniques.

U-RISC: an Ultra-high Resolution EM 1 Dataset Challenging Existing Deep 2 Learning Algorithms 3

U-RISC: An Annotated Ultra-High-Resolution Electron Microscopy Dataset Challenging the Existing Deep Learning Algorithms

Fast and Lightweight Network Improves Serial Brain Section Stitching

TF-Unet:An Automatic Cardiac MRI Image Segmentation Method

DeU-Net 2.0: Enhanced deformable U-Net for 3D cardiac cine MRI segmentation

High-Speed Automated Reconstruction of Drosophila Larval Brain from Volumetric EM Data

Deep residual contextual and subpixel convolution network for automated neuronal structure segmentation in micro-connectomics

Large Scale Image Segmentation with Structured Loss Based Deep Learning for Connectome Reconstruction

AxonEM Dataset: 3D Axon Instance Segmentation of Brain Cortical Regions

A hierarchically annotated dataset drives tangled filament recognition in digital neuron reconstruction

Advanced Deep Networks for 3D Mitochondria Instance Segmentation

SmartEM: machine-learning guided electron microscopy

RoboEM: automated 3D flight tracing for synaptic-resolution connectomics

Automatic Reconstruction of Mitochondria and Endoplasmic Reticulum in Electron Microscopy Volumes by Deep Learning

Rapid and Extreme Low-light Superresolution Imaging Via Artificial Intelligence

CMRxRecon: A publicly available k -space dataset and benchmark to advance deep learning for cardiac MRI

Reconstructing Circuit Connectivity from in vivo Spike Trains Using Deep Domain-Adaptive Matching

AxonCallosumEM Dataset: Axon Semantic Segmentation of Whole Corpus Callosum cross section from EM Images

EASE: EM-Assisted Source Extraction from calcium imaging data

3D U-Net Improves Automatic Brain Extraction for Isotropic Rat Brain Magnetic Resonance Imaging Data

Exploiting Large Neuroimaging Datasets to Create Connectome-Constrained Approaches for more Robust, Efficient, and Adaptable Artificial Intelligence