SmartEM: machine-learning guided electron microscopy

Yaron Meirovitch,Core Francisco Park,Lu Mi,Pavel Potocek,Shashata Sawmya,Yicong Li,Ishaan Singh Chandok,Thomas L. Athey,Neha Karlupia,Yuelong Wu,Daniel R. Berger,Richard Schalek,Hanspeter Pfister,Remco Schoenmakers,Maurice Peemen,Jeff W Lichtman,Aravinthan Samuel,Nir Shavit

DOI: https://doi.org/10.1101/2023.10.05.561103

2024-06-15

Abstract:Connectomics provides essential nanometer-resolution, synapse-level maps of neural circuits to understand brain activity and behavior. However, few researchers have access to the high-throughput electron microscopes necessary to generate enough data for whole circuit or brain reconstruction. To date, machine-learning methods have been used after the collection of images by electron microscopy (EM) to accelerate and improve neuronal segmentation, synapse reconstruction and other data analysis. With the computational improvements in processing EM images, acquiring EM images has now become the rate-limiting step. Here, in order to speed up EM imaging, we integrate machine-learning into real-time image acquisition in a singlebeam scanning electron microscope. This SmartEM approach allows an electron microscope to perform intelligent, data-aware imaging of specimens. SmartEM allocates the proper imaging time for each region of interest -- scanning all pixels equally rapidly, then re-scanning small subareas more slowly where a higher quality signal is required to achieve accurate segmentability, in significantly less time. We demonstrate that this pipeline achieves a 7-fold acceleration of image acquisition time for connectomics using a commercial single-beam SEM. We apply SmartEM to reconstruct a portion of mouse cortex with the same accuracy as traditional microscopy but in less time.

Neuroscience

What problem does this paper attempt to address?

This paper mainly discusses how to use machine learning to accelerate the application of electron microscopy (EM) in neuroscience, especially in the field of connectomics (the study of neural network structures) in the image acquisition process. Currently, although machine learning methods have been used to improve the speed of image analysis, image acquisition is still a bottleneck. The paper proposes a method called SmartEM, which integrates machine learning into real-time image acquisition, enabling single-beam scanning electron microscopy to image samples intelligently and data-driven.

SmartEM rapidly scans all pixels by identifying the appropriate imaging time needed for each region, and then performs a slow rescan of small regions that require higher signal quality, ensuring accuracy while significantly reducing time. This method is targeted at regions that require long imaging time for accurate segmentation, such as neural fibers and synapses. The paper demonstrates that SmartEM can speed up the image acquisition time of commercial single-beam SEM by 7 times while maintaining the same reconstruction accuracy as traditional microscopy.

The experimental results show that for connectomics, the accuracy of image is crucial for neuron segmentation and synaptic identification. SmartEM can significantly improve imaging speed while maintaining high accuracy by automatically detecting error regions and performing targeted rescans. By running machine learning algorithms on SEM computer hardware, SmartEM can collect connectomics datasets on widely available point scanning SEMs without expensive high-throughput devices.

In summary, the problem addressed by this paper is how to use machine learning techniques to optimize the imaging process of electron microscopy, accelerate data acquisition in connectomics, and keep up with existing fast data analysis capabilities, thereby promoting large-scale comparative connectomics research.