Abstract:Mask-based lensless fluorescence microscopy is a compact, portable imaging technique promising for biomedical research. It forms images through a thin optical mask near the camera without bulky optics, enabling snapshot three-dimensional imaging and a scalable field of view (FOV) without increasing device thickness. Lensless microscopy relies on computational algorithms to solve the inverse problem of object reconstruction. However, there has been a lack of efficient reconstruction algorithms for large-scale data. Furthermore, the entire FOV is typically reconstructed as a whole, which demands substantial computational resources and limits the scalability of the FOV. Here, we developed DeepLeMiN, a lensless microscope with a custom designed optical mask and a multi-stage physics-informed deep learning model. This not only enables the reconstruction of localized FOVs, but also significantly reduces the computational resource demands and facilitates real-time reconstruction. Our deep learning algorithm can reconstruct object volumes over 4×6×0.6 mm , achieving lateral and axial resolution of ∼10 µm and ∼50 µm respectively. We demonstrated significant improvement in both reconstruction quality and speed compared to traditional methods, across various fluorescent samples with dense structures. Notably, we achieved high-quality reconstruction of 3D motion of hydra and the neuronal activity with cellular resolution in awake mouse cortex. DeepLeMiN holds great promise for scalable, large FOV, real-time, 3D imaging applications with compact device footprint.

What problem does this paper attempt to address?

The problems that this paper attempts to solve mainly focus on improving the reconstruction efficiency and quality of lensless microscopy when dealing with large - scale data, especially in three - dimensional imaging. Specifically, the paper proposes solutions to the following problems: 1. **Efficient Reconstruction of Large - Scale Data**: Traditional reconstruction algorithms require a large amount of computational resources when dealing with large - scale data, which limits the expansion of the Field of View (FOV). The paper proposes a multi - stage physics - aware deep - learning model (multiFOV - ADMM - Net), which significantly reduces the demand for computational resources through local Field of View reconstruction and fusion, achieving real - time reconstruction. 2. **High - Resolution 3D Imaging**: When performing 3D imaging with a lensless microscope, due to the spatial distribution characteristics of the Point Spread Function (PSF), reconstructing high - quality 3D images is challenging. The paper designs an optical mask with a double - lens unit, generating a sparse and spatially - restricted PSF, which helps to reduce background noise and improve image quality. 3. **High - Quality Reconstruction of Complex Samples**: For fluorescent samples with dense features, such as biological tissues, it is difficult for traditional reconstruction methods to obtain high - quality images. The multi - view - field adaptive deep - learning network (multiFOV - ADMM - Net) proposed in the paper can effectively handle these complex samples, providing higher reconstruction quality and speed. 4. **Real - Time 3D Imaging Applications**: The paper demonstrates the application potential of DeepLeMiN in real - time 3D imaging, especially for imaging the neural activity in the cerebral cortex of living mice. Through the optimized reconstruction algorithm, DeepLeMiN can achieve high - speed 3D imaging while maintaining high resolution, and is suitable for portable and large - Field of View application scenarios. In summary, the main objective of the paper is to develop an efficient lensless microscope system DeepLeMiN. Through innovative optical design and multi - stage physics - aware deep - learning algorithms, it can achieve large - scale, high - resolution, real - time 3D imaging, especially suitable for imaging complex samples in biomedical research.

DeepLeMiN: Deep-learning-empowered Physics-aware Lensless Miniscope

Deep Learning Enhanced Fluorescence Emission Difference Microscopy

Wavefront Reconstruction Based on Deep Transfer Learning for Microscopy.

Deep-learning-augmented Computational Miniature Mesoscope

Advancing Computational Miniature Mesoscope with Simulator-based Deep Learning Reconstruction

DeepLFM: Deep Learning-based 3D Reconstruction for Light Field Microscopy

Deep-learning Light-Field Microscopy with Improved Resolution and Reconstruction Speed

GEOMScope: Large Field-of-view 3D Lensless Microscopy with Low Computational Complexity

Wide-Field, High-Resolution Reconstruction in Computational Multi-Aperture Miniscope Using a Fourier Neural Network

Efficient high-resolution fluorescence projection imaging over an extended depth of field through optical hardware and deep learning optimizations

Single-Shot 3D Widefield Fluorescence Imaging with a Computational Miniature Mesoscope

Field-dependent deep learning enables high-throughput whole-cell 3D super-resolution imaging

Generative deep learning-enabled ultra-large field-of-view lens-free imaging

Large‐Scale Lensless Microscopy with Fast Acquisition and Region‐Wise Focusing

Deep Learning Microscopy

Deep Learning Enables Large Depth-of-Field Images for Sub-Diffraction-Limit Scanning Superlens Microscopy

Large depth-of-field ultra-compact microscope by progressive optimization and deep learning

Deep‐MSIM: Fast Image Reconstruction with Deep Learning in Multifocal Structured Illumination Microscopy

Deep Learning Light Field Microscopy for Video-Rate Volumetric Functional Imaging of Behaving Animal

Deep Learning Enables Confocal Laser-Scanning Microscopy with Enhanced Resolution

Deep‐MSIM: Fast Image Reconstruction with Deep Learning in Multifocal Structured Illumination Microscopy (Adv. Sci. 27/2023)