Local Conditional Neural Fields for Versatile and Generalizable Large-Scale Reconstructions in Computational Imaging

Hao Wang,Jiabei Zhu,Yunzhe Li,QianWan Yang,Lei Tian
2023-07-22
Abstract:Deep learning has transformed computational imaging, but traditional pixel-based representations limit their ability to capture continuous, multiscale details of objects. Here we introduce a novel Local Conditional Neural Fields (LCNF) framework, leveraging a continuous implicit neural representation to address this limitation. LCNF enables flexible object representation and facilitates the reconstruction of multiscale information. We demonstrate the capabilities of LCNF in solving the highly ill-posed inverse problem in Fourier ptychographic microscopy (FPM) with multiplexed measurements, achieving robust, scalable, and generalizable large-scale phase retrieval. Unlike traditional neural fields frameworks, LCNF incorporates a local conditional representation that promotes model generalization, learning multiscale information, and efficient processing of large-scale imaging data. By combining an encoder and a decoder conditioned on a learned latent vector, LCNF achieves versatile continuous-domain super-resolution image reconstruction. We demonstrate accurate reconstruction of wide field-of-view, high-resolution phase images using only a few multiplexed measurements. LCNF robustly captures the continuous object priors and eliminates various phase artifacts, even when it is trained on imperfect datasets. The framework exhibits strong generalization, reconstructing diverse objects even with limited training data. Furthermore, LCNF can be trained on a physics simulator using natural images and successfully applied to experimental measurements on biological samples. Our results highlight the potential of LCNF for solving large-scale inverse problems in computational imaging, with broad applicability in various deep-learning-based techniques.
Image and Video Processing,Machine Learning,Computational Physics,Optics
What problem does this paper attempt to address?
The problem this paper attempts to address is the large-scale inverse problem in computational imaging, particularly in Fourier Ptychographic Microscopy (FPM), focusing on how to efficiently and robustly reconstruct high-resolution phase images from low-resolution multi-channel measurement data. Traditional methods rely on pixel representation, which limits the ability to capture continuous, multi-scale details and poses challenges when dealing with large-scale imaging data. Specifically, the paper proposes a new framework—Local Conditional Neural Fields (LCNF), which utilizes continuous implicit neural representation to overcome these limitations. By combining an encoder and a decoder, LCNF can flexibly represent objects and facilitate the learning of multi-scale information and efficient processing of large-scale imaging data. This framework demonstrates strong robustness, scalability, and generalization ability in FPM, accurately reconstructing wide-field, high-resolution phase images using only a small amount of multi-channel measurement data. The main contributions of the paper include: 1. **Continuous Implicit Neural Representation**: LCNF uses continuous implicit neural representation to capture fine-grained details and multi-scale features of objects. 2. **Local Conditional Representation**: Through local conditional representation, LCNF enhances the model's generalization ability, adapting to the reconstruction of different objects. 3. **Efficient Processing of Large-Scale Data**: LCNF can efficiently handle large-scale imaging data, making it suitable for various deep learning techniques. 4. **Robustness**: LCNF can eliminate various phase artifacts present in traditional methods, maintaining robustness even when the training data is imperfect. 5. **Generalization Ability**: LCNF maintains high performance with limited training data or under different experimental conditions, demonstrating strong generalization ability. In summary, by introducing the LCNF framework, this paper addresses the challenges of large-scale inverse problems in computational imaging, providing a new solution for the reconstruction of high-resolution phase images.