Advances in Speckle and Compressive Computational Imaging
Wang Xia,Ma Xu,Ke Jun,He Si,Hao Xiaowen,Lei Jingwen,Ma Kai
DOI: https://doi.org/10.3788/AOS230735
2023-01-01
Acta Optica Sinica
Abstract:Significance This study reports several typical advances in three categories of computational imaging techniques based on multidimensional optical field manipulation: speckle imaging, spatial and temporal compressive imaging, and compressive computational spectral imaging. Additionally, existing problems and future research prospects are analyzed and discussed herein. High-quality imaging through scattering media has crucial applications in biomedicine, astronomy, remote sensing, traffic safety, etc. Object photons traveling through a scattering medium can be classified as ballistic, snake, or diffusive photons based on the degree of deviation from their initial propagation directions. Ballistic photons can maintain their initial directions and retain undistorted object information. Using gated ballistic photons, optical coherence tomography, multiphoton microscopy, and confocal microscopy have been employed to successfully image objects hidden behind scattering media. However, in the presence of a strong scattering medium, all incident photons become diffusive after multiple scatterings and form a speckle pattern. Hence, the abovementioned techniques based on gated ballistic photons fail to image hidden objects. Therefore, the speckle imaging technology was developed to overcome this limitation. This technology involves three main steps: first, establishing a physical model of speckle formation; second, measuring and statistically analyzing the speckle light field; and finally, computationally reconstructing the hidden objects. An imaging system with high spatial and temporal resolution can obtain rich spatial and motion details of high- speed moving scenes. Improvement in spatial and temporal resolutions depends on hardware-performance improvement, including attaining high resolution and low noise in a detector array and satisfactory optical design. However, owing to the limitations in the development of semiconductors and manufacturing technologies, manufacturing a high- performance detector is difficult and costly. Additionally, the huge volume of data collected using an imaging system mandates strict requirements for read-out circuits and back- end data processing platforms. Moreover, miniaturization of the system becomes a general concern that conflicts with these high-performance requirements. Hence, further improvement in the performance of imaging systems cannot be realized based solely on hardware improvement. Compressive imaging is an imaging technology based on the compressed sensing principle and development in computer science, which realizes signal coding and compression simultaneously. Combined with back-end reconstruction algorithms, compressive imaging greatly improves the performance of an imaging system and is widely used in various imaging applications. Spectral imaging technology combines imaging and spectral technologies; thus, this technology can obtain the spatial and spectral information of an object simultaneously. Compared with traditional imaging technologies, the spectral imaging technology possesses a remarkable advantage of sensing information from a multidimensional optical field. By analyzing spectral images, highly detailed target information can be obtained, which is helpful for target recognition as well as substance detection and classification. With the development of compressed sensing theory, a new type of computational imaging technology termed as coded aperture snapshot spectral imaging (CASSI) was proposed. Subsequently, CASSI has become an advanced research topic in the field of imaging. CASSI integrates optical modulation, multiplexing detection, and numerical reconstruction algorithm to address the issues of imaging complex systems, low efficiency of data acquisition, and limited resolution in traditional snapshot spectral imaging technologies. In future, CASSI can play an important role in agriculture, military, biomedicine, and other fields, realizing fast and accurate spectral imaging approaches using intelligent perception capability. Progress The speckle correlation imaging method proposed by Bertolotti et al. introduced the concept of speckle imaging. They analyzed the autocorrelation of speckle images captured under different laser illumination angles and subsequently achieved noninvasive reconstruction of objects with phase retrieval. Katz et al. simplified the speckle imaging system using incoherent light illumination and then achieved reconstruction using a single speckle image. Since then, substantial progress has been observed in speckle imaging technology, pertaining to improving accuracy and scene applicability, expanding the imaging field of view and depth of field, and enhancing the ability of the technology to decode objects' optical field parameters, thus becoming a highly researched topic in computational imaging. This study introduces our primary research results regarding key technologies related to speckle imaging, including recursion-driven bispectral imaging with respect to dynamic scattering scenes, learning to image and track moving objects through scattering media via speckle difference, and imaging through scattering media under ambient-light interference. Developing high resolution detectors in the infrared band is considerably difficult compared with developing detectors in the visible band. Therefore, herein, we focused on studying compressive imaging in infrared band. The optical hardware systems and reconstruction algorithms related to spatial and temporal infrared compressive imaging are introduced and our related research is introduced in this study. We set up a mediumwave infraredblock compressive imaging system (Fig. 9) and discussed obtained results herein, including reducing block effect, removing stray light, limiting nonuniform ( Fig. 10), improving real- time performance (Fig. 11). For the back-end processing of measured data, we reviewed the traditional methods and proposed several reconstruction algorithms based on deep learning in this study. With respect to spatial compressive imaging, we designed Meta-TR, which combined meta-attention and transformer (Fig. 12); furthermore, we designed a multiframe reconstruction network named Joinput-CiNet ( Fig. 13). Moreover, we introduced a novel version of a 3D-TCI network to achieve temporal reconstruction (Fig. 14). Moreover, the spatial temporal compressive imaging method, which combines temporal and spatial compression, is briefly discussed herein (Fig. 16). Furthermore, we reviewed relevant studies in the field of compressive computational spectral imaging that covered the development of color-coded aperture and use of the latest transformer network to improve the image-reconstruction quality. Additionally, we summarized our research achievements. First, we proposed an optical- axis-shift CASSI system based on a digital micromirror device, which can effectively suppress off-axis aberration (Fig. 17). Second, we proposed a 3D coded convolutional neural network capable of realizing hyperspectral image classification (Fig. 19) based on the established dual-disperser CASSI system (Fig. 18). Subsequently, we proposed a hexagonal, blue-noise, complementarycoded aperture ( Fig. 20) and spatial-target adaptive-coded aperture ( Fig. 21) for improving the perceptual efficiency of CASSI systems. Finally, to enhance the quality of reconstructed spectral images, we proposed a fast alternating minimization algorithm based on the sparsity and deep image priors ( Fama-SDIP) (Fig. 22). Conclusions and Prospects We achieved remarkable results in three categories of computational imaging techniques based on multidimensional optical field manipulation: speckle imaging, spatial and temporal compressive imaging, and compressive computational spectral imaging. However, these techniques still face numerous challenges in terms of practical applications, including realizing a compact system design, mounting and error calibration, coded aperture preparation, fast and accurate reconstruction of optical fields, and lightweight design of networks. In future, researchers can combine the field of micro-/nano-optics with computational imaging mechanisms to further improve the manipulation ability of imaging systems. Moreover, artificial intelligence can be used to improve the scope of practical application of imaging systems.