Multi-Task Optoelectronic Hybrid Neural Network Based on Nonlinear Metasurface(Invited)
Luo Xuhao,Dong Siyu,Wei Zeyong,Wang Zhanshan,Cheng Xinbin
DOI: https://doi.org/10.3788/aos240437
2024-01-01
Acta Optica Sinica
Abstract:Objective Optical and electronic computing systems attempt to understand large amounts of visual data originating from scenarios such as autonomous driving, machine vision, medical diagnostics, remote sensing, defense, and the Internet of Everything. These data are interpreted by artificial intelligence (AI) algorithms, where electronic deep neural networks swiftly emerge as the standard algorithm for visual data processing. Optical neural networks that utilize photons as computational carriers feature high speed, low power, high throughput, and massive parallelism. With the deep integration of optics, optoelectronics, and electronics, the optoelectronic hybrid network model combines the bandwidth of optical computation with the flexibility of electronic computation, providing an order of magnitude increase in the density, speed, and energy of computational systems. However, existing architectures are usually designed for a single task and lack the ability to process multiple tasks in parallel. Therefore, we propose a novel optoelectronic hybrid neural network computational model for multidimensional modulation of linear and nonlinear light waves via nonlinear metasurfaces to simultaneously process information from multiple channels. Methods The nonlinear metasurface structure employs a one- fold symmetric U-shaped resonant unit to realize the nonlinear multiplexing function of the network and high second harmonic efficiency. Based on the geometrical phase, continuous phase modulation is realized by rotating its spatial angle. The transmission spectrum of the structure at circular polarization is simulated using FDTD. Its resonance wavelength is identified and selected as 1160 nm. Meanwhile, an optoelectronic hybrid network architecture for nonlinear metasurfaces is constructed to realize item classification and reconstruction of coding by phase multiplexing of fundamental and two-fold frequencies. The front-end optical network and the two back- end electronic networks are trained as a package, and the co-objective loss function for recognition and reconstruction is optimized using the gradient descent algorithm and the error back-propagation algorithm. After the networks are trained, unique metasurface structures are identified by extracting the multiplexed phases. Results and Discussions The results after network training are shown in Fig. 1, and the classification part achieves a blind test accuracy of 95.53% on the MNIST test dataset, with the normalized loss function of the reconstruction of the coding converging to 0.031. The design results and simulated transmission spectra of the nonlinear metasurface are shown in Fig. 2, where its fundamental mid-wave resonates most intensely near 1160 nm and a second- order resonance exists near 704 nm. The simulation results of the network classification performance at fundamental frequency are shown in Fig. 3. The distribution of the fundamental frequency output behind the nonlinear metasurface is irregular, but the back- end electronic network can efficiently learn these differences and classify them. Figure 4 reveals the reconstruction results of the network, where the more random optical output is more favorable for encrypting the information. The PSNR of the reconstructed digit 1 reaches a maximum of 30.12 dB, and the average PSNR of the whole test set is 26.13 dB, which indicates that the model yields high reconstruction performance. Conclusions We demonstrate an optoelectronic hybrid network model based on a nonlinear metasurface that achieves the dual tasks of performing handwritten digit classification and reconstruction in both fundamental and twofold channels. High accuracy ( 95.53%) and high image quality ( average PSNR > 26 dB) are obtained by joint training optimization. The proposed method combines the advantages of optical and electronic computations and simultaneously achieves high parallelism, high accuracy, and low power consumption. Since the nonlinear high-frequency channels can be increased, the machine vision concept can also be extended to perform other learning tasks in parallel, such as edge extraction, image segmentation, and super-resolution imaging. In future work, we will experimentally prepare the device using processes such as electron beam lithography and deposition. The proposed methodology is important for massively parallel optoelectronic computing and will find new applications in machine vision, smart driving, bio- imaging, and smart medicine.