Abstract:The ultimate goal of artificial intelligence is to mimic the human brain to perform decision-making and control directly from high-dimensional sensory input. Diffractive optical networks provide a promising solution for implementing artificial intelligence with high-speed and low-power consumption. Most of the reported diffractive optical networks focus on single or multiple tasks that do not involve environmental interaction, such as object recognition and image classification. In contrast, the networks capable of performing decision-making and control have not yet been developed to our knowledge. Here, we propose using deep reinforcement learning to implement diffractive optical networks that imitate human-level decision-making and control capability. Such networks taking advantage of a residual architecture, allow for finding optimal control policies through interaction with the environment and can be readily implemented with existing optical devices. The superior performance of these networks is verified by engaging three types of classic games, Tic-Tac-Toe, Super Mario Bros., and Car Racing. Finally, we present an experimental demonstration of playing Tic-Tac-Toe by leveraging diffractive optical networks based on a spatial light modulator. Our work represents a solid step forward in advancing diffractive optical networks, which promises a fundamental shift from the target-driven control of a pre-designed state for simple recognition or classification tasks to the high-level sensory capability of artificial intelligence. It may find exciting applications in autonomous driving, intelligent robots, and intelligent manufacturing.

What problem does this paper attempt to address?

The problem this paper attempts to address is achieving human-level decision-making and control capabilities using Diffractive Optical Networks (DON). Specifically, most existing diffractive optical networks focus on single-task or multi-task applications such as object recognition and image classification, without involving interaction with the environment. Therefore, the authors propose a method based on deep reinforcement learning, enabling diffractive optical networks to learn optimal control strategies through interaction with the environment. The effectiveness of this method is validated in three classic games: Tic-Tac-Toe, Super Mario Bros., and Car Racing. Additionally, the authors conducted experimental demonstrations showcasing the practical application of diffractive optical networks in Tic-Tac-Toe. ### Main Issues: 1. **Limitations of existing diffractive optical networks**: Existing diffractive optical networks mainly focus on tasks like image classification and object recognition, lacking the ability to interact with the environment. 2. **Achieving human-level decision-making and control**: How to utilize diffractive optical networks to achieve human-level decision-making and control capabilities, especially in complex environments. 3. **Application of deep reinforcement learning**: How to combine deep reinforcement learning with diffractive optical networks to learn optimal control strategies through interaction with the environment. ### Solution: - **Network architecture**: A diffractive optical network architecture is proposed, consisting of an input layer, multiple hidden layers, and an output layer, where the hidden layers are composed of multiple diffractive blocks. - **Training method**: The network is trained using deep reinforcement learning algorithms, learning optimal control strategies through interaction with a simulated environment. - **Experimental validation**: The effectiveness of the network is validated in three classic games, and experimental demonstrations are conducted to showcase the practical application of diffractive optical networks in Tic-Tac-Toe. ### Experimental Results: - **Tic-Tac-Toe**: The network successfully completes the game with a high accuracy rate, achieving 100% accuracy for the X player and 90.56% accuracy for the O player. - **Super Mario Bros.**: The network successfully controls Mario to complete levels by selecting optimal actions to overcome obstacles in the game. - **Car Racing**: The network can control the car in real-time on the track, maintaining stable performance even when random disturbances are introduced. ### Potential Applications: - **Autonomous driving**: Utilizing the high-speed and low-power characteristics of diffractive optical networks to achieve efficient autonomous driving systems. - **Intelligent robots**: Enhancing the perception and decision-making capabilities of robots, enabling them to act autonomously in complex environments. - **Smart manufacturing**: Achieving intelligent control in industrial manufacturing, improving production efficiency and quality. In summary, this paper combines deep reinforcement learning with diffractive optical networks to achieve efficient decision-making and control capabilities in complex environments, providing a new technological pathway for future intelligent systems.

Decision-making and control with diffractive optical networks

Optical Analog Computing Devices Designed by Deep Neural Network

Design of task-specific optical systems using broadband diffractive neural networks

Solving Computer Vision Tasks with Diffractive Neural Networks

Advances and Progress of Diffractive Deep Neural Networks

An Optical Satellite Controller Based on Diffractive Deep Neural Network.

Large-scale neuromorphic optoelectronic computing with a reconfigurable diffractive processing unit

Scale-, shift- and rotation-invariant diffractive optical networks

Physically transparent diffractive optical networks

Deep Learning-designed Diffractive Neural Networks

<i>In situ</i> optical backpropagation training of diffractive optical neural networks

A programmable diffractive deep neural network based on a digital-coding metasurface array

In Situ Optical Backpropagation Training of Diffractive Optical Neural Networks

Pluggable Multitask Diffractive Neural Networks Based on Cascaded Metasurfaces

Space-efficient optical computing with an integrated chip diffractive neural network

Ensemble learning of diffractive optical networks

Optical Control of Adaptive Nanoscale Domain Networks

Non-Volatile Reconfigurable Optical Digital Diffractive Neural Network Based on Phase Change Material

Scientific Computing with Diffractive Optical Neural Networks

High-frame-rate Reconfigurable Diffractive Neural Network Based on Superpixels.

Multimodal deep learning using on-chip diffractive optics with in situ training capability