Abstract:Photonics inverse design relies on human experts to search for a design topology that satisfies certain optical specifications with their experience and intuitions, which is relatively labor-intensive, slow, and sub-optimal. Machine learning has emerged as a powerful tool to automate this inverse design process. However, supervised or semi-supervised deep learning is unsuitable for this task due to: (1) a severe shortage of available training data due to the high computational complexity of physics-based simulations along with a lack of open-source datasets and/or the need for a pre-trained neural network model; (2) the issue of one-to-many mapping or non-unique solutions; and (3) the inability to perform optimization of the photonic structure beyond inverse designing. Reinforcement Learning (RL) has the potential to overcome the above three challenges. Here, we propose Learning to Design Optical-Resonators (L2DO) to leverage RL that learns to autonomously inverse design nanophotonic laser cavities without any prior knowledge while retrieving unique design solutions. L2DO incorporates two different algorithms – Deep Q-learning and Proximal Policy Optimization. We evaluate L2DO on two laser cavities: a long photonic crystal (PC) nanobeam and a PC nanobeam with an L3 cavity, both popular structures for semiconductor lasers. Trained for less than 152 hours on limited hardware resources, L2DO has improved state-of-the-art results in the literature by over 2 orders of magnitude and obtained 10 times better performance than a human expert working the same task for over a month. L2DO first learned to meet the required maxima of Q -factors (>50 million) and then proceeded to optimize some additional good-to-have features (e.g., resonance frequency, modal volume). Compared with iterative human designs and inverse design via supervised learning, L2DO can achieve over two orders of magnitude higher sample-efficiency without suffering from the three issues above. This work confirms the potential of deep RL algorithms to surpass human designs and marks a solid step towards a fully automated AI framework for photonics inverse design.

Enhanced DBR mirror design via D3QN: A reinforcement learning approach

Millimeter Wave Communications With an Intelligent Reflector: Performance Optimization and Distributional Reinforcement Learning

Micro-LED backlight module by deep reinforcement learning and micro-macro-hybrid environment control agent

Joint Deep Reinforcement Learning and Unfolding: Beam Selection and Precoding for Mmwave Multiuser MIMO with Lens Arrays

Deep Learning Based Joint Beam Selection and Precoding Design for Mmwave Systems with Lens Arrays

A DRL-based Reflection Enhancement Method for RIS-assisted Multi-receiver Communications

Secure Transmission for Intelligent Reflecting Surface Assisted Communication with Deep Learning

Refined-Deep Reinforcement Learning for MIMO Bistatic Backscatter Resource Allocation

Deep Reinforcement Learning Based Intelligent Reflecting Surface Optimization for TDD MultiUser MIMO Systems

Deep Reinforcement Learning Based Intelligent Reflecting Surface Optimization for TDD Multi-User MIMO Systems

Exploring High-Performance Photonic Crystal Slow Light Waveguides Through Deep Reinforcement Learning

Deep Reinforcement Learning Based Dynamic Beam Selection in Dual-Band Communication Systems

Autonomous design of noise-mitigating structures using deep reinforcement learning

Deep reinforcement learning empowers automated inverse design and optimization of photonic crystals for nanoscale laser cavities

Reinforcement learning enabled the design of compact and efficient integrated photonic devices

Deep Reinforcement Learning-aided Transmission Design for Energy-efficient Link Optimization in Vehicular Communications

Double Deep Q-Network Based Distributed Resource Matching Algorithm for D2D Communication

Automated multi-layer optical design via deep reinforcement learning

Deep Reinforcement Learning for Beam Angle Optimization of Intensity-Modulated Radiation Therapy

An Enhanced Dueling Double Deep Q-Network With Convolutional Block Attention Module for Traffic Signal Optimization in Deep Reinforcement Learning

Joint Trajectory and Passive Beamforming Design for Intelligent Reflecting Surface-Aided UAV Communications: A Deep Reinforcement Learning Approach