Abstract:Emerging spatial computing systems seamlessly superimpose digital information on the physical environment observed by a user, enabling transformative experiences across various domains, such as entertainment, education, communication and training 1,2,3 . However, the widespread adoption of augmented-reality (AR) displays has been limited due to the bulky projection optics of their light engines and their inability to accurately portray three-dimensional (3D) depth cues for virtual content, among other factors 4,5 . Here we introduce a holographic AR system that overcomes these challenges using a unique combination of inverse-designed full-colour metasurface gratings, a compact dispersion-compensating waveguide geometry and artificial-intelligence-driven holography algorithms. These elements are co-designed to eliminate the need for bulky collimation optics between the spatial light modulator and the waveguide and to present vibrant, full-colour, 3D AR content in a compact device form factor. To deliver unprecedented visual quality with our prototype, we develop an innovative image formation model that combines a physically accurate waveguide model with learned components that are automatically calibrated using camera feedback. Our unique co-design of a nanophotonic metasurface waveguide and artificial-intelligence-driven holographic algorithms represents a significant advancement in creating visually compelling 3D AR experiences in a compact wearable device.

What problem does this paper attempt to address?

The problem that this paper attempts to solve is the limitations in the optical design of existing augmented reality (AR) display systems, especially the large volume of the projection optical system and the inability to accurately present the three - dimensional depth cues of virtual content. These problems limit the wide application of AR display technology in fields such as entertainment, education, communication, and training. To overcome these challenges, the paper introduces a full - color 3D holographic augmented reality display system. This system combines a uniquely reverse - designed full - color metasurface grating, a compact dispersion - compensated waveguide geometry, and an AI - driven holographic algorithm to achieve the display of vivid, full - color, 3D AR content in a compact device. The design of this system aims to eliminate the need for bulky collimating optical elements between the spatial light modulator and the waveguide and to provide unprecedented visual quality, thus paving the way for creating visually engaging 3D AR experiences, especially in compact wearable devices. Specifically, the key technological breakthroughs in the paper include: - **Reverse - designed metasurface grating**: Optimized for dispersion correction, transmission efficiency, and angular uniformity to meet the demanding 3D AR applications. - **Compact dispersion - compensated waveguide geometry**: By precisely designing the waveguide thickness and the size and distance of the symmetric metasurface couplers, multi - stage internal reflections of the wavefronts of red, green, and blue wavelengths within the waveguide are achieved, and finally converge at the out - coupling device. - **AI - driven holographic algorithm**: Combining a physically accurate waveguide model and a learning component automatically calibrated through camera feedback, an innovative image - formation model has been developed for generating high - quality full - color multi - plane 3D holographic images. The integration of these technologies not only improves the performance of the AR display system but also significantly reduces its volume, making it closer to the style of everyday - worn glasses, thus expected to promote the wide application of AR technology in the consumer market.

Full-colour 3D holographic augmented-reality displays with metasurface waveguides

Wide Field-of-view Waveguide Coupling Metasurfaces Used on Augmented Reality System

Hybrid Meta-Optics Enabled Compact Augmented Reality Display with Computational Image Reinforcement

Computer-generated holographic near-eye display system based on LCoS phase only modulator

Multifunctional Metasurface: Holography and Spot Cloud Projection

Metasurface wavefront control for high-performance user-natural augmented reality waveguide glasses

Color Waveguide Transparent Screen Using Lens Array Holographic Optical Element

Color Transparent Screen Using Planar Glass Combined with Lens Array Holographic Optical Elements

Holographic near-eye display system based on double-convergence light Gerchberg-Saxton algorithm.

3D holographic near-eye display based on Fraunhofer diffraction using metalens as an eyepiece

Waveguide holography for 3D augmented reality glasses

Metasurfaces for near-eye augmented reality

Accommodation-capable holographic waveguide head-up display with extended field of view

Augmented Reality Enabled by On‐Chip Meta‐Holography Multiplexing

Large-scale Huygens metasurfaces for holographic 3D near-eye displays

Enlarged Eye-Box Accommodation-Capable Augmented Reality with Hologram Replicas

3D Multiview Holographic Display with Wide Field of View Based on Metasurface

Compact Stereo Waveguide Display Based on a Unidirectional Polarization-Multiplexed Metagrating In-Coupler

Holographic Sampling Display Based on Metagratings

Metasurface for imaging and AR/VR devices

2D/3D convertible holographic waveguide-type AR system using the liquid-crystalline microlens array with the electro-switching polarizer