Neural Étendue Expander for Ultra-Wide-Angle High-Fidelity Holographic Display

Ethan Tseng,Grace Kuo,Seung-Hwan Baek,Nathan Matsuda,Andrew Maimone,Florian Schiffers,Praneeth Chakravarthula,Qiang Fu,Wolfgang Heidrich,Douglas Lanman,Felix Heide

DOI: https://doi.org/10.1038/s41467-024-46915-3

2024-04-27

Abstract:Holographic displays can generate light fields by dynamically modulating the wavefront of a coherent beam of light using a spatial light modulator, promising rich virtual and augmented reality applications. However, the limited spatial resolution of existing dynamic spatial light modulators imposes a tight bound on the diffraction angle. As a result, modern holographic displays possess low étendue, which is the product of the display area and the maximum solid angle of diffracted light. The low étendue forces a sacrifice of either the field-of-view (FOV) or the display size. In this work, we lift this limitation by presenting neural étendue expanders. This new breed of optical elements, which is learned from a natural image dataset, enables higher diffraction angles for ultra-wide FOV while maintaining both a compact form factor and the fidelity of displayed contents to human viewers. With neural étendue expanders, we experimentally achieve 64$\times$ étendue expansion of natural images in full color, expanding the FOV by an order of magnitude horizontally and vertically, with high-fidelity reconstruction quality (measured in PSNR) over 29 dB on retinal-resolution images.

Image and Video Processing,Computer Vision and Pattern Recognition,Optics

What problem does this paper attempt to address?

The problem that this paper attempts to solve is the trade - off between the field - of - view (FOV) and the display size in modern holographic display technology. Specifically, due to the limited spatial resolution of existing dynamic spatial light modulators (SLMs), this results in severely restricted diffraction angles. Consequently, modern holographic displays have a low étendue (i.e., the product of the display area and the solid angle of the maximum diffracted light). Low étendue forces a sacrifice between the FOV or the display size. The paper proposes a new solution - the Neural Étendue Expander. Through this optical element, an ultra - wide FOV can be achieved while maintaining a compact form factor, and without sacrificing the fidelity of the display content to human observers.

The core contributions of the paper are:
1. **Proposing the Neural Étendue Expander**: This is a new type of optical element learned from natural image datasets, which can achieve higher diffraction angles, thereby significantly expanding the FOV of holographic displays.
2. **Experimental verification**: Through experiments, the paper shows that using the Neural Étendue Expander can expand the étendue of natural images by 64 times, increase the FOV in the horizontal and vertical directions by more than 10 times, while maintaining a high reconstruction quality of more than 29 dB (measured by the peak signal - to - noise ratio PSNR).
3. **Multi - wavelength support**: The Neural Étendue Expander supports multi - wavelength illumination, is suitable for color holographic displays, and supports 3D color holographic display and viewer pupil perception.

In summary, the paper aims to break through the limitations of existing holographic display technologies by introducing the Neural Étendue Expander, achieve a wider field - of - view and higher display quality, and provide a more immersive experience for virtual reality (VR) and augmented reality (AR) devices.