Abstract:Retrieving images transmitted through multi-mode fibers is of growing interest, thanks to their ability to confine and transport light efficiently in a compact system. Here, we demonstrate machine-learning-based decoding of large-scale digital images (pages), maximizing page capacity for optical storage applications. Using a millimeter-sized square cross-section waveguide, we image an 8-bit spatial light modulator, presenting data as a matrix of symbols. Normally, decoders will incur a prohibitive O(n^2) computational scaling to decode n symbols in spatially scrambled data. However, by combining a digital twin of the setup with a U-Net, we can retrieve up to 66 kB using efficient convolutional operations only. We compare trainable ray-tracing-based with eigenmode-based twins and show the former to be superior thanks to its ability to overcome the simulation-to-experiment gap by adjusting to optical imperfections. We train the pipeline end-to-end using a differentiable mutual-information estimator based on the von-Mises distribution, generally applicable to phase-coding channels.

What problem does this paper attempt to address?

This paper attempts to address the challenges encountered when transmitting large - size images through multimode optical fibers (also known as light pipes). Specifically, the main problems include: 1. **Efficiency of information transmission and decoding**: When an image is transmitted through a multimode optical fiber, due to multiple reflections and scattering inside the fiber, the image will experience severe distortion. This distortion makes it extremely difficult to accurately recover the original image from the receiving end. Traditional decoding methods usually need to construct a transfer matrix (TM), with a computational complexity of \(O(n^2)\), which is not feasible when dealing with a large number of symbols. 2. **Transmission of high - density data pages**: In order to achieve efficient optical storage applications, it must be able to support data transmission with large - page capacity. For example, in holographic optical storage, a large number of spatial channels need to be supported to make full use of the storage medium. Therefore, how to efficiently transmit and decode these high - density data pages is a key issue. 3. **Overcoming the gap between simulation and experiment**: Theoretical simulation models may not fully reflect the optical imperfections in actual experiments. For example, a small deviation in the shape of the light pipe during the manufacturing process will affect the light - transmission characteristics. How to make the model adapt to these actual deviations is also an important challenge. ### Solutions The paper proposes a method that combines differentiable ray - tracing and machine learning to solve the above problems. Specific measures include: - **Using U - Net for convolution operations**: By combining physical modeling with deep learning and using the U - Net network structure for efficient convolution operations, the quadratic complexity problem in traditional methods is avoided. - **Introducing a differentiable ray - tracer**: Compared with mode - based locators, ray - tracing - based locators can better adapt to optical imperfections in experiments and can be automatically adjusted during the training process to overcome the gap between simulation and experiment. - **Mutual information estimation**: The von - Mises distribution is used to estimate the mutual information (MI) between the input symbols and the decoded output to evaluate the amount of information at each symbol position. This method not only improves the decoding accuracy but also provides a quantification of the prediction uncertainty. In summary, this paper aims to achieve efficient large - scale image data transmission and decoding, especially in the multimode optical fiber environment, through innovative physical modeling and machine - learning techniques.

Efficient data transport over multimode light-pipes with Megapixel images using differentiable ray tracing and Machine-learning

Learning to transmit images through optical speckle of a multimode fiber with high fidelity

Deep-learning-based high-resolution recognition of fractional-spatial-mode-encoded data for free-space optical communications

Spatially Variant Deblur and Image Enhancement in a Single Multimode Fiber Imaged by Deep Learning

Multimode optical fiber transmission with a deep learning network

Seeing at a Distance with Multicore Fibers

Direct Optical Convolution Computing Based on Arrayed Waveguide Grating Router

Exploiting holographically encoded variance to transmit labelled images through a multimode optical fiber

Deep Learning with Passive Optical Nonlinear Mapping

Versatile multimode fiber network with high capacity enabled by deep learning

Image transmission through a flexible multimode fiber by deep learning

Nonlinear optical encoding enabled by recurrent linear scattering

Learning-based multiplexed transmission of scattered twisted light through a kilometer-scale standard multimode fiber

Learning-enabled recovering scattered data from twisted light transmitted through a long standard multimode fiber

Shear-interference assisted deep-learning for enhancing spatially multiplexing capacity of free-space communication

Self-supervised dynamic learning for long-term high-fidelity image transmission through unstabilized diffusive media

Massively Parallel Universal Linear Transformations using a Wavelength-Multiplexed Diffractive Optical Network

All-optically untangling light propagation through multimode fibres

Photonic Unsupervised Learning Variational Autoencoder for High-Throughput and Low-Latency Image Transmission

Feature decoupled knowledge distillation enabled lightweight image transmission through multimode fibers

Asynchronous transfer of individually addressable optical channels through a multimode fiber using phase conjugation