Abstract:In recent years, convolutional-neural-network based stereo matching methods have achieved significant gains compared to conventional methods in terms of both speed and accuracy. Current state-of-the-art disparity estimation algorithms require many parameters and large amounts of computational resources and are not suited for applications on edge devices. In this paper, we propose an end-to-end light-weight network (LWNet) for fast stereo matching, which consists of an efficient backbone with multi-scale feature fusion for feature extraction, a 3D U-Net aggregation architecture for disparity computation, and color guidance in a 2D convolutional neural network (CNN) for disparity refinement. We adopt MobileNetV2 as an efficient backbone in feature extraction. The channel attention module is applied to improve the representational capacity of features and multi-resolution information is adaptively incorporated into the cost volume via cross-scale connections. In addition, instead of using regular 3D convolutions, we utilize pseudo 3D convolutions in the 3D U-Net architecture to aggregate the cost volume for a better balance between computational cost and accuracy. Further, we introduce a left-right consistency check and color guidance and design a robust disparity refinement network with skip connections and dilated convolutions to capture global context information and further improve disparity-estimation accuracy with little computational cost and memory space. A depth-wise separable convolution is proposed to replace all the standard convolutions in the section of disparity refinement, which can decrease computational complexity and the number of parameters without significant accuracy reduction. Extensive experiments on Scene Flow, KITTI 2015, and KITTI 2012 benchmarks demonstrate that the proposed LWNet achieves competitive accuracy when compared with state-of-the-art stereo matching methods.

DSCNet: lightweight and efficient self-supervised network via depthwise separable cross convolution blocks for speckle image matching

Untrained Neural Network Enabling Fast and Universal Structured-Illumination Microscopy

A lightweight and stochastic depth residual attention network for remote sensing scene classification

Light-field-depth-estimation Network Based on Epipolar Geometry and Image Segmentation.

High-generalization deep sparse pattern reconstruction: feature extraction of speckles using self-attention armed convolutional neural networks

Mutual Support and Promotion: Learning Structure Compensation and Context Completion for Low-Light Vision

Deep-learned speckle pattern and its application to ghost imaging

RSCNN: A CNN-Based Method to Enhance Low-Light Remote-Sensing Images

Machine learning meets Singular Optics: Speckle-based Structured light demultiplexing

Depth_ ASPP_ UNet++: A novel 2D ESPI phase unwrapping based on convolutional neural network

Double-Shot 3D Shape Measurement with a Dual-Branch Network for Structured Light Projection Profilometry

Seeing the Invisible through Speckle Images

Accurate 3D reconstruction of single-frame speckle-encoded textureless surfaces based on densely connected stereo matching network

DeepSCI: scalable speckle correlation imaging using physics-enhanced deep learning

A Light-Weight Network with Multi-Scale Features Fusion and Color Guidance for Stereo Matching

StrainNet-3D: Real-time and Robust 3-Dimensional Speckle Image Correlation Using Deep Learning

A Light-Weight Stereo Matching Network Based on Multi-Scale Features Fusion and Robust Disparity Refinement

High performance few-mode fiber-based light field direction sensing system using deep convolutional neural network: fiber speckle demodulation network (FSDNET)

Image reconstruction from speckle patterns on double diffusers condition by deep learning

Imaging through a Scattering Medium under Different Intensities of Ambient Light Interference

Real-time X-ray Phase-contrast Imaging Using SPINNet -- A Speckle-based Phase-contrast Imaging Neural Network