Abstract:Irregular boundaries in image stitching naturally occur due to freely moving cameras. To deal with this problem, existing methods focus on optimizing mesh warping to make boundaries regular using the traditional explicit solution. However, previous methods always depend on hand-crafted features (e.g., keypoints and line segments). Thus, failures often happen in overlapping regions without distinctive features. In this paper, we address this problem by proposing RecStitchNet, a reasonable and effective network for image stitching with rectangular boundaries. Considering that both stitching and imposing rectangularity are non-trivial tasks in the learning-based framework, we propose a three-step progressive learning based strategy, which not only simplifies this task, but gradually achieves a good balance between stitching and imposing rectangularity. In the first step, we perform initial stitching by a pre-trained state-of-the-art image stitching model, to produce initially warped stitching results without considering the boundary constraint. Then, we use a regression network with a comprehensive objective regarding mesh, perception, and shape to further encourage the stitched meshes to have rectangular boundaries with high content fidelity. Finally, we propose an unsupervised instance-wise optimization strategy to refine the stitched meshes iteratively, which can effectively improve the stitching results in terms of feature alignment, as well as boundary and structure preservation. Due to the lack of stitching datasets and the difficulty of label generation, we propose to generate a stitching dataset with rectangular stitched images as pseudo-ground-truth labels, and the performance upper bound induced from the it can be broken by our unsupervised refinement. Qualitative and quantitative results and evaluations demonstrate the advantages of our method over the state-of-the-art.

Rethinking Supervision in Document Unwarping: A Self-consistent Flow-free Approach

Deep Unrestricted Document Image Rectification.

DocScanner: Robust Document Image Rectification with Progressive Learning

DocUNet: Document Image Unwarping Via a Stacked U-Net

Effective Document Image Rectification via a Deep Learning Framework

RecStitchNet: Learning to Stitch Images with Rectangular Boundaries

Learning from Documents in the Wild to Improve Document Unwarping

Geometric Representation Learning for Document Image Rectification

DocMAE: Document Image Rectification via Self-supervised Representation Learning

Learning By Analogy: Reliable Supervision From Transformations For Unsupervised Optical Flow Estimation

DewarpNet: Single-Image Document Unwarping with Stacked 3D and 2D Regression Networks.

UVDoc: Neural Grid-based Document Unwarping

Layout-aware Single-image Document Flattening

Fourier Document Restoration for Robust Document Dewarping and Recognition

DocTr: Document Image Transformer for Geometric Unwarping and Illumination Correction

Restoring Camera-Captured Distorted Document Images

Unsupervised Domain Adaptation Approach for Vision-Based Semantic Understanding of Bridge Inspection Scenes Without Manual Annotations

A Document Rectification Approach Dealing with Both Perspective Distortion and Warping Based on Text Flow Curve Fitting

Efficient Joint Rectification of Photometric and Geometric Distortions in Document Images.

Supervision-by-Registration: an Unsupervised Approach to Improve the Precision of Facial Landmark Detectors

RecRecNet: Rectangling Rectified Wide-Angle Images by Thin-Plate Spline Model and DoF-based Curriculum Learning