Abstract:Irregular boundaries in image stitching naturally occur due to freely moving cameras. To deal with this problem, existing methods focus on optimizing mesh warping to make boundaries regular using the traditional explicit solution. However, previous methods always depend on hand-crafted features (e.g., keypoints and line segments). Thus, failures often happen in overlapping regions without distinctive features. In this paper, we address this problem by proposing RecStitchNet, a reasonable and effective network for image stitching with rectangular boundaries. Considering that both stitching and imposing rectangularity are non-trivial tasks in the learning-based framework, we propose a three-step progressive learning based strategy, which not only simplifies this task, but gradually achieves a good balance between stitching and imposing rectangularity. In the first step, we perform initial stitching by a pre-trained state-of-the-art image stitching model, to produce initially warped stitching results without considering the boundary constraint. Then, we use a regression network with a comprehensive objective regarding mesh, perception, and shape to further encourage the stitched meshes to have rectangular boundaries with high content fidelity. Finally, we propose an unsupervised instance-wise optimization strategy to refine the stitched meshes iteratively, which can effectively improve the stitching results in terms of feature alignment, as well as boundary and structure preservation. Due to the lack of stitching datasets and the difficulty of label generation, we propose to generate a stitching dataset with rectangular stitched images as pseudo-ground-truth labels, and the performance upper bound induced from the it can be broken by our unsupervised refinement. Qualitative and quantitative results and evaluations demonstrate the advantages of our method over the state-of-the-art.

Paper Stitching Using Maximum Tolerant Seam under Local Distortions.

Remote Sensing Image Stitch Using Modified Structure Deformation

Seamless stitching with shape deformation for historical document images

Use of Shape Deformation to Seamlessly Stitch Historical Document Images

RecStitchNet: Learning to Stitch Images with Rectangular Boundaries

Studies on Image Stitching Algorithms in Machine Vision Inspection of Solar Panel

A Robust Image Stitching Based on Solution Set Smoothing

Research on Image Stitching Algorithm Based on Point-Line Consistency and Local Edge Feature Constraints

Coarse-to-fine Seam Estimation for Image Stitching

Semantic segmentation guided feature point classification and seam fusion for image stitching

Sparse Feature Matching and Deformation Propagation for Seamless Image Stitching

Structure Preservation and Seam Optimization for Parallax-Tolerant Image Stitching

A new optimal seam selection method for airborne image stitching

Image stitching by line-guided local warping with global similarity constraint

A content-aware image stitching algorithm for mobile multimedia devices

Modification Takes Courage: Seamless Image Stitching via Reference-Driven Inpainting

Research on seamless image stitching based on fast marching method

Automatic Image Stitching Based on Scale Invariant Feature

Research on Natural Image Stitching Based on Extended Point-line Feature

Seamless UAV Hyperspectral Image Stitching Using Optimal Seamline Detection via Graph Cuts

Multimodel Fore-/Background Alignment for Seam-Based Parallax-Tolerant Image Stitching