Abstract:Irregular boundaries in image stitching naturally occur due to freely moving cameras. To deal with this problem, existing methods focus on optimizing mesh warping to make boundaries regular using the traditional explicit solution. However, previous methods always depend on hand-crafted features (e.g., keypoints and line segments). Thus, failures often happen in overlapping regions without distinctive features. In this paper, we address this problem by proposing RecStitchNet, a reasonable and effective network for image stitching with rectangular boundaries. Considering that both stitching and imposing rectangularity are non-trivial tasks in the learning-based framework, we propose a three-step progressive learning based strategy, which not only simplifies this task, but gradually achieves a good balance between stitching and imposing rectangularity. In the first step, we perform initial stitching by a pre-trained state-of-the-art image stitching model, to produce initially warped stitching results without considering the boundary constraint. Then, we use a regression network with a comprehensive objective regarding mesh, perception, and shape to further encourage the stitched meshes to have rectangular boundaries with high content fidelity. Finally, we propose an unsupervised instance-wise optimization strategy to refine the stitched meshes iteratively, which can effectively improve the stitching results in terms of feature alignment, as well as boundary and structure preservation. Due to the lack of stitching datasets and the difficulty of label generation, we propose to generate a stitching dataset with rectangular stitched images as pseudo-ground-truth labels, and the performance upper bound induced from the it can be broken by our unsupervised refinement. Qualitative and quantitative results and evaluations demonstrate the advantages of our method over the state-of-the-art.

Image rectangling network based on reparameterized transformer and assisted learning

RecStitchNet: Learning to Stitch Images with Rectangular Boundaries

Deep Rectangling for Image Stitching: A Learning Baseline

Remote Sensing Image Rectangling With Iterative Warping Kernel Self-Correction Transformer

Rectification for Stitched Images with Deformable Meshes and Residual Networks

SDR: stepwise deep rectangling model for stitched images

RecRecNet: Rectangling Rectified Wide-Angle Images by Thin-Plate Spline Model and DoF-based Curriculum Learning

Reconstructing the Image Stitching Pipeline: Integrating Fusion and Rectangling into a Unified Inpainting Model

Rectangular-Output Image Stitching

RectanglingGAN: Deep Rectangling for Stitched Image Via Image Inpainting

RecDiffusion: Rectangling for Image Stitching with Diffusion Models

Streamlining the Image Stitching Pipeline: Integrating Fusion and Rectangling into a Unified Model

A rectification algorithm for un-calibrated multi-view images based on SIFT features

Real-Time Image Stitching with Transformers for Complex Traffic Environment

CTFCD: Channel Transformer Based on Full Convolutional Decoder for Single Image Deraining

SegStitch: Multidimensional Transformer for Robust and Efficient Medical Imaging Segmentation

MGHE-Net: A Transformer-Based Multi-Grid Homography Estimation Network for Image Stitching

Unsupervised Deep Image Stitching: Reconstructing Stitched Features to Images

Geometric Rectification of Document Images Using Adversarial Gated Unwarping Network

Stitching Transformer and Convolution in U-Net for Medical Image Segmentation

Rectangling and Enhancing Underwater Stitched Image Via Content-Aware Warping and Perception Balancing