Abstract:Irregular boundaries in image stitching naturally occur due to freely moving cameras. To deal with this problem, existing methods focus on optimizing mesh warping to make boundaries regular using the traditional explicit solution. However, previous methods always depend on hand-crafted features (e.g., keypoints and line segments). Thus, failures often happen in overlapping regions without distinctive features. In this paper, we address this problem by proposing RecStitchNet, a reasonable and effective network for image stitching with rectangular boundaries. Considering that both stitching and imposing rectangularity are non-trivial tasks in the learning-based framework, we propose a three-step progressive learning based strategy, which not only simplifies this task, but gradually achieves a good balance between stitching and imposing rectangularity. In the first step, we perform initial stitching by a pre-trained state-of-the-art image stitching model, to produce initially warped stitching results without considering the boundary constraint. Then, we use a regression network with a comprehensive objective regarding mesh, perception, and shape to further encourage the stitched meshes to have rectangular boundaries with high content fidelity. Finally, we propose an unsupervised instance-wise optimization strategy to refine the stitched meshes iteratively, which can effectively improve the stitching results in terms of feature alignment, as well as boundary and structure preservation. Due to the lack of stitching datasets and the difficulty of label generation, we propose to generate a stitching dataset with rectangular stitched images as pseudo-ground-truth labels, and the performance upper bound induced from the it can be broken by our unsupervised refinement. Qualitative and quantitative results and evaluations demonstrate the advantages of our method over the state-of-the-art.

Real-Time Image Stitching with Convolutional Neural Networks

RecStitchNet: Learning to Stitch Images with Rectangular Boundaries

Unsupervised Deep Image Stitching: Reconstructing Stitched Features to Images

Accelerating High-resolution Image Stitching for the Dual Camera System Based on GPU

MGHE-Net: A Transformer-Based Multi-Grid Homography Estimation Network for Image Stitching

Real-Time Image Stitching with Transformers for Complex Traffic Environment

A Joint Framework for Underwater Sequence Images Stitching Based on Deep Neural Network Convolutional Neural Network

Image Stitching Based on Planar Region Consensus

Image Stitching Based on Semantic Planar Region Consensus

Towards All Weather and Unobstructed Multi-Spectral Image Stitching: Algorithm and Benchmark

A view-free image stitching network based on global homography

Image stitching via deep homography estimation

Multi-Spectral Image Stitching via Spatial Graph Reasoning

Real-Time Registration In Image Stitching Under The Microscope

VIGCN: an isotropic natural image stitching network based on graph convolution

Edge-guided Composition Network for Image Stitching

Robust Cylindrical Panorama Stitching for Low-Texture Scenes Based on Image Alignment Using Deep Learning and Iterative Optimization

A Fast and Robust Real-Time Surveillance Video Stitching Method

SynStitch: a Self-Supervised Learning Network for Ultrasound Image Stitching Using Synthetic Training Pairs and Indirect Supervision

Multi-scenes Image Stitching Based on Autonomous Driving