Elimination of Irregular Boundaries and Seams for UAV Image Stitching with a Diffusion Model

Jun Chen,Yongxi Luo,Jie Wang,Honghua Tang,Yixian Tang,Jianhui Li

DOI: https://doi.org/10.3390/rs16091483

IF: 5

2024-04-24

Remote Sensing

Abstract:Unmanned aerial vehicle (UAV) image stitching refers to the process of combining multiple UAV images into a single large-format, wide-field image, and the stitched image often contains large irregular boundaries and multiple stitching seams. Usually, irregular boundaries are addressed using grid-constrained methods, while seams are optimized through the design of energy functions and penalty terms applied to the pixels at the seams. The above-mentioned two solutions can only address one of the two issues individually and are often limited to pairwise stitching of images. To the best of our knowledge, there is no unified approach that can handle both seams and irregular boundaries in the context of multi-image stitching for UAV images. Considering that addressing irregular boundaries involves completing missing information for regularization and that mitigating seams involves generating images near the stitching seams, both of these challenges can be viewed as instances of a mask-based image completion problem. This paper proposes a UAV image stitching method based on a diffusion model. This method uniformly designs masks for irregular boundaries and stitching seams, and the unconditional score function of the diffusion model is then utilized to reverse the process. Additional manifold gradient constraints are applied to restore masked images, eliminating both irregular boundaries and stitching seams and resulting in higher perceptual quality. The restoration maintains high consistency in texture and semantics. This method not only simultaneously addresses irregular boundaries and stitching seams but also is unaffected by factors such as the number of stitched images, the shape of irregular boundaries, and the distribution of stitching seams, demonstrating its robustness.

environmental sciences,imaging science & photographic technology,remote sensing,geosciences, multidisciplinary

What problem does this paper attempt to address?

### Problems the Paper Attempts to Solve

The paper primarily aims to address the issues of irregular boundaries and stitching seams that occur during the image stitching process of Unmanned Aerial Vehicles (UAVs). Specifically:

1. **Irregular Boundaries**: When stitching multiple images, irregular boundaries often appear due to the inability to achieve perfect alignment.
2. **Stitching Seams**: When stitching multiple images, noticeable stitching seams appear in the overlapping areas due to incomplete alignment.

Existing methods often can only solve one of these problems individually, and most methods are only applicable to stitching two images at a time. This paper proposes a diffusion model-based approach that unifies the issues of irregular boundaries and stitching seams into a mask-based image inpainting problem. This method not only addresses both irregular boundaries and stitching seams simultaneously but also exhibits good robustness, unaffected by the number of stitched images, the shape of irregular boundaries, or the distribution of stitching seams.

By introducing the diffusion model and Manifold Gradient Constraint, this method maintains texture and semantic consistency during the inpainting process, thereby improving perceptual quality. Additionally, to accommodate input images of different sizes, the paper introduces the Local Implicit Image Function (LIIF) to adjust image sizes to match the requirements of the diffusion model.