Abstract:Limited by aircraft flight altitude and camera parameters, it is necessary to obtain wide-angle panoramas quickly by stitching aerial images, which is helpful in rapid disaster investigation, recovery after earthquakes, and aerial reconnaissance. However, most existing stitching algorithms do not simultaneously meet practical real-time, robustness, and accuracy requirements, especially in the case of a long-distance multistrip flight. In this paper, we propose a novel image-only real-time UAV image mosaic framework for long-distance multistrip flights that does not require any auxiliary information, such as GPS or GCPs. The framework has a complete structure, mainly consisting of the three tasks of automatic initialization, current frame tracking, and real-time mosaic generation. The stitching plane is determined in the initialization process, the homography transformation of the current image is estimated in the tracking task, and the image is mapped to the stitching plane to generate and update the panorama in the real-time mosaic process. The core idea is that, in the tracking task, we introduce and develop a keyframe insertion strategy to generate a keyframe list and, on this basis, design a homography matrix estimation based on a local optimization strategy to reduce the accumulated error when continuously stitching image sequences collected online by UAVs and to realize real-time, effective UAV image mosaic construction. In addition, this framework has good scalability, which is not limited to a specific algorithm. To evaluate the effectiveness of the proposed framework, we carry out a large number of experiments on the AirSim simulation platform and present an exhaustive evaluation in some sequences from a popular dataset. Qualitative and quantitative experimental results in simulation and real environments demonstrate that our algorithm can obtain an effective and robust mosaic image in real-time. Through strategy comparison experiments, it is proven that the keyframe insertion strategy and the local optimization strategy both improve the stitching performance. Compared with five state-of-art image stitching approaches, the mosaic effect of the proposed method is comparable or better. In terms of algorithm speed, its performance is superior to them. Additionally, experiments of illumination change and feature replacement in the framework verify the good adaptability and scalability of the algorithm.

Intrinsic Hyperspectral Image Recovery for UAV Strips Stitching

Unaligned Hyperspectral Image Fusion Via Registration and Interpolation Modeling

Satellite-ground Integrated Destriping Network: A New Perspective for EO-1 Hyperion and Chinese Hyperspectral Satellite Datasets

Residual image recovery method based on the dual-camera design of a compressive hyperspectral imaging system

Infrared and Visible Image Registration in UAV Inspection

An adaptive 3D reconstruction method for asymmetric dual-angle multispectral stereo imaging system on UAV platform

Seamless Mosaicking of UAV-Based Push-Broom Hyperspectral Images for Environment Monitoring

A Simplified Method For Uav Multispectral Images Mosaicking

Seamless UAV Hyperspectral Image Stitching Using Optimal Seamline Detection via Graph Cuts

Hyperspectral Image Restoration Combining Intrinsic Image Characterization With Robust Noise Modeling

Enhanced Incremental Image Stitching for Low-Altitude UAV Imagery With Depth Estimation

Weighted Low-rank Tensor Recovery for Hyperspectral Image Restoration

Elimination of Irregular Boundaries and Seams for UAV Image Stitching with a Diffusion Model

Image-Only Real-Time Incremental UAV Image Mosaic for Multi-Strip Flight

A Stereo Dual-Channel Dynamic Programming Algorithm for UAV Image Stitching

Image Restoration Via UAVFormer for Under-Display Camera of UAV.

Unleashing the full potential of hyperspectral imaging: Decoupled image and frequency-domain spatial–spectral framework

A Deep Unfolding Network Based on Intrinsic Image Decomposition for Pansharpening

Image Quality Assessment of UAV Hyperspectral Images Using Radiant, Spatial, and Spectral Features Based on Fuzzy Comprehensive Evaluation Method

UAV remote sensing image stitching via improved VGG16 Siamese feature extraction network

Hyperspectral Image Restoration Using Weighted Group Sparsity-Regularized Low-Rank Tensor Decomposition