Real-Time Image Stitching with Transformers for Complex Traffic Environment

Zhiwei Shen,Bin Kong

DOI: https://doi.org/10.1007/978-981-99-7549-5_23

2023-01-01

Abstract:The existing traditional image stitching methods suffer from accumulating errors in the matching process, which severely limits the final stitching results. Additionally, they cannot find correct correspondences in complex scenes with poor texture, repetitive patterns, and lighting variations. To address these issues, we propose a real-time image stitching method based on the Transformer structure. The self-attention mechanism of the Transformer can dynamically adjust the receptive field based on input content, allowing for effective global information retrieval. Compared to traditional manual and CNN-based deep learning methods, our approach generates dense and high-quality matches even in complex environments. We reduce the complexity of the Transformer model and improve matching speed through knowledge distillation. We use the Kmeans algorithm to optimize point selection for feature point screening and compute the homography matrix value based on the matching result for pixel transformation. After aligning the overlapping regions of the images and averaging them, we apply mask weights obtained from Gaussian fusion of target image layers to achieve the final stitching result. Experimental results demonstrate that our method produces more accurate matches in complex scenes with weak textures compared to traditional and deep learning methods, while also having the shortest matching and stitching times and good stitching results.