DSP-based Parallel Optimization for Real-Time Video Stitching
Huang Xiaofeng,Tang Ran,Zhou Yang,Yin Haibing,Yan Chenggang
DOI: https://doi.org/10.1007/s11554-023-01275-x
IF: 2.293
2023-01-01
Journal of Real-Time Image Processing
Abstract:Video stitching is a technique that stitches multiple overlapped videos acquired from different cameras, which is widely applied in many applications, including video surveillance, autonomous driving, and virtual reality. Feature-based stitching methods are popular in this area because of their invariance property and efficiency. However, the video stitching pipeline is relatively complicated and the amount of data calculation is large, which impedes its real-time applications. In this paper, we propose a real-time video stitching framework based on vision Digital Signal Processing (DSP). Real-time processing is achieved by the algorithm-level and system-level optimizations. In the algorithm of ORB feature extraction, methods, including look-up table, linear approximation, and single instruction multiple data (SIMD), are adopted to optimize its computation on DSP. In the algorithm of feature matching, the Hamming distance calculation is eliminated for some feature point pairs when their coordinates and angles are not satisfied with certain conditions. Reverse feature matching is proposed to improve registration accuracy. In system-level optimization, the directed acyclic graph (DAG)-based scheduling is proposed to improve the calculation efficiency on dual DSPs, and a ping-pong buffer is utilized to speed up the data transmission between DSP and external memory. Experimental results show that the proposed method can achieve a ten times speedup than that of the CPU, and it can achieve 1536 × 1024@37fps real-time processing on vision DSP.