Key frame extraction based on quaternion Fourier transform with multiple features fusion
Yunzuo Zhang,Jiayu Zhang,Ruixue Liu,Pengfei Zhu,Yameng Liu
DOI: https://doi.org/10.1016/j.eswa.2022.119467
IF: 8.5
2023-01-04
Expert Systems with Applications
Abstract:Key frame extraction, an important technique to improve video viewing efficiency and reduce video redundancy, has attracted considerable attention. Notwithstanding, existing key frame extraction for surveillance video not only fails to extract the local detail information of the target accurately but also preprocess the image with grayscale, so the integrity of the image information is also damaged. In order to solve the above-described problems, the paper presents a key frame extraction method based on the quaternion Fourier transform with the multiple features fusion. Initially, the method extracts dynamic and static features that can characterize the color image information separately. Then, the quaternion matrix is used to perform the quaternion Fourier transform on the different features to obtain the fused phase spectrum of the Fourier transform. Immediately after, the information containing the overall structure of the image is filtered by Gaussian filtering, and the quaternion Fourier inverse transform is performed to obtain the fused feature map. Finally, an adaptive key frame filtering criterion, which uses the fused feature map to extract the key frames is constructed in this paper for accurately extracting key frames. Notably, experimental results indicate that the proposed method ensures the integrity of the image information while accurately capturing the global and local motion state of the target.
computer science, artificial intelligence,engineering, electrical & electronic,operations research & management science