Learning Hierarchical Fingerprints via Multi-Level Fusion for Video Integrity and Source Analysis
Yuanman Li,Jiaxiong Ye,Limin Zeng,Rongqin Liang,Xianwei Zheng,WeiWei Sun,Na Wang
DOI: https://doi.org/10.1109/tce.2024.3357977
2024-01-01
IEEE Transactions on Consumer Electronics
Abstract:As a prevalent form of multimodal data, video data plays a crucial role in numerous applications, offering various benefits. Meanwhile, video integrity and source issues also pose security risks. Video data is multimodal, containing a container describing video coding and packaging, along with a video data stream featuring visual and audio information. Many works on video integrity and source analysis focus on video containers, and they overlook the fact that a malicious user can readily manipulate these traces within the containers by reconstructing them without transcoding. In our research, we propose a hierarchical fingerprint learning framework through multi-level fusion for video integrity and source analysis. Our approach integrates video encoding attributes, extracting multi-level features from both decoded video key frames and reference frames. We model the dependencies between these features based on encoding characteristics, effectively revealing hidden clues in spatial and temporal domains related to various video processing techniques. Additionally, we introduce a hierarchical framework to fuse multiple clues from different groups of pictures (GOPs), facilitating collaborative feature learning across multiple GOPs. Extensive experiments on publicly available datasets validate the effectiveness of our method in tasks related to video integrity verification and source identification. Our approach provides support for ensuring the credibility and traceability of video content in consumer applications.
telecommunications,engineering, electrical & electronic