Abstract:The use of social media networks and mobile devices has experienced tremendous growth in recent years. This has led to a surge in the number of videos recorded and uploaded to social media platforms like TikTok and YouTube. However, this increase has also resulted in the rise of illegal duplicate videos, which are essentially the same as the original videos but with minor editing effects and variations in coding. In addition, the large number of duplicate videos is a major storage and communication efficiency issue. The task of finding duplicate videos from a large repository is referred to as video deduplication. Video deduplication is a crucial task for applications like saving storage space and detecting copyright infringement. This work proposes a fast and robust location-aware video deduplication system capable of retrieving duplicate videos from a large repository extremely quickly. In addition, the proposed system has the ability to find the precise location of the query video in the retrieved videos. To identify and localize short video clips against large video repositories, we utilize robust image-level features from keypoint aggregation and deep learning along with an efficient KNN search of query frames with a multiple k-d tree setup, giving us a set of candidate video clips. Then, a fast temporal consistence pruning algorithm re-ranks the clip-level candidates and identifies the matching clip along with its temporal location in a sequence in an efficient way. The system was tested on 1 million frame/145 hour and 4.5 million frame/636 hour repositories generated via the large-scale FIVR-200K and VCSL datasets, respectively. The proposed system achieves a recall of 98.8% and 94.1% for the FIVR-200K and VCSL datasets, respectively. A query frame is searched as fast as 83.96ms and 462.59ms from a 1 million frame/145 hour and a 4.5 million frame/636 hour repository, respectively. These experimental results demonstrate that our system is highly accurate and that the time consumption is extremely low for retrieving video along with its timestamp information from large-scale repositories.

Online Near-Duplicate Video Clip Detection and Retrieval: an Accurate and Fast System

Practical Online Near-Duplicate Subsequence Detection for Continuous Video Streams.

Efficient and Continuous Near-duplicate Video Detection

Interactive near-duplicate video retrieval and detection.

Statistical Summarization of Content Features for Fast Near-Duplicate Video Detection.

Quick Identification of Near-Duplicate Video Sequences with Cut Signature

A Detection Method for Near Duplicate Video Clips Based on Content Similarity

UQLIPS: a Real-Time Near-Duplicate Video Clip Detection System

An Improved Video Identification Scheme Based on Video Tomography.

A Fast Algorithm for Near-duplicate Video Detection

An Image-Based Near-Duplicate Video Retrieval and Localization Using Improved Edit Distance

On Real-Time Detecting Duplicate Web Videos

Fast Video Deduplication and Localization with Temporal Consistence Re-Ranking

Near-duplicate Keyframe Retrieval by Nonrigid Image Matching.

Fast And Robust Detection Of Near-Duplicates In Web Video Database

Practical on-line near-duplicate detection for web video

A content-based video copy detection method with randomly projected binary features.

Web video duplicate detection based on video vocabulary

Continuous Content-Based Copy Detection over Streaming Videos

A Fast Video Copy Detection Approach By Dynamic Programming

A Novel Video Copy Sub-sequence Detection and Location Method.