Abstract:In recent decades, with the ever-growing scale of video data, near-duplicate videos continue to emerge. Data quality issues caused by near-duplicate videos are becoming more and more prominent, which has affected the application of normal videos. Although current studies on near-duplicate video detection can help uncover data quality issues for videos, they still lack a process of automatic merging for the video data represented by high-dimensional features, which makes it difficult to automatically clean the near-duplicate videos to improve data quality for video datasets. At present, there are few studies on near-duplicate video data cleaning. The existing studies have the sensitive problems of video data orderliness and initial clustering centers under a condition that prior distribution is unknown, which seriously affects the accuracy of near-duplicate video data cleaning. To address the above issues, an automatic near-duplicate video data cleaning method based on a consistent feature hash ring is proposed in this paper. First, a residual network with convolutional block attention modules, a long short-term memory deep network, and an attention model are integrated to construct an RCLA deep network with the multi-head attention mechanism to extract spatiotemporal features of video data. Then, a consistent feature hash ring is constructed, which can effectively alleviate the sensitivity of video data orderliness while providing a condition of near-duplicate video merging. To reduce the sensitivity of the initial cluster centers to the results of near-duplicate video cleansing, an optimized feature distance-means clustering algorithm is constructed by utilizing a mountain peak function on a consistent feature hash ring, which can implement automatic cleaning of near-duplicate video data. Finally, experiments are conducted based on a commonly used dataset named CC_WEB_VIDEO and a coal mining video dataset. Compared with some existing studies, simulation results demonstrate the performance of the proposed method.

Fast and accurate near-duplicate image elimination for visual sensor networks

Self-Similarity Superresolution for Resource-Constrained Image Sensor Node in Wireless Sensor Networks

Efficient near-duplicate image detection by learning from examples

Near-duplicate Keyframe Retrieval by Nonrigid Image Matching.

Near-Duplicate Image Detection System Using Coarse-to-Fine Matching Scheme Based on Global and Local CNN Features

Near-duplicate Keyframe Retrieval by Semi-Supervised Learning and Nonrigid Image Matching

Progressive Image Retrieval with Quality Guarantee under MapReduce Framework

Evolution of a Web-Scale Near Duplicate Image Detection System

Query Oriented Subspace Shifting for Near-Duplicate Image Detection

Near duplicate detection of images with area and proposed pixel‐based feature extraction

Squirrel Search Optimization-based near-duplicate image detection

An Automatic Near-Duplicate Video Data Cleaning Method Based on a Consistent Feature Hash Ring

Dataset and Case Studies for Visual Near-Duplicates Detection in the Context of Social Media

Fine-search for image copy detection based on local affine-invariant descriptor and spatial dependent matching

Efficient and Continuous Near-duplicate Video Detection

Near-duplicate image recognition

Efficient Feature Detection and Effective Post-Verification for Large Scale Near-Duplicate Image Search

Fast And Robust Detection Of Near-Duplicates In Web Video Database

Large-Scale Duplicate Detection for Web Image Search

Partial-Duplicate Clustering and Visual Pattern Discovery on Web Scale Image Database

Partial-Duplicate Image Retrieval via Saliency-Guided Visual Matching