GPU-based MapReduce for large-scale near-duplicate video retrieval

WangHanli,ZhuFengkuangtian,XiaoBo,WangLei,JiangYu-Gang

IF: 2.577

2015-01-01

Multimedia Tools and Applications

Abstract:With the exponential growth of multimedia data, people are overwhelmed with massive amount of online videos, of which Near-Duplicate Videos (NDVs) occupy a large portion. In this paper, we present ...

What problem does this paper attempt to address?

GPU-based MapReduce for Large-Scale Near-Duplicate Video Retrieval.

Hanli Wang,Fengkuangtian Zhu,Bo Xiao,Lei Wang,Yu-Gang Jiang

DOI: https://doi.org/10.1007/s11042-014-2185-x

IF: 2.577

2014-01-01

Multimedia Tools and Applications

Abstract:With the exponential growth of multimedia data, people are overwhelmed with massive amount of online videos, of which Near-Duplicate Videos (NDVs) occupy a large portion. In this paper, we present a novel framework for NDV retrieval, which explores the parallel power of two promising techniques: Graphics Processing Unit (GPU) and MapReduce. With the power of the proposed framework, various key algorithms in the field of computer vision, such as K-Means clustering, bag of features, inverted file index with hamming embedding and weak geometric consistency, are applied to NDV retrieval. Experimental results on the benchmark CC_WEB_VIDEO NDV dataset demonstrate that the proposed framework can significantly speed up processing huge amounts of video repositories.
Effective Multiple Feature Hashing for Large-Scale Near-Duplicate Video Retrieval

Jingkuan Song,Yi Yang,Zi Huang,Heng Tao Shen,Jiebo Luo

DOI: https://doi.org/10.1109/TMM.2013.2271746

IF: 7.3

2013-01-01

IEEE Transactions on Multimedia

Abstract:Near-duplicate video retrieval (NDVR) has recently attracted much research attention due to the exponential growth of online videos. It has many applications, such as copyright protection, automatic video tagging and online video monitoring. Many existing approaches use only a single feature to represent a video for NDVR. However, a single feature is often insufficient to characterize the video content. Moreover, while the accuracy is the main concern in previous literatures, the scalability of NDVR algorithms for large scale video datasets has been rarely addressed. In this paper, we present a novel approach—Multiple Feature Hashing (MFH) to tackle both the accuracy and the scalability issues of NDVR. MFH preserves the local structural information of each individual feature and also globally considers the local structures for all the features to learn a group of hash functions to map the video keyframes into the Hamming space and generate a series of binary codes to represent the video dataset. We evaluate our approach on a public video dataset and a large scale video dataset consisting of 132,647 videos collected from YouTube by ourselves. This dataset has been released (http://itee.uq.edu.au/shenht/UQ_VIDEO/). The experimental results show that the proposed method outperforms the state-of-the-art techniques in both accuracy and efficiency.
Multiple Feature Hashing for Real-Time Large Scale Near-Duplicate Video Retrieval

Jingkuan Song,Yi Yang,Zi Huang,Heng Tao Shen,Richang Hong

DOI: https://doi.org/10.1145/2072298.2072354

IF: 7.3

2013-01-01

IEEE Transactions on Multimedia

Abstract:Near-duplicate video retrieval (NDVR) has recently attracted lots of research attention due to the exponential growth of online videos. It helps in many areas, such as copyright protection, video tagging, online video usage monitoring, etc. Most of existing approaches use only a single feature to represent a video for NDVR. However, a single feature is often insufficient to characterize the video content. Besides, while the accuracy is the main concern in previous literatures, the scalability of NDVR algorithms for large scale video datasets has been rarely addressed. In this paper, we present a novel approach - Multiple Feature Hashing (MFH) to tackle both the accuracy and the scalability issues of NDVR. MFH preserves the local structure information of each individual feature and also globally consider the local structures for all the features to learn a group of hash functions which map the video keyframes into the Hamming space and generate a series of binary codes to represent the video dataset. We evaluate our approach on a public video dataset and a large scale video dataset consisting of 132,647 videos, which was collected from YouTube by ourselves. The experiment results show that the proposed method outperforms the state-of-the-art techniques in both accuracy and efficiency.
Million-scale near-duplicate video retrieval system.

Yang Cai,Linjun Yang,Wei Ping,Fei Wang,Tao Mei,Xian-Sheng Hua,Shipeng Li

DOI: https://doi.org/10.1145/2072298.2072484

2011-01-01

Abstract:In this paper, we present a novel near-duplicate video retrieval system serving one million web videos. To achieve both the effectiveness and efficiency, a visual word based approach is proposed, which quantizes each video frame into a word and represents the whole video as a bag of words. The system can respond to a query in 41ms with 78.4% MAP on average.
A Framework of Video Coding for Compressing Near-Duplicate Videos

Hanli Wang,Ming Ma,Yu-Gang Jiang,Zhihua Wei

DOI: https://doi.org/10.1007/978-3-319-04114-8_44

2014-01-01

Abstract:With the development of multimedia technique and social network, the amount of videos has grown rapidly, which brings about an increasingly substantial percentage of Near-Duplicate Videos (NDVs). It has been a hot research topic to retrieve NDVs for a number of applications such as copyright detection, Internet video ranking, etc. However, there exist a lot of redundancies in NDVs, and to the best of our knowledge it is an untouched research area on how to efficiently compress NDVs in a joint manner. In this work, a novel video coding framework is proposed to effectively compress NDVs by making full use of the relevance among them. Experimental results demonstrate that a significant storage saving can be achieved by the proposed NDV coding framework.
Fast And Robust Detection Of Near-Duplicates In Web Video Database

Hui Xu,Lu Liu,Li-feng Sun,Shi-qiang Yang

DOI: https://doi.org/10.1109/ICME.2008.4607429

2008-01-01

Abstract:Web video databases tend to contain tremendous near-duplicates with the explosive growth of online videos. How to detect and eliminate these near-duplicates has become an essential problem for web video storage and indexing. In this paper, we propose a hierarchical approach to solve this problem efficiently and effectively. For an incoming video, firstly we compare global features to exclude videos with large differences. Then pairwise comparison of key-frame level features with more computation cost is performed. For pairwise comparison, we adopt the Maximum Matching (MM) framework on bipartite graph. Instead of using standard well-known MM algorithm, a simple approximate algorithm of MM and a graph cut algorithm are applied for a higher speed. Experimental results over two databases summing up to more than 20,000 videos demonstrate our algorithm's excellent performance in both speed and accuracy.
Progressive Image Retrieval with Quality Guarantee under MapReduce Framework

Xing Gao,Xiangyu Shi,Guangyu Zhang,Juncong Lin,Minghong Liao,Kuan-Ching Li,Chaoyong Li

DOI: https://doi.org/10.1109/access.2018.2842796

IF: 3.9

2018-01-01

IEEE Access

Abstract:Because of the diversity and popularity of image acquisition techniques, data-driven methods for image analysis and editing have become popular. However, the explosive growth of images also presents challenges. Helping users to retrieve their expected images quickly and effectively is one of the most difficult tasks. Although various methods have been proposed, most methods cannot guarantee the quality of the image, which is typically required for analysis and authoring tasks. In this paper, we present a progressive image retrieval method with a quality guarantee. Images are gradually filtered by various criteria: starting from the quickest textual comparison, proceeding through a series of quality criteria, and ending with the most time-consuming contour match. The entire framework is parallelized under MapReduce to improve the performance. Various experiments are conducted to validate the performance and accuracy of the algorithm and the quality of the retrieved results. We also demonstrate the potential of the algorithm with an image synthesis prototype system.
Joint Compression of Near-Duplicate Videos.

Hanli Wang,Tao Tian,Ming Ma,Jun Wu

DOI: https://doi.org/10.1109/tmm.2016.2645398

IF: 7.3

2017-01-01

IEEE Transactions on Multimedia

Abstract:The expanding social network and multimedia technologies encourage more and more people to store and transmit information in visual format, such as image and video. However, the cost of this convenience brings about a shock to traditional video severs and exposes them under the risk of overloading. In the huge volume of online videos, there are a large amount of near-duplicate videos (NDVs). Although quite a number of research work have been proposed to detect NDVs, little research effort is made to compress these NDVs in a more effective manner than independent video compression. In this study, we make an in-depth exploration of the data redundancy of NDVs and propose a video analysis and coding framework to jointly compress NDVs. In order to employ the proposed NDV analysis and coding framework, a graph-based similar video grouping method and a number of preprocessing functions are designed to explore the correlation of visual information among NDVs and thus suit the requirement of joint video coding. Experimental results verify that the proposed NDV analysis and coding framework is able to effectively compress NDVs and thus save video data storage.
Near-duplicate Keyframe Retrieval by Nonrigid Image Matching.

Jianke Zhu,Steven C. H. Hoi,Michael R. Lyu,Shuicheng Yan

DOI: https://doi.org/10.1145/1459359.1459366

2008-01-01

Abstract:Near-duplicate image retrieval plays an important role in many real-world multimedia applications. Most previous approaches have some limitations. For example, conventional appearance-based methods may suffer from the illumination variations and occlusion issue, and local feature correspondence-based methods often do not consider local deformations and the spatial coherence between two point sets. In this paper, we propose a novel and effective Nonrigid Image Matching (NIM) approach to tackle the task of near-duplicate keyframe retrieval from real-world video corpora. In contrast to previous approaches, the NIM technique can recover an explicit mapping between two near-duplicate images with a few deformation parameters and find out the correct correspondences from noisy data effectively. To make our technique applicable to large-scale applications, we suggest an effective multi-level ranking scheme that filters out the irrelevant results in a coarse-to-fine manner. In our ranking scheme, to overcome the extremely small training size challenge, we employ a semi-supervised learning method for improving the performance using unlabeled data. To evaluate the effectiveness of our solution, we have conducted extensive experiments on two benchmark testbeds extracted from the TRECVID2003 and TRECVID2004 corpora. The promising results show that our proposed method is more effective than other state-of-the-art approaches for near-duplicate keyframe retrieval.
Joint Multi-View Hashing for Large-Scale Near-Duplicate Video Retrieval

Xiushan Nie,Weizhen Jing,Chaoran Cui,Chen Jason Zhang,Lei Zhu,Yilong Yin

DOI: https://doi.org/10.1109/tkde.2019.2913383

IF: 9.235

2020-10-01

IEEE Transactions on Knowledge and Data Engineering

Abstract:Multi-view hashing can well support large-scale near-duplicate video retrieval, due to its desirable advantages of mutual reinforcement of multiple features, low storage cost, and fast retrieval speed. However, there are still two limitations that impede its performance. First, existing methods only consider local structures in multiple features. They ignore the global structure that is important for near-duplicate video retrieval, and cannot fully exploit the dependence and complementarity of multiple features. Second, existing works always learn hashing functions bit by bit, which unfortunately increases the time complexity of hash function learning. In this paper, we propose a supervised hashing scheme, termed as joint multi-view hashing (JMVH), to address the aforementioned problems. It jointly preserves the global and local structures of multiple features while learning hashing functions efficiently. Specially, JMVH considers features of video as items, based on which an underlying Hamming space is learned by simultaneously preserving their local and global structures. In addition, a simple but efficient multi-bit hash function learning based on generalized eigenvalue decomposition is devised to learn multiple hash functions within a single step. It can significantly reduce the time complexity of conventional hash function learning processes that sequentially learn multiple hash functions bit by bit. The proposed JMVH is evaluated on two public databases: CC_WEB_VIDEO and UQ_VIDEO. Experimental results demonstrate that the proposed JMVH achieves more than a 5 percent improvement compared to several state-of-the-art methods which indicates the superior performance of JMVH.

computer science, information systems, artificial intelligence,engineering, electrical & electronic
Near-duplicate Keyframe Retrieval by Semi-Supervised Learning and Nonrigid Image Matching

Jianke Zhu,Steven C. H. Hoi,Michael R. Lyu,Shuicheng Yan

DOI: https://doi.org/10.1145/1870121.1870125

IF: 4.094

2011-01-01

ACM Transactions on Multimedia Computing Communications and Applications

Abstract:Near-duplicate keyframe (NDK) retrieval techniques are critical to many real-world multimedia applications. Over the last few years, we have witnessed a surge of attention on studying near-duplicate image/keyframe retrieval in the multimedia community. To facilitate an effective approach to NDK retrieval on large-scale data, we suggest an effective Multi-Level Ranking (MLR) scheme that effectively retrieves NDKs in a coarse-to-fine manner. One key stage of the MLR ranking scheme is how to learn an effective ranking function with extremely small training examples in a near-duplicate detection task. To attack this challenge, we employ a semi-supervised learning method, semi-supervised support vector machines, which is able to significantly improve the retrieval performance by exploiting unlabeled data. Another key stage of the MLR scheme is to perform a fine matching among a subset of keyframe candidates retrieved from the previous coarse ranking stage. In contrast to previous approaches based on either simple heuristics or rigid matching models, we propose a novel Nonrigid Image Matching (NIM) approach to tackle near-duplicate keyframe retrieval from real-world video corpora in order to conduct an effective fine matching. Compared with the conventional methods, the proposed NIM approach can recover explicit mapping between two near-duplicate images with a few deformation parameters and find out the correct correspondences from noisy data simultaneously. To evaluate the effectiveness of our proposed approach, we performed extensive experiments on two benchmark testbeds extracted from the TRECVID2003 and TRECVID2004 corpora. The promising results indicate that our proposed method is more effective than other state-of-the-art approaches for near-duplicate keyframe retrieval.
SVD: A Large-Scale Short Video Dataset for Near-Duplicate Video Retrieval.

Qing-Yuan Jiang,Yi He,Gen Li,Jian Lin,Lei Li,Wu-Jun Li

DOI: https://doi.org/10.1109/iccv.2019.00538

2019-01-01

Abstract:With the explosive growth of video data in real applications, near-duplicate video retrieval (NDVR) has become indispensable and challenging, especially for short videos. However, all existing NDVR datasets are introduced for long videos. Furthermore, most of them are small-scale and lack of diversity due to the high cost of collecting and labeling near-duplicate videos. In this paper, we introduce a large-scale short video dataset, called SVD, for the NDVR task. SVD contains over 500,000 short videos and over 30,000 labeled videos of near-duplicates. We use multiple video mining techniques to construct positive/negative pairs. Furthermore, we design temporal and spatial transformations to mimic user-attack behavior in real applications for constructing more difficult variants of SVD. Experiments show that existing state-of-the-art NDVR methods, including real-value based and hashing based methods, fail to achieve satisfactory performance on this challenging dataset. The release of SVD dataset will foster research and system engineering in the NDVR area. The SVD dataset is available at https://svdbase.github.io.
Learn from Unlabeled Videos for Near-duplicate Video Retrieval

Xiangteng He,Yulin Pan,Mingqian Tang,Yiliang Lv,Yuxin Peng

DOI: https://doi.org/10.1145/3477495.3532010

2022-01-01

Abstract:Near-duplicate video retrieval (NDVR) aims to find the copies or transformations of the query video from a massive video database. It plays an important role in many video related applications, including copyright protection, tracing, filtering and etc. Video representation and similarity search are crucial to any video retrieval system. To derive effective video representation, most video retrieval systems require a large amount of manually annotated data for training, making it costly inefficient. In addition, most retrieval systems are based on frame-level features for video similarity searching, making it expensive both storage wise and search wise. To address the above issues, we propose a video representation learning (VRL) approach to effectively address the above shortcomings. It first effectively learns video representation from unlabeled videos via contrastive learning to avoid the expensive cost of manual annotation. Then, it exploits transformer structure to aggregate frame-level features into clip-level to reduce both storage space and search complexity. It can learn the complementary and discriminative information from the interactions among clip frames, as well as acquire the frame permutation and missing invariant ability to support more flexible retrieval manners. Comprehensive experiments on two challenging near-duplicate video retrieval datasets, namely FIVR-200K and SVD, verify the effectiveness of our proposed VRL approach, which achieves the best performance of video retrieval on accuracy and efficiency.
CNN Retrieval based Unsupervised Metric Learning for Near-Duplicated Video Retrieval

Hao Cheng,Ping Wang,Chun Qi

DOI: https://doi.org/10.48550/arXiv.2105.14566

2021-05-30

Abstract:As important data carriers, the drastically increasing number of multimedia videos often brings many duplicate and near-duplicate videos in the top results of search. Near-duplicate video retrieval (NDVR) can cluster and filter out the redundant contents. In this paper, the proposed NDVR approach extracts the frame-level video representation based on convolutional neural network (CNN) features from fully-connected layer and aggregated intermediate convolutional layers. Unsupervised metric learning is used for similarity measurement and feature matching. An efficient re-ranking algorithm combined with k-nearest neighborhood fuses the retrieval results from two levels of features and further improves the retrieval performance. Extensive experiments on the widely used CC\_WEB\_VIDEO dataset shows that the proposed approach exhibits superior performance over the state-of-the-art.

Information Retrieval
GPU-Accelerated Video Copy Detection Based on Incremental Clustering

Ren Huamin,Zhang Yongdong,Lin Shouxun

2010-01-01

Journal of Computer-Aided Design & Computer Graphics

Abstract:For effectiveness of privacy protection and efficiency of copy detection on large video datasets,a fully GPU-based incremental copy detection scheme is proposed.When a newly added video arrives into the database,GPU on-chip decoder is called for video stream decoding.At the same time,high dimensional SIFT features are extracted on the frame in real-time,which is followed by an incremental K-means clustering method responding to the dynamic database used to update visual words codebook.Then,each frame is represented with a visual words frequency vector.Finally,a spatiotemporal sequence matching method based on visual words representation at frame level is used to determine whether the query is a copy.Experimental results show that our GPU implementation achieves up to a 63 times speedup over the CPU version.
Mining Near-Duplicate Graph for Cluster-Based Reranking of Web Video Search Results

Zi Huang,Bo Hu,Hong Cheng,Heng Tao Shen,Hongyan Liu,Xiaofang Zhou

DOI: https://doi.org/10.1145/1852102.1852108

2010-01-01

Abstract:Recently, video search reranking has been an effective mechanism to improve the initial text-based ranking list by incorporating visual consistency among the result videos. While existing methods attempt to rerank all the individual result videos, they suffer from several drawbacks. In this article, we propose a new video reranking paradigm called cluster-based video reranking (CVR). The idea is to first construct a video near-duplicate graph representing the visual similarity relationship among videos, followed by identifying the near-duplicate clusters from the video near-duplicate graph, then ranking the obtained near-duplicate clusters based on cluster properties and intercluster links, and finally for each ranked cluster, a representative video is selected and returned. Compared to existing methods, the new CVR ranks clusters and exhibits several advantages, including superior reranking by utilizing more reliable cluster properties, fast reranking on a small number of clusters, diverse and representative results. Particularly, we formulate the near-duplicate cluster identification as a novel maximally cohesive subgraph mining problem. By leveraging the designed cluster scoring properties indicating the cluster's importance and quality, random walk is applied over the near-duplicate cluster graph to rank clusters. An extensive evaluation study proves the novelty and superiority of our proposals over existing methods.
Video Organization: Near-Duplicate Video Clustering.

Tzu-Yi Hung,Ce Zhu,Gao Yang,Yap-Peng Tan

DOI: https://doi.org/10.1109/iscas.2012.6271637

2012-01-01

Abstract:It is not uncommon to see several videos of almost identical content on the internet. These near duplicates, coupled with the sheer number of videos, pose a big challenge to the effective organization of video clips online. We propose an adaptive classification approach to detect near-duplicate versions, and an integrated voting strategy to group clusters and to elect a representative for each cluster. Our proposed methods are based on our observation that near-duplicate videos usually span a small, albeit variable area in the feature space, while videos of different contents are scattered far apart. The classification method aims to select a suitable threshold by maximizing the margin for each video sequence in the similarity space, and the voting scheme focuses on merging subsets with mutual information based on neighbor information and inverted indices. Experimental results on an unconstrained web dataset including over 10000 videos demonstrate the efficacy of the proposed methods.
Near-Duplicate Video Retrieval Through Toeplitz Kernel Partial Least Squares.

Jia-Li Tao,Jian-Ming Zhang,Liangjun Wang,Xiangjun Shen,Zheng-Jun Zha

DOI: https://doi.org/10.1007/978-3-030-05716-9_29

2019-01-01

Abstract:The existence of huge volumes of near-duplicate videos shows a rising demand on effective near-duplicate video retrieval technique in copyright violation and search result re-ranking. In this paper, Kernel Partial Least Squares (KPLS) is used to find strong information correlation in near-duplicate videos. Furthermore, to solve the problem of “curse of kernelization” when querying a large-scale video database, we propose a Toeplitz Kernel Partial Least Squares method. The Toeplitz matrix multiplication can be implemented by the Fast Fourier Transform (FFT) to accelerate the computation. Extensive experiments on the widely used CC_WEB_VIDEO dataset demonstrate that the proposed approach exhibits superior performance of near-duplicate video retrieval (NDVR) over state-of-the-art methods, such as BCS, SE, SSBelt and CCA, achieving a mean average precision (MAP) score of 0.9665.
Distributed Video Processing Platform Based on Map Reduce

GENG Chen-yao,YAO Dan-ya,ZHANG Ying-ying,ZHANG Xu,CHANG Gang

DOI: https://doi.org/10.3969/j.issn.1000-3428.2012.10.086

2012-01-01

Abstract:Most advanced video incident detection algorithms are hard to apply in real-time systems because of its complexity.A distribute video processing platform based on Map Reduce is presented to solve the computation bottleneck,through which video processing algorithm is paralleled on time domain.A distributed video processing architecture is implemented with general-purpose computer.Test experiment proves the flexibility of system’s computation ability.
An Efficient Partial Video Copy Detection for a Large-scale Video Database

Zhan Luo,Lan Zhang,Jiewei Lai,Xinming Wang,Chen Tang

DOI: https://doi.org/10.1109/bigcom61073.2023.00024

2023-01-01

Abstract:Partial video copy detection (PVCD) aims to discover copy segments of query videos from a video database, which plays an important role in video copyright protection, filtering, tracking, etc. For a large-scale video database, PVCD can be divided into two stages: the first stage involves searching for video-level copies of the query video in the database, and the second stage is to further localize the copy segments within the video-level copies. Thus, two major challenges arise: (1) efficiently and effectively calculating the similarity between videos; (2) localizing mixed-duration video pairs. To address the above challenges, we propose an efficient PVCD approach for a large-scale video database, based on the Bag-of-Words (BoW) framework, which decouples video-level similarity and copy localization into cell-level. This approach consists of two modules. The first is an efficient video similarity measurement (VSM) module for the large-scale video database. VSM aggregates cell-level similarity into video-level similarity, and with a dual index, it greatly improves retrieval speed while accurately measuring spatiotemporal transformations. The second is a greedy pattern detection (GPD) module for video copy localization. GPD quickly and accurately detects similarity patterns through a greedy strategy on the similarity matrix formed by matching frames in each cell, then aggregates them into complete predicted copy segments. On the comprehensive dataset self-SVD, VSM significantly outperforms state-of-the-art methods by 7.28% in mAP, and the retrieval speed is increased by over 318 times. Additionally, for short videos at the scale of hundreds of millions, the response speed can theoretically reach seconds. On the copy localization dataset MIX, composed of mixed-duration videos, GPD also achieves the best performance.

GPU-based MapReduce for large-scale near-duplicate video retrieval

GPU-based MapReduce for Large-Scale Near-Duplicate Video Retrieval.

Effective Multiple Feature Hashing for Large-Scale Near-Duplicate Video Retrieval

Multiple Feature Hashing for Real-Time Large Scale Near-Duplicate Video Retrieval

Million-scale near-duplicate video retrieval system.

A Framework of Video Coding for Compressing Near-Duplicate Videos

Fast And Robust Detection Of Near-Duplicates In Web Video Database

Progressive Image Retrieval with Quality Guarantee under MapReduce Framework

Joint Compression of Near-Duplicate Videos.

Near-duplicate Keyframe Retrieval by Nonrigid Image Matching.

Joint Multi-View Hashing for Large-Scale Near-Duplicate Video Retrieval

Near-duplicate Keyframe Retrieval by Semi-Supervised Learning and Nonrigid Image Matching

SVD: A Large-Scale Short Video Dataset for Near-Duplicate Video Retrieval.

Learn from Unlabeled Videos for Near-duplicate Video Retrieval

CNN Retrieval based Unsupervised Metric Learning for Near-Duplicated Video Retrieval

GPU-Accelerated Video Copy Detection Based on Incremental Clustering

Mining Near-Duplicate Graph for Cluster-Based Reranking of Web Video Search Results

Video Organization: Near-Duplicate Video Clustering.

Near-Duplicate Video Retrieval Through Toeplitz Kernel Partial Least Squares.

Distributed Video Processing Platform Based on Map Reduce

An Efficient Partial Video Copy Detection for a Large-scale Video Database