Temporal Matching Kernel with Embedded Stability-Sensitive Filter

Fan Yang,Sebastien Poullot,Shin'ichi Satoh

DOI: https://doi.org/10.1109/ism.2017.48

2017-01-01

Abstract:This paper presents a method that embeds stability sensitive filter in the temporal matching kernel with explicit feature maps (TE). The added filter improves the robustness of TE to noise for content-based video retrieval. Originally TE embeds temporal information of frame descriptors by using explicit feature mapping in a fixed length video vector by using a temporal invariant match kernel. TE matches relevant videos and at the same time estimates the temporal offset by referring only to these vectors. It has achieved state-of-the-art performance in video event retrieval. However, since TE defines video similarity as a sum of frame-wise similarities, we argue that TE is not so robust to noise. Usually matching videos have consecutive frames in correspondence, which result in series of high similarities. A simple sum of frame-wise similarities cannot differentiate this situation from that of non-consecutive high similarities which seems to lead to false matches. To circumvent this problem, we propose to apply a non-linear low-pass filter to the frame-wise similarities. In order to enjoy the benefit of the TE framework, we further develop a technique to embed the filter into frame descriptors. Consequently the video similarity can be evaluated as the dot product of video vectors with a stability-sensitive filter enabled. As the original TE, our method is seamlessly compatible with query expansion with temporal consistency checking. Our approach is evaluated on the EVVE dataset for particular event retrieval, it achieves state-of-the-art performance.