Tsinghua University at TRECVID 2005.
Jinhui Yuan,Huiyi Wang,Lan Xiao,Dong Wang,Dayong Ding,Yuanyuan Zuo,Zijan Tong,Xiaobing Liu,Shuping Xu,Wujie Zheng,Xirong Li,Zhangzhang Si,Jianmin Li,Fuzong Lin,Bo Zhang
2005-01-01
Abstract:Our shot boundary determination system consists of three components, including a FOI detector, a generalized CUT detector, and a long gradual transition detector. One support vector machine, taking score vector calculated with graph partition model as input, is used to detect CUT. Long gradual transition is determined by another three support vector machines with multi-resolution score vectors as input. After these detectors make decision successively, the locations of shot boundaries and the corresponding types are obtained. It is found in the experiments on development data that by tuning penalty ratio of loss of misclassifying the positive and the negative samples, it is possible to control the trade-off between precision and recall. 31 runs are generated from the same system with the 4 support vector machines being trained with different parameters. Among them, 10 runs are submitted for evaluation. And the results show that our system is among the best. In our system for low level feature extraction, some spatial features of motion vectors are proposed to select the motion vectors which describe the camera motion in deed. The four-parameter affine model is used to describe the camera motion, and the ILSE technique is used to calculate the parameters. Then camera motion will be classified into three classes: pan, tilt and zoom with an accurate classification method based on finite-state automata. Our system achieves best results in this task of TRECVID2005. Our systems for high level feature extraction rely heavily on the visual information. Visual features include Color Auto-Correlograms, Color Coherence Vector, Color Histogram, Color Moment, Edge Histogram and Wavelet Texture. Two different systems using regional and global image features are compared to explore the effectiveness of regional features. In the regional system, keyframes are segmented and regional feature of all the six types mentioned above are extracted. Then support vector machine classifier with Earth Mover Distance (EMD) kernel is built. In the global system, the six types of global feature are extracted for each keyframe directly. Then the classifier ensembles for detecting each concept are formed by using the Relay Boost algorithm. This is followed by a concept context module. We tried mainly two approaches, one based on stacked SVM and the other based on weighted sum of the confidence scores of the related concepts. We then apply time clustered post-filtering to remove false positive shots. Based on these two systems we have our 7 runs. From the results, we find that multi-feature fusion improves over any single modality significantly. Our automatic video search systems have three basic retrieval models: a text model based on script generated by ASR, an image model based on region-based image matching and a concept model which automatically parses the queries and video shots into concept vectors, and then searches video shots through query-shot similarity computing in concept space. Based on these models, we also develop some combination systems. In the score fusing system, the results are ranked by fusing the scores generated from the basic retrieval models. In the fusing system based on query type, queries are classified into two classes, and then retrieved using different models. Among our 7 submissions, the results show that when searching for general topics, which are always less related to person, combining text and concept models performs better than only using text model.