Supervised Topic Models for Video Activity Recognition

M. Hughes

Abstract:Topic models successfully capture latent structure useful for unsupervised analysis of bag-of-words data. Applying these models to domains such as video activity recognition requires two critical extensions: (1) incorporating supervised information (activity labels) to recover topic structure with greater discriminative power and (2) moving beyond the bag-of-words assumption to model temporal dynamics. We propose two parallel investigations to accomplish these tasks. First, we will study generic supervision techniques for topic models, exposing shortcomings in previously published generative approaches and exploring new discriminative models based on Mixtures of Experts. Second, we will apply these supervised models to video activity classification on the challenging Hollywood2 and Olympic Sports datasets, and explore extensions that capture chronological structure inherent in real-world activities.