Abstract:The performance of action recognition in video sequences depends significantly on the representation of actions and the similarity measurement between the representations. In this paper, we combine two kinds of features extracted from the spatio-temporal interest points with context-aware kernels for action recognition. For the action representation, local cuboid features extracted around interest points are very popular using a Bag of Visual Words (BOVW) model. Such representations, however, ignore potentially valuable information about the global spatio-temporal distribution of interest points. We propose a new global feature to capture the detailed geometrical distribution of interest points. It is calculated by using the 3D ℛ transform which is defined as an extended 3D discrete Radon transform, followed by the application of a two-directional two-dimensional principal component analysis. For the similarity measurement, we model a video set as an optimized probabilistic hypergraph and propose a context-aware kernel to measure high order relationships among videos. The context-aware kernel is more robust to the noise and outliers in the data than the traditional context-free kernel which just considers the pairwise relationships between videos. The hyperedges of the hypergraph are constructed based on a learnt Mahalanobis distance metric. Any disturbing information from other classes is excluded from each hyperedge. Finally, a multiple kernel learning algorithm is designed by integrating the l_2 norm regularization into a linear SVM classifier to fuse the ℛ feature and the BOVW representation for action recognition. Experimental results on several datasets demonstrate the effectiveness of the proposed approach for action recognition.

The Feature and Spatial Covariant Kernel

KPB-SIFT

Video object matching across multiple non-overlapping camera views based on multi-feature fusion and incremental learning.

A unified feature-spatial cycle consistency fusion framework for robust image matching

Efficient Covisibility-based Image Matching for Large-Scale SfM

An image representation method based on the similarity of feature points

Fusing $${\mathcal {R}}$$R Features and Local Features with Context-Aware Kernels for Action Recognition

Adaptively feature matching via joint transformational-spatial clustering

A Scale Adaptive Kernel Correlation Filter Tracker With Feature Integration

Feature-matching method based on keypoint response constraint using binary encoding of phase congruency

A Novel Cnn-Based Match Kernel For Image Retrieval

Spatial Neighborhood-Constrained Linear Coding for Visual Object Tracking

Descriptor Ensemble: An Unsupervised Approach to Descriptor Fusion in the Homography Space

Local coding based matching kernel method for image classification

Image Classification with Kernelized Spatial-Context.

More About Covariance Descriptors for Image Set Coding: Log-Euclidean Framework based Kernel Matrix Representation

Linear spatial pyramid matching using sparse coding for image classification

Incremetal Spatio-Temporal Feature Extraction and Retrieval for Large Video Database

Kernel Spectral Curvature Clustering (KSCC)

Fast and Robust Spatial Matching for Object Retrieval

Adaptive Feature-Spatial Representation for Mean-shift Tracker