Kernel-based representation for 2D/3D motion trajectory retrieval and classification

LinWei-Yang,HsiehChung-Yang

IF: 8

2013-01-01

Pattern Recognition

Abstract:This paper proposes a novel kernel-space representation for motion trajectories. Contrasted to most trajectory representation methods in the literature, our method is more generic in the sense that...

What problem does this paper attempt to address?

Kernel-based representation for 2D/3D motion trajectory retrieval and classification

Wei-Yang Lin,Chung-Yang Hsieh

DOI: https://doi.org/10.1016/j.patcog.2012.09.014

IF: 8

2013-01-01

Pattern Recognition

Abstract:This paper proposes a novel kernel-space representation for motion trajectories. Contrasted to most trajectory representation methods in the literature, our method is more generic in the sense that it can be applied to either 2D or 3D trajectories. In the proposed method, a trajectory is firstly projected by the Kernel Principal Component Analysis (KPCA) which can be considered as an implicit mapping to a much higher-dimensional feature space. The high dimensionality can effectively improve the accuracy in recognizing motion trajectories. Then, Nonparametric Discriminant Analysis (NDA) is used to extract the most discriminative features from the KPCA feature space. The synergistic effect of KPCA and NDA leads to better class separability and makes the proposed trajectory representation a more powerful discriminator. The experimental validation of the proposed method is conducted on the Australian Sign Language (ASL) data set. The results show that our method performs significantly better, in both trajectory classification and retrieval, than the state-of-the-art techniques.
On Double-Level Kernel Self-Similarities for 3D Motion Trajectory Description and Recognition

Yao Guo,Youfu Li,Zhanpeng Shao

DOI: https://doi.org/10.1109/rcar.2017.8311938

2017-01-01

Abstract:3D Spatiotemporal trajectories can provide compact and informative clues in motion analysis of human bodies, robots and moving objects. This paper proposes a new framework for 3D motion trajectory-based recognition, which can achieve satisfactory performance in both accuracy and efficiency. Motion trajectories are firstly transformed into the Double-level Kernel Self-similarity Matrices (DKSM). The DKSM representation is constructed by investigating the pair-wise kernel distance within each trajectory itself at the trajectory level and the component level respectively, which has shown strong invariance ability and descriptive power. As each matrix in the DKSM representation can be viewed as a gray-scale image, the well-proven Histogram of Oriented Gradients (HOG) descriptors extracted from the DKSM are concatenated as the final DKSM-HOG descriptor. Next, we train a SVM classifier for multiple class recognition with the training DKSM-HOG descriptors as the input. Finally, extensive motion trajectory recognition experiments are conducted on two public datasets to demonstrate the effectiveness of the proposed method.
Distributional Kernel: an Effective and Efficient Means for Trajectory Retrieval

Yuanyi Shang,Kai Ming Ting,Zijing Wang,Yufan Wang

DOI: https://doi.org/10.1007/978-981-97-2262-4_22

2024-01-01

Abstract:In this paper, we propose a new and powerful way to represent trajectories and measure the distance between them using a distributional kernel. Our method has two unique properties: (i) the identity property which ensures that dissimilar trajectories have no short distances, and (ii) a runtime orders of magnitude faster than that of existing distance measures. An extensive evaluation on several large real-world trajectory datasets confirms that our method is more effective and efficient in trajectory retrieval tasks than traditional and deep learning-based distance measures.
A new descriptor for 3D trajectory recognition via modified CDTW

Jianyu Yang,Youfu Li,Keyi Wang

DOI: https://doi.org/10.1109/ICAL.2010.5585379

2010-01-01

Abstract:Motion recognition via trajectory is important in motion analysis for many robotic tasks. A descriptor for motion trajectories plays an important role in the recognition algorithm. Existing invariant descriptors and discrete matching algorithms are not flexible enough under certain circumstances for recognition. In this paper we propose a new descriptor to solve this problem, with a modified data alignment method in the matching algorithm for motion recognition. Experimental results demonstrate the effectiveness of the method in the motion recognition.
A Principled Distributional Approach to Trajectory Similarity Measurement and its Application to Anomaly Detection

Yufan Wang,Zijing Wang,Kai Ming Ting,Yuanyi Shang

DOI: https://doi.org/10.1613/jair.1.15849

2024-03-14

Journal of Artificial Intelligence Research

Abstract:This paper aims to solve two enduring challenges in existing trajectory similarity measures: computational inefficiency and the absence of the 'uniqueness' property that should be guaranteed in a distance function: dist(X, Y ) = 0 if and only if X = Y , where X and Y are two trajectories. In this work, we present a novel approach utilizing a distributional kernel for trajectory representation and similarity measurement, based on the kernel mean embedding framework. It is the very first time a distributional kernel is used for trajectory representation and similarity measurement. Our method does not rely on point-to-point distances which are used in most existing distances for trajectories. Unlike prevalent learning and deep learning approaches, our method requires no learning. We show the generality of this new approach in anomalous trajectory and sub-trajectory detection. We identify that the distributional kernel has (i) a data-dependent property and the 'uniqueness' property which are the key factors that lead to its superior task-specific performance, and (ii) runtime orders of magnitude faster than existing distance measures.

computer science, artificial intelligence
Representing and Recognizing Motion Trajectories

Yang Zhou,Weiyao Lin,Hang Su,Jianxin Wu,Jinjun Wang,Yu Zhou

DOI: https://doi.org/10.1145/2647868.2654955

2014-01-01

Abstract:This paper addresses the problem of representing and recognizing motion trajectories. We first propose to derive scene-related equipotential lines for points in a motion trajectory and concatenate them to construct a 3D tube for representing the trajectory. Based on this 3D tube, a droplet-based method is further proposed which derives a "water droplet" from the 3D tube and recognizes trajectory activities accordingly. Our proposed 3D tube can effectively embed both motion and scene-related information of a motion trajectory while the proposed droplet- based method can suitably catch the characteristics of the 3D tube for activity recognition. Experimental results demonstrate the effectiveness of our approach.
A Tube-and-Droplet-based Approach for Representing and Analyzing Motion Trajectories

Weiyao Lin,Yang Zhou,Hongteng Xu,Junchi Yan,Mingliang Xu,Jianxin Wu,Zicheng Liu

DOI: https://doi.org/10.1109/TPAMI.2016.2608884

2017-04-26

Abstract:Trajectory analysis is essential in many applications. In this paper, we address the problem of representing motion trajectories in a highly informative way, and consequently utilize it for analyzing trajectories. Our approach first leverages the complete information from given trajectories to construct a thermal transfer field which provides a context-rich way to describe the global motion pattern in a scene. Then, a 3D tube is derived which depicts an input trajectory by integrating its surrounding motion patterns contained in the thermal transfer field. The 3D tube effectively: 1) maintains the movement information of a trajectory, 2) embeds the complete contextual motion pattern around a trajectory, 3) visualizes information about a trajectory in a clear and unified way. We further introduce a droplet-based process. It derives a droplet vector from a 3D tube, so as to characterize the high-dimensional 3D tube information in a simple but effective way. Finally, we apply our tube-and-droplet representation to trajectory analysis applications including trajectory clustering, trajectory classification & abnormality detection, and 3D action recognition. Experimental comparisons with state-of-the-art algorithms demonstrate the effectiveness of our approach.

Computer Vision and Pattern Recognition,Artificial Intelligence,Multimedia
Point Context: An Effective Shape Descriptor for RST-Invariant Trajectory Recognition

Xingyu Wu,Xia Mao,Lijiang Chen,Yuli Xue,Alberto Rovetta

DOI: https://doi.org/10.1007/s10851-016-0648-6

IF: 1.627

2016-04-04

Journal of Mathematical Imaging and Vision

Abstract:Motion trajectory recognition is important for characterizing the moving property of an object. The speed and accuracy of trajectory recognition rely on a compact and discriminative feature representation, and the situations of varying rotation, scaling, and translation have to be specially considered. In this paper, we propose a novel feature extraction method for trajectories. Firstly, a trajectory is represented by a proposed point context, which is a rotation-scale-translation invariant shape descriptor with a flexible tradeoff between the complexity and discrimination, yet we prove that it is a complete shape descriptor. Secondly, the point context is nonlinearly mapped to a subspace by kernel nonparametric discriminant analysis to get a compact feature representation, and thus a trajectory is projected to a low-dimensional feature space. Experimental results show that the proposed trajectory feature demonstrates encouraging improvement than state-of-the-art methods.

mathematics, applied,computer science, software engineering, artificial intelligence
Describing Local Reference Frames for 3-D Motion Trajectory Recognition

Zhanpeng Shao,Youfu Li,Yao Guo,Xiaolong Zhou

DOI: https://doi.org/10.1109/access.2018.2849690

IF: 3.9

2018-01-01

IEEE Access

Abstract:Motion trajectories tracked from the points of interest can provide the key relevant features for characterizing the motion patterns in video. As the increasing number of 3-D vision sensors rises, the 3-D motion trajectories that serve as motion representations have been applied successfully to video retrieval and analysis, scene understanding, motion recognition, and so on, in existing works. Most of these works use raw data of motion trajectories directly or draw simple geometric quantities to describe the motion trajectories, whereas these simple descriptions are not intrinsically complete as they cannot feature the orientation changes of moving points along the 3-D motion trajectories. In principle, orientation changes of a single moving point in 3-D space have to been obtained by resorting to high-order derivatives, but the high-order derivatives would result in high sensitivity to noise. This paper tackles the problem by describing the local reference frames along 3-D motion trajectories, while we consider a motion trajectory as a temporal sequence of local reference frames. The maximal blurred segment of the noisy discrete curves is employed to estimate the local reference frames without high-order derivatives involved, and the local reference frame contains complete information of positions and orientations in the 3-D Euclidean space. To describe such local reference frames, we use the rotations and local square root velocities of local reference frames as the proposed descriptor to characterize the position and orientation changes of the moving points along the motion trajectories. In the experiments, we evaluate the effectiveness of the proposed descriptor by applying it to the gesture recognition on two large benchmark data sets that contain hand motion trajectories. The results show that our proposed descriptor can achieve superior performance compared to the existing descriptors and state-of-the-art methods in the 3 D motion trajectory recognition.
Trajectory-Based Modeling Of Human Actions With Motion Reference Points

Yu-Gang Jiang,Qi Dai,Xiangyang Xue,Wei Liu,Chong-Wah Ngo

DOI: https://doi.org/10.1007/978-3-642-33715-4_31

2012-01-01

Abstract:Human action recognition in videos is a challenging problem with wide applications. State-of-the-art approaches often adopt the popular bag-of-features representation based on isolated local patches or temporal patch trajectories, where motion patterns like object relationships are mostly discarded. This paper proposes a simple representation specifically aimed at the modeling of such motion relationships. We adopt global and local reference points to characterize motion information, so that the final representation can be robust to camera movement. Our approach operates on top of visual codewords derived from local patch trajectories, and therefore does not require accurate foreground-background separation, which is typically a necessary step to model object relationships. Through an extensive experimental evaluation, we show that the proposed representation offers very competitive performance on challenging benchmark datasets, and combining it with the bag-of-features representation leads to substantial improvement. On Hollywood2, Olympic Sports, and HMDB51 datasets, we obtain 59.5%, 80.6% and 40.7% respectively, which are the best reported results to date.
Invariant Trajectory Indexing for Real Time 3D Motion Recognition

Jianyu Yang,Y. F. Li,Keyi Wang

DOI: https://doi.org/10.1109/robio.2011.6181595

2011-01-01

Abstract:Motion trajectory description plays an important role in motion recognition for many robotic tasks. The previous methods are too high in computational cost for real time recognition. In this paper, we propose a novel indexing method for motion trajectories as a high level invariant descriptor. Trajectories are segmented into basic segments and represented in segment level rather than point level. We index the trajectories by their segment sequences to recognize them. The computational cost is significantly decreased and the experimental results show that this method is effective and it can be used for real time motion recognition. The accuracy is also preserved for long term trajectories.
Trajectory Space: A Dual Representation for Nonrigid Structure from Motion

Ijaz Akhter,Yaser Sheikh,Sohaib Khan,Takeo Kanade

DOI: https://doi.org/10.1109/TPAMI.2010.201

Abstract:Existing approaches to nonrigid structure from motion assume that the instantaneous 3D shape of a deforming object is a linear combination of basis shapes. These bases are object dependent and therefore have to be estimated anew for each video sequence. In contrast, we propose a dual approach to describe the evolving 3D structure in trajectory space by a linear combination of basis trajectories. We describe the dual relationship between the two approaches, showing that they both have equal power for representing 3D structure. We further show that the temporal smoothness in 3D trajectories alone can be used for recovering nonrigid structure from a moving camera. The principal advantage of expressing deforming 3D structure in trajectory space is that we can define an object independent basis. This results in a significant reduction in unknowns and corresponding stability in estimation. We propose the use of the Discrete Cosine Transform (DCT) as the object independent basis and empirically demonstrate that it approaches Principal Component Analysis (PCA) for natural motions. We report the performance of the proposed method, quantitatively using motion capture data, and qualitatively on several video sequences exhibiting nonrigid motions, including piecewise rigid motion, partially nonrigid motion (such as a facial expressions), and highly nonrigid motion (such as a person walking or dancing).
On Multiscale Self-Similarities Description for Effective Three-Dimensional/Six-Dimensional Motion Trajectory Recognition

Yao Guo,You-Fu Li,Zhanpeng Shao

DOI: https://doi.org/10.1109/tii.2017.2751072

IF: 12.3

2017-01-01

IEEE Transactions on Industrial Informatics

Abstract:Motion trajectories provide compact informative clues in characterizingmotion behaviors of human bodies, robots, and moving objects. This paper devises an invariant and unified descriptor for three-dimensional/six-dimensional (3-D/6-D) motion trajectories recognition by exploring the latent motion patterns in the multiscale self-similarity matrices (MSM) within a motion trajectory and its components. The MSM approach transforms a motion trajectory in Euclidean space into a set of similarity matrices and exhibits strong invariances, in which each matrix can be regarded as a grayscale image. Next, the histograms of oriented gradients features extracted from the MSM representation are concatenated as the final trajectory descriptor. In addition, an improved kernel MSM is raised by calculating the pairwise kernel distances. Finally, extensive 3-D/6-D motion trajectory recognition experiments on three public datasets with a linear support vector machine classifier are conducted to verify the effectiveness and efficiency of the proposed approach.
A COMPACT PAIRWISE TRAJECTORY REPRESENTATION FOR ACTION RECOGNITION

Qingya Huang,Shan Sun,Feng Wang

DOI: https://doi.org/10.1109/icassp.2017.7952460

2017-01-01

Abstract:Dense trajectories are widely used in human action recognition. However, the relationships among trajectories are rarely exploited and a large mount of useful information is missing. In this paper, we propose a novel approach to employ the space-time relationships between different trajectories for action recognition. In our approach, each trajectory is paired up with several neighbors which are spatially and temporally close to it. A GMM (Gaussian Mixture Model) is then trained and Fisher Vector is employed to quantify the pairwise trajectories. In this way, the local spatial and temporal structure information around each trajectory is explored for feature representation, which improves the discriminative ability of the features. The experimental results on several benchmark datasets show that our pairwise trajectory representation outperforms the state-of-the-art approaches.
Mixed Signature Descriptor with Global Invariants for 3D Motion Trajectory Perception and Recognition

Jianyu Yang,Li, Y.F.,Keyi Wang

DOI: https://doi.org/10.1109/ieem.2010.5674635

2010-01-01

Abstract:Motion perception and recognition by trajectory are important for motion analysis. A flexible and adaptable descriptor of motion trajectory plays important role in the performance of perception and recognition. Motion trajectories usually use raw data for many tasks, which is not effective. Shape descriptors and signature of differential invariants are good for local features but global feature is lacking. In this paper, we present a mixed signature descriptor with global invariants which were not included in previous descriptors. The mixed signature is capable of capturing the global feature and is flexible in many tasks. The conducted experiments verified the capability of our method.
A Study of Relative Motion Point Trajectories for Action Recognition

Tongchi Zhou,Nijun Li,Xu Cheng,Qinjun Xu,Lin Zhou,Zhenyang Wu

DOI: https://doi.org/10.1109/wcsp.2015.7340988

2015-01-01

Abstract:Trajectories extracted by previous methods for human action recognition contain irrelevant changes, and the Orientation-Magnitude descriptors of their shapes lack the robustness to camera motion. To solve these problems, action recognition by tracking salient relative motion points is proposed in this paper. Firstly, motion boundary detector which suppresses the camera constant motion is utilized to extract motion features. After processing the detected boundaries by the adaptive threshold, the super-pixels that contain salient points are defined as relative motion regions. Then tracking the points within super-pixels is to generate trajectories. For the trajectory shape, the pre-defined orientation assignments with coarse-to fine quantization levels are used to produce orientation statistics. Finally, the descriptors of oriented gradient, motion boundary, oriented statistic and their combination are adopted to represent action videos, respectively. On the benchmark KTH and UCF-sports action datasets, experimental results show that the extracted trajectories can describe the movement process of object. Compared with the conventional algorithms, our method with multiple kernel learning obtains good performance.
Integrating Graph Partitioning and Matching for Trajectory Analysis in Video Surveillance

Liang Lin,Yongyi Lu,Yan Pan,Xiaowu Chen

DOI: https://doi.org/10.48550/arXiv.1502.00377

2015-02-02

Computer Vision and Pattern Recognition

Abstract:In order to track the moving objects in long range against occlusion, interruption, and background clutter, this paper proposes a unified approach for global trajectory analysis. Instead of the traditional frame-by-frame tracking, our method recovers target trajectories based on a short sequence of video frames, e.g. $15$ frames. We initially calculate a foreground map at each frame, as obtained from a state-of-the-art background model. An attribute graph is then extracted from the foreground map, where the graph vertices are image primitives represented by the composite features. With this graph representation, we pose trajectory analysis as a joint task of spatial graph partitioning and temporal graph matching. The task can be formulated by maximizing a posteriori under the Bayesian framework, in which we integrate the spatio-temporal contexts and the appearance models. The probabilistic inference is achieved by a data-driven Markov Chain Monte Carlo (MCMC) algorithm. Given a peroid of observed frames, the algorithm simulates a ergodic and aperiodic Markov Chain, and it visits a sequence of solution states in the joint space of spatial graph partitioning and temporal graph matching. In the experiments, our method is tested on several challenging videos from the public datasets of visual surveillance, and it outperforms the state-of-the-art methods.
Mixed Signature: an Invariant Descriptor for 3d Motion Trajectory Perception and Recognition

Jianyu Yang,Y. F. Li,Keyi Wang,Yuan Wu,Giuseppe Altieri,Massimo Scalia

DOI: https://doi.org/10.1155/2012/613939

IF: 1.43

2012-01-01

Mathematical Problems in Engineering

Abstract:Motion trajectory contains plentiful motion information of moving objects, for example, human gestures and robot actions. Motion perception and recognition via trajectory are useful for characterizing them and a flexible descriptor of motion trajectory plays important role in motion analysis. However, in the existing tasks, trajectories were mostly used in raw data and effective descriptor is lacking. In this paper, we present a mixed invariant signature descriptor with global invariants for motion perception and recognition. The mixed signature is viewpoint invariant for local and global features. A reliable approximation of the mixed signature is proposed to reduce the noise in high-order derivatives. We use this descriptor for motion trajectory description and explore the motion perception with DTW algorithm for salient motion features. To achieve better accuracy, we modified the CDTW algorithm for trajectory matching in motion recognition. Furthermore, a controllable weight parameter is introduced to adjust the global features for tasks in different circumstances. The conducted experiments validated the proposed method.
Representing And Recognizing Motion Trajectories: A Tube And Droplet Approach

Yang Zhou,Weiyao Lin,Hang Su,Jianxin Wu,Jinjun Wang,Yu Zhou

DOI: https://doi.org/10.1145/2647868.2654955

2014-01-01

Abstract:This paper addresses the problem of representing and recognizing motion trajectories. We first propose to derive scene-related equipotential lines for points in a motion trajectory and concatenate them to construct a 3D tube for representing the trajectory. Based on this 3D tube, a droplet-based method is further proposed which derives a "water droplet" from the 3D tube and recognizes trajectory activities accordingly. Our proposed 3D tube can effectively embed both motion and scene-related information of a motion trajectory while the proposed droplet based method can suitably catch the characteristics of the 3D tube for activity recognition. Experimental results demonstrate the effectiveness of our approach.
Multi-cue Visual Tracking Based on Sparse Representation

Xiping Duan,Jiafeng Liu,Xianglong Tang

DOI: https://doi.org/10.1007/978-3-642-42057-3_54

2017-01-01

International Journal of Control and Automation

Abstract:Recently, the sparse representation based visual tracking is very popular, which is robust to occlusion and noise, but not satisfactory in the scenarios of fast motion and blur. In order to solve this problem, a new tracking method based on kernel sparse representation is proposed. In this method, each candidate is represented by kernel sparse representation, then the computed reconstruction error is used to obtain the observation probability, and at last the candidate with the maximal observation probability is determined as the target. Aspect to the solving of kernel sparse representation, the accelerated proximal gradient (APG) is adopted. Experiments on several representative image sequences shows that the proposed tracking method performs better than the sparse representation based visual tracking method in the fast motion and blur scenarios.

Kernel-based representation for 2D/3D motion trajectory retrieval and classification

Kernel-based representation for 2D/3D motion trajectory retrieval and classification

On Double-Level Kernel Self-Similarities for 3D Motion Trajectory Description and Recognition

Distributional Kernel: an Effective and Efficient Means for Trajectory Retrieval

A new descriptor for 3D trajectory recognition via modified CDTW

A Principled Distributional Approach to Trajectory Similarity Measurement and its Application to Anomaly Detection

Representing and Recognizing Motion Trajectories

A Tube-and-Droplet-based Approach for Representing and Analyzing Motion Trajectories

Point Context: An Effective Shape Descriptor for RST-Invariant Trajectory Recognition

Describing Local Reference Frames for 3-D Motion Trajectory Recognition

Trajectory-Based Modeling Of Human Actions With Motion Reference Points

Invariant Trajectory Indexing for Real Time 3D Motion Recognition

Trajectory Space: A Dual Representation for Nonrigid Structure from Motion

On Multiscale Self-Similarities Description for Effective Three-Dimensional/Six-Dimensional Motion Trajectory Recognition

A COMPACT PAIRWISE TRAJECTORY REPRESENTATION FOR ACTION RECOGNITION

Mixed Signature Descriptor with Global Invariants for 3D Motion Trajectory Perception and Recognition

A Study of Relative Motion Point Trajectories for Action Recognition

Integrating Graph Partitioning and Matching for Trajectory Analysis in Video Surveillance

Mixed Signature: an Invariant Descriptor for 3d Motion Trajectory Perception and Recognition

Representing And Recognizing Motion Trajectories: A Tube And Droplet Approach

Multi-cue Visual Tracking Based on Sparse Representation