Abstract:Recognition of activities in the video is an important field in computer vision. Many successful works have been done on activity recognition and they achieved acceptable results in recent years. However, their training is completely static, meaning that all classes are taught to the system in one training step. The system is only able to recognize the equivalent classes. The main disadvantage of this type of training is that if new classes need to be taught to the system, the system must be retrained from scratch and all classes retaught to the system. This specification has many challenges, such as storing and retaining data and respending training costs. We propose an approach for training the action recognition system in video data which can teach new classes to the system without the need for previous data. We will provide an incremental learning algorithm for class recognition tasks in video data. Two different approaches are combined to prevent catastrophic forgetting in the proposed algorithm. In the proposed incremental learning algorithm, two approaches are introduced and used to maintain network information in combination. These two approaches are network sharing and network knowledge distillation. We introduce a neural network architecture for action recognition to understand and represent the video data. We propose the distillation of network knowledge at the classification and feature level, which can be divided into spatial and temporal parts at the feature level. We also suggest initializing new classifiers using previous classifiers. The proposed algorithm is evaluated on the USCF101, HMDB51, and Kinetics-400 datasets. We will consider various factors such as the amount of distillation knowledge, the number of new classes and the incremental learnings stages, and their impact on the final recognition system. Finally, we will show that the proposed algorithm can teach new classes to the recognition system without forgetting the previous classes and does not need the previous data or exemplar data.

Class-Incremental Learning with Multiscale Distillation for Weakly Supervised Temporal Action Localization.

Unsupervised Temporal Action Localization Via Self-paced Incremental Learning

When Video Classification Meets Incremental Classes

Class-Incremental Learning for Action Recognition in Videos

Adaptive Mutual Supervision for Weakly-Supervised Temporal Action Localization

Transferable Knowledge-Based Multi-Granularity Fusion Network for Weakly Supervised Temporal Action Detection

Weakly-Supervised Temporal Action Localization by Progressive Complementary Learning

Multilevel Semantic and Adaptive Actionness Learning for Weakly Supervised Temporal Action Localization

Weakly-Supervised Temporal Action Localization with Bidirectional Semantic Consistency Constraint

Class-Incremental Learning for SAR Muti-Class Target Detection

Class-Incremental Learning on Video-Based Action Recognition by Distillation of Various Knowledge

Temporal Action Detection with Dynamic Weights Based on Curriculum Learning

Multi-Hierarchical Category Supervision for Weakly-Supervised Temporal Action Localization

A Class Incremental Learning Method with Weight Re-Initialization and Multiple Distillation for Image Classification

Weakly Supervised Temporal Action Localization With Bidirectional Semantic Consistency Constraint

Class Incremental Learning with Deep Contrastive Learning and Attention Distillation

MCF-VC: Mitigate Catastrophic Forgetting in Class-Incremental Learning for Multimodal Video Captioning

Multi-Dimensional Attention with Similarity Constraint for Weakly-Supervised Temporal Action Localization

Adaptive Prototype Learning for Weakly-supervised Temporal Action Localization

Class incremental learning of remote sensing images based on class similarity distillation

Dual-Feature Enhancement for Weakly Supervised Temporal Action Localization