Abstract:Current few-shot action recognition involves two primary sources of information for classification:(1) intra-video information, determined by frame content within a single video clip, and (2) inter-video information, measured by relationships (e.g., feature similarity) among videos. However, existing methods inadequately exploit these two information sources. In terms of intra-video information, current sampling operations for input videos may omit critical action information, reducing the utilization efficiency of video data. For the inter-video information, the action misalignment among videos makes it challenging to calculate precise relationships. Moreover, how to jointly consider both inter- and intra-video information remains under-explored for few-shot action recognition. To this end, we propose a novel framework, Video Information Maximization (VIM), for few-shot video action recognition. VIM is equipped with an adaptive spatial-temporal video sampler and a spatiotemporal action alignment model to maximize intra- and inter-video information, respectively. The video sampler adaptively selects important frames and amplifies critical spatial regions for each input video based on the task at hand. This preserves and emphasizes informative parts of video clips while eliminating interference at the data level. The alignment model performs temporal and spatial action alignment sequentially at the feature level, leading to more precise measurements of inter-video similarity. Finally, These goals are facilitated by incorporating additional loss terms based on mutual information measurement. Consequently, VIM acts to maximize the distinctiveness of video information from limited video data. Extensive experimental results on public datasets for few-shot action recognition demonstrate the effectiveness and benefits of our framework.

HyRSM++: Hybrid relation guided temporal set matching for few-shot action recognition

Hybrid Relation Guided Set Matching for Few-shot Action Recognition

Revisiting the Spatial and Temporal Modeling for Few-shot Action Recognition

Boosting Few-shot Action Recognition with Graph-guided Hybrid Matching

Hybrid Attentive Prototypical Network for Few-Shot Action Recognition

Two-stream joint matching method based on contrastive learning for few-shot action recognition

On the Importance of Spatial Relations for Few-shot Action Recognition

TA2N: Two-Stage Action Alignment Network for Few-Shot Action Recognition.

TTAN: Two-Stage Temporal Alignment Network for Few-shot Action Recognition

Few-shot action recognition with implicit temporal alignment and pair similarity optimization

Task-Specific Alignment and Multiple Level Transformer for Few-Shot Action Recognition

Mask Guided Two-Stream Network for End-to-end Few-Shot Action Recognition

Task-Agnostic Self-Distillation for Few-Shot Action Recognition

SMAM: Self and Mutual Adaptive Matching for Skeleton-Based Few-Shot Action Recognition

Task-specific alignment and multiple-level transformer for few-shot action recognition

Matching Compound Prototypes for Few-Shot Action Recognition

MoLo: Motion-augmented Long-short Contrastive Learning for Few-shot Action Recognition

Hierarchical Temporal Memory Enhanced One-Shot Distance Learning for Action Recognition

Few-shot Action Recognition via Intra- and Inter-Video Information Maximization

Task-adaptive Spatial-Temporal Video Sampler for Few-shot Action Recognition

Hierarchical compositional representations for few-shot action recognition