Abstract:Graph convolutional networks are widely used in skeleton-based action recognition because of their good fitting ability to non-Euclidean data. While conventional multi-scale temporal convolution uses several fixed-size convolution kernels or dilation rates at each layer of the network, we argue that different layers and datasets require different receptive fields. We use multi-scale adaptive convolution kernels and dilation rates to optimize traditional multi-scale temporal convolution with a simple and effective self attention mechanism, allowing different network layers to adaptively select convolution kernels of different sizes and dilation rates instead of being fixed and unchanged. Besides, the effective receptive field of the simple residual connection is not large, and there is a great deal of redundancy in the deep residual network, which will lead to the loss of context when aggregating spatio-temporal information. This article introduces a feature fusion mechanism that replaces the residual connection between initial features and temporal module outputs, effectively solving the problems of context aggregation and initial feature fusion. We propose a multi-modality adaptive feature fusion framework (MMAFF) to simultaneously increase the receptive field in both spatial and temporal dimensions. Concretely, we input the features extracted by the spatial module into the adaptive temporal fusion module to simultaneously extract multi-scale skeleton features in both spatial and temporal parts. In addition, based on the current multi-stream approach, we use the limb stream to uniformly process correlated data from multiple modalities. Extensive experiments show that our model obtains competitive results with state-of-the-art methods on the NTU-RGB+D 60 and NTU-RGB+D 120 datasets.

MSB-FCN: Multi-Scale Bidirectional FCN for Object Skeleton Extraction.

DeepSkeleton: Learning Multi-task Scale-associated Deep Side Outputs for Object Skeleton Extraction in Natural Images

Hi-Fi: Hierarchical Feature Integration for Skeleton Detection

Multi-scale Matching Networks for Semantic Correspondence

Automatic Feature Extraction Based Structure Decomposition Method for Multi-Classification

Multi-Stage Attention-Enhanced Sparse Graph Convolutional Network for Skeleton-Based Action Recognition

Enhanced Multi-Scale Feature Adaptive Fusion Sparse Convolutional Network for Large-Scale Scenes Semantic Segmentation

Channel attention and multi-scale graph neural networks for skeleton-based action recognition

Adaptive Linear Span Network for Object Skeleton Detection

MSF-CSPNet: A Specially Designed Backbone Network for Faster R-CNN

Multi-Scale Spatial Temporal Graph Convolutional Network for Skeleton-Based Action Recognition

Multi-Modality Adaptive Feature Fusion Graph Convolutional Network for Skeleton-Based Action Recognition

MDCN: Multi-Scale, Deep Inception Convolutional Neural Networks for Efficient Object Detection

Lightweight Multi-Scale Spatiotemporal Graph Convolutional Network for Skeleton-Based Action Recognition

Deep-Aligned Convolutional Neural Network for Skeleton-Based Action Recognition and Segmentation

Multi-scale coupled attention for visual object detection

MRMNet: Multi-scale residual multi-branch neural network for object detection

SBN: Scale Balance Network for Accurate Salient Object Detection

Skeleton-based action recognition by part-aware graph convolutional networks

MM-FPN: Multi-path and Multi-scale Feature Pyramid Network for Object Detection

Full-Resolution Encoder-Decoder Networks with Multi-Scale Feature Fusion for Human Pose Estimation