Abstract:Nowadays, with the rapid development of consumer Unmanned Aerial Vehicles (UAVs), utilizing UAV platforms for visual surveillance has become very attractive, and a key part of this is remote vision-based pedestrian attribute recognition. Pedestrian Attribute Recognition (PAR) is dedicated to predicting multiple attribute labels of a single pedestrian image extracted from surveillance videos and aerial imagery, which presents significant challenges in the computer vision community due to factors such as poor imaging quality and substantial pose variations. Despite recent studies demonstrating impressive advancements in utilizing complicated architectures and exploring relations, most of them may fail to fully and systematically consider the inter-region, inter-attribute, and region-attribute mapping relations simultaneously and be stuck in the dilemma of information redundancy, leading to the degradation of recognition accuracy. To address the issues, we construct a novel Mask-Relation-Guided Transformer (MRG-T) framework that consists of three relation modeling modules to fully exploit spatial and semantic relations in the model learning process. Specifically, we first propose a Masked Region Relation Module (MRRM) to focus on precise spatial attention regions to extract more robust features with masked random patch training. To explore the semantic association of attributes, we further present a Masked Attribute Relation Module (MARM) to extract intrinsic and semantic inter-attribute relations with an attribute label masking strategy. Based on the cross-attention mechanism, we finally design a Region and Attribute Mapping Module (RAMM) to learn the cross-modal alignment between spatial regions and semantic attributes. We conduct comprehensive experiments on three public benchmarks such as PETA, PA-100K, and RAPv1, and conduct inference on a large-scale airborne person dataset named PRAI-1581. The extensive experimental results demonstrate the superior performance of our method compared to state-of-the-art approaches and validate the effectiveness of mask-relation-guided modeling in the remote vision-based PAR task.

Non-Hierarchical Transformers for Pedestrian Segmentation

Unifying Terrain Awareness Through Real-Time Semantic Segmentation

Learning Spatiotemporal Relationships with a Unified Framework for Video Object Segmentation

TransVOS: Video Object Segmentation with Transformers

SDPT: Semantic-Aware Dimension-Pooling Transformer for Image Segmentation

DSViT: Dynamically Scalable Vision Transformer for Remote Sensing Image Segmentation and Classification.

Multidirection and Multiscale Pyramid in Transformer for Video-Based Pedestrian Retrieval

Trans4Trans: Efficient Transformer for Transparent Object Segmentation to Help Visually Impaired People Navigate in the Real World

Human Body Segmentation in Wide-Angle Images Based on Fast Vision Transformers

MRG-T: Mask-Relation-Guided Transformer for Remote Vision-Based Pedestrian Attribute Recognition in Aerial Imagery

A Bio-Inspired Visual Perception Transformer for Cross-Domain Semantic Segmentation of High-Resolution Remote Sensing Images

DAT++: Spatially Dynamic Vision Transformer with Deformable Attention

Trans4Trans: Efficient Transformer for Transparent Object and Semantic Scene Segmentation in Real-World Navigation Assistance

P2AT: Pyramid Pooling Axial Transformer for Real-time Semantic Segmentation

Scalable Vision Transformers with Hierarchical Pooling.

Multi-direction and Multi-scale Pyramid in Transformer for Video-based Pedestrian Retrieval

Image segmentation using Vision Transformer for tunnel defect assessment

AdaViT: Adaptive Vision Transformers for Efficient Image Recognition

Revolutionizing Traffic Sign Recognition: Unveiling the Potential of Vision Transformers

Representation Separation for Semantic Segmentation with Vision Transformers

HiViT: Hierarchical Vision Transformer Meets Masked Image Modeling