Abstract:The manner of walking (gait) is a powerful biometric that is used as a unique fingerprinting method, allowing unobtrusive behavioral analytics to be performed at a distance without subject cooperation. As opposed to more traditional biometric authentication methods, gait analysis does not require explicit cooperation of the subject and can be performed in low-resolution settings, without requiring the subject's face to be unobstructed/clearly visible. Most current approaches are developed in a controlled setting, with clean, gold-standard annotated data, which powered the development of neural architectures for recognition and classification. Only recently has gait analysis ventured into using more diverse, large-scale, and realistic datasets to pretrained networks in a self-supervised manner. Self-supervised training regime enables learning diverse and robust gait representations without expensive manual human annotations. Prompted by the ubiquitous use of the transformer model in all areas of deep learning, including computer vision, in this work, we explore the use of five different vision transformer architectures directly applied to self-supervised gait recognition. We adapt and pretrain the simple ViT, CaiT, CrossFormer, Token2Token, and TwinsSVT on two different large-scale gait datasets: GREW and DenseGait. We provide extensive results for zero-shot and fine-tuning on two benchmark gait recognition datasets, CASIA-B and FVG, and explore the relationship between the amount of spatial and temporal gait information used by the visual transformer. Our results show that in designing transformer models for processing motion, using a hierarchical approach (i.e., CrossFormer models) on finer-grained movement fairs comparatively better than previous whole-skeleton approaches.

Transfer Learning with Self-Supervised Vision Transformers for Snake Identification

Multi-Label Plant Species Classification with Self-Supervised Vision Transformers

SLViT: Scale-Wise Language-Guided Vision Transformer for Referring Image Segmentation.

Fish Detection and Classification Based on Improved ViT

Watch out Venomous Snake Species: A Solution to SnakeCLEF2023.

Solutions for Fine-grained and Long-tailed Snake Species Recognition in SnakeCLEF 2022

Self-Supervised Vision Transformers Learn Visual Concepts in Histopathology

Analyzing Local Representations of Self-supervised Vision Transformers

Monocular Robot Navigation with Self-Supervised Pretrained Vision Transformers

HL-ESViT: High-Low Frequency Efficient Spiking Vision Transformer

SERE: Exploring Feature Self-Relation for Self-Supervised Transformer

Animal Species Recognition with Deep Convolutional Neural Networks from Ecological Camera Trap Images

PDiscoFormer: Relaxing Part Discovery Constraints with Vision Transformers

Self-Supervised Learning with Swin Transformers

DenseDINO: Boosting Dense Self-Supervised Learning with Token-Based Point-Level Consistency.

Exploring Self-Supervised Vision Transformers for Deepfake Detection: A Comparative Analysis

Exploring Self-Supervised Vision Transformers for Gait Recognition in the Wild

A convolutional vision transformer for semantic segmentation of side-scan sonar data

Deep Snake for Real-Time Instance Segmentation

Deep Reinforcement Learning with Swin Transformers

Analyzing Vision Transformers for Image Classification in Class Embedding Space