Abstract:Introduction: This paper presents an innovative Intelligent Robot Sports Competition Tactical Analysis Model that leverages multimodal perception to tackle the pressing challenge of analyzing opponent tactics in sports competitions. The current landscape of sports competition analysis necessitates a comprehensive understanding of opponent strategies. However, traditional methods are often constrained to a single data source or modality, limiting their ability to capture the intricate details of opponent tactics. Methods: Our system integrates the Swin Transformer and CLIP models, harnessing cross-modal transfer learning to enable a holistic observation and analysis of opponent tactics. The Swin Transformer is employed to acquire knowledge about opponent action postures and behavioral patterns in basketball or football games, while the CLIP model enhances the system's comprehension of opponent tactical information by establishing semantic associations between images and text. To address potential imbalances and biases between these models, we introduce a cross-modal transfer learning technique that mitigates modal bias issues, thereby enhancing the model's generalization performance on multimodal data. Results: Through cross-modal transfer learning, tactical information learned from images by the Swin Transformer is effectively transferred to the CLIP model, providing coaches and athletes with comprehensive tactical insights. Our method is rigorously tested and validated using Sport UV, Sports-1M, HMDB51, and NPU RGB+D datasets. Experimental results demonstrate the system's impressive performance in terms of prediction accuracy, stability, training time, inference time, number of parameters, and computational complexity. Notably, the system outperforms other models, with a remarkable 8.47% lower prediction error (MAE) on the Kinetics dataset, accompanied by a 72.86-second reduction in training time. Discussion: The presented system proves to be highly suitable for real-time sports competition assistance and analysis, offering a novel and effective approach for an Intelligent Robot Sports Competition Tactical Analysis Model that maximizes the potential of multimodal perception technology. By harnessing the synergies between the Swin Transformer and CLIP models, we address the limitations of traditional methods and significantly advance the field of sports competition analysis. This innovative model opens up new avenues for comprehensive tactical analysis in sports, benefiting coaches, athletes, and sports enthusiasts alike.

Sports-ACtrans Net: research on multimodal robotic sports action recognition driven via ST-GCN

TL-CStrans Net: a vision robot for table tennis player action recognition driven via CS-Transformer

An Attention Enhanced Spatial–Temporal Graph Convolutional LSTM Network for Action Recognition in Karate

An Attentional Spatial Temporal Graph Convolutional Network with Co-Occurrence Feature Learning for Action Recognition

Swimtrans Net: a multimodal robotic system for swimming action recognition driven via Swin-Transformer

Sports competition tactical analysis model of cross-modal transfer learning intelligent robot based on Swin Transformer and CLIP

ACA-Net: adaptive context-aware network for basketball action recognition

Deep Spatial/temporal-level feature engineering for Tennis-based action recognition

A Spatio-Temporal Transformer Network for Human Motion Prediction in Human-Robot Collaboration

RL-CWtrans Net: multimodal swimming coaching driven via robot vision

Study about Football Action Recognition Method Based on Deep Learning and Improved Dynamic Time Warping Algorithm

DMA-SGCN for Video Motion Recognition: A Tool for Advanced Sports Analysis

Traditional Martial Arts Assessmet Based on Graph Convolutional Network

A novel approach for automatic detection and identification of inappropriate postures and movements of table tennis players

An Action Recognition Technology for Badminton Players Using Deep Learning

Wearable Sensors for Activity Recognition in Ultimate Frisbee Using Convolutional Neural Networks and Transfer Learning

Attention-enhanced gated recurrent unit for action recognition in tennis

An interpretable composite CNN and GRU for fine-grained martial arts motion modeling using big data analytics and machine learning

Transferable two-stream convolutional neural network for human action recognition

Research on Sports Training Action Recognition Based on Deep Learning

Action recognition in rehabilitation: combining 3D convolution and LSTM with spatiotemporal attention