Feature Combination Meets Attention: Baidu Soccer Embeddings and Transformer based Temporal Detection

Xin Zhou,Le Kang,Zhiyu Cheng,Bo He,Jingyu Xin
DOI: https://doi.org/10.48550/arXiv.2106.14447
2021-06-28
Abstract:With rapidly evolving internet technologies and emerging tools, sports related videos generated online are increasing at an unprecedentedly fast pace. To automate sports video editing/highlight generation process, a key task is to precisely recognize and locate the events in the long untrimmed videos. In this tech report, we present a two-stage paradigm to detect what and when events happen in soccer broadcast videos. Specifically, we fine-tune multiple action recognition models on soccer data to extract high-level semantic features, and design a transformer based temporal detection module to locate the target events. This approach achieved the state-of-the-art performance in both two tasks, i.e., action spotting and replay grounding, in the SoccerNet-v2 Challenge, under CVPR 2021 ActivityNet workshop. Our soccer embedding features are released at <a class="link-external link-https" href="https://github.com/baidu-research/vidpress-sports" rel="external noopener nofollow">this https URL</a>. By sharing these features with the broader community, we hope to accelerate the research into soccer video understanding.
Computer Vision and Pattern Recognition,Artificial Intelligence,Machine Learning
What problem does this paper attempt to address?
The problem that this paper attempts to solve is to automatically identify and locate events in football match videos, specifically including two tasks: 1. **Action Spotting**: Precisely locate the time points when key actions such as goals, shots, yellow cards, and red cards occur in the complete football match video. 2. **Replay Grounding**: Determine the timestamp in the original match of the action shown in a specific replay segment. These two tasks are crucial for the process of automated sports video editing and highlight generation. The paper proposes a two - stage method to solve these problems. First, extract high - level semantic features by fine - tuning multiple action recognition models, and then design a Transformer - based temporal detection module to precisely locate the target events. This method has achieved state - of - the - art performance in the SoccerNet - v2 challenge, especially in the action detection and replay grounding tasks.