Abstract:This paper focuses on image and video content analysis of handball scenes and applying deep learning methods for detecting and tracking the players and recognizing their activities. Handball is a team sport of two teams played indoors with the ball with well-defined goals and rules. The game is dynamic, with fourteen players moving quickly throughout the field in different directions, changing positions and roles from defensive to offensive, and performing different techniques and actions. Such dynamic team sports present challenging and demanding scenarios for both the object detector and the tracking algorithms and other computer vision tasks, such as action recognition and localization, with much room for improvement of existing algorithms. The aim of the paper is to explore the computer vision-based solutions for recognizing player actions that can be applied in unconstrained handball scenes with no additional sensors and with modest requirements, allowing a broader adoption of computer vision applications in both professional and amateur settings. This paper presents semi-manual creation of custom handball action dataset based on automatic player detection and tracking, and models for handball action recognition and localization using Inflated 3D Networks (I3D). For the task of player and ball detection, different configurations of You Only Look Once (YOLO) and Mask Region-Based Convolutional Neural Network (Mask R-CNN) models fine-tuned on custom handball datasets are compared to original YOLOv7 model to select the best detector that will be used for tracking-by-detection algorithms. For the player tracking, DeepSORT and Bag of tricks for SORT (BoT SORT) algorithms with Mask R-CNN and YOLO detectors were tested and compared. For the task of action recognition, I3D multi-class model and ensemble of binary I3D models are trained with different input frame lengths and frame selection strategies, and the best solution is proposed for handball action recognition. The obtained action recognition models perform well on the test set with nine handball action classes, with average F1 measures of 0.69 and 0.75 for ensemble and multi-class classifiers, respectively. They can be used to index handball videos to facilitate retrieval automatically. Finally, some open issues, challenges in applying deep learning methods in such a dynamic sports environment, and direction for future development will be discussed.

Recognition of basketball referee signals from real-time videos

Recognition of basketball referee signals from videos using Histogram of Oriented Gradients (HOG) and Support Vector Machine (SVM)

A real-time hand gesture recognition method

A real-time basketball referee gesture recognition system based on multi-scale spatio-temporal features

A Sensor-Based Official Basketball Referee Signals Recognition System Using Deep Belief Networks

A Method of Simultaneously Action Recognition and Video Segmentation of Video Streams.

A Hierarchical Hand Gesture Recognition Framework for Sports Referee Training-Based EMG and Accelerometer Sensors

Recognizing Scoring in Basketball Game from AER Sequence by Spiking Neural Networks.

A Real-Time Score Detection and Recognition Approach for Broadcast Basketball Video

Design of Gesture Recognition System for Basketball Referee Scoring Based on Recorder

Basketball Tactics Analysis Based on Improved Openpose Algorithm and Its Application

Real-Time Hand Gesture Recognition Using RGB-D Sensor

A Sports Video Behavior Recognition Using Local Spatiotemporal Patterns

Research on Basketball Shooting Action Based on Image Feature Extraction and Machine Learning

A Real Time Vision-Based Hand Gestures Recognition System

Basketball shooting angle calculation and analysis by deeply-learned vision model

Football referee gesture recognition algorithm based on YOLOv8s

Video Analysis and System Construction of Basketball Game by Lightweight Deep Learning under the Internet of Things

Fusing Motion Patterns and Key Visual Information for Semantic Event Recognition in Basketball Videos

Analysis of Movement and Activities of Handball Players Using Deep Neural Networks

A Unified System for Segmentation and Tracking of Face and Hands in Sign Language Recognition