Abstract:Trajectories are fundamental to winning in alpine skiing. Tools enabling the analysis of such curves can enhance the training activity and enrich broadcasting content. In this paper, we propose SkiTraVis, an algorithm to visualize the sequence of points traversed by a skier during its performance. SkiTraVis works on monocular videos and constitutes a pipeline of a visual tracker to model the skier's motion and of a frame correspondence module to estimate the camera's motion. The separation of the two motions enables the visualization of the trajectory according to the moving camera's perspective. We performed experiments on videos of real-world professional competitions to quantify the visualization error, the computational efficiency, as well as the applicability. Overall, the results achieved demonstrate the potential of our solution for broadcasting media enhancement and coach assistance.
What problem does this paper attempt to address?
The problem that this paper attempts to solve is: visualizing the trajectories of skiers in monocular videos. Specifically, the authors propose an algorithm named SkiTraVis, which aims to calculate and visualize the motion trajectories of skiers from videos captured by a monocular camera. This technology can assist coaches in training and also enrich broadcast content, providing more detailed motion analysis.
### Problem Background
In alpine skiing competitions, the trajectories of athletes are crucial to their competition results. By optimizing their trajectories, athletes can reduce the distance to the slalom gates, thus saving time. Existing methods for measuring skiing trajectories usually rely on wearable devices (such as GNSS trackers, IMU) and 3D terrain models, but these methods have disadvantages such as time - consuming installation and calibration, and inability to obtain context information that affects skiing performance (such as slalom gate positions, slope surface conditions, weather and visibility).
### Solution
To overcome the above limitations, this paper proposes the SkiTraVis algorithm, which can directly visualize the trajectories of skiers in monocular videos. The workflow of SkiTraVis is as follows:
1. **Skier Motion Tracking**:
- Use a visual object - tracking algorithm (such as STARK [67]) to track the skier's motion and provide their position \( p(t)=(x(t), y(t)) \) in each frame.
- Represent the skier's appearance by a bounding box \( b(t)=(x(b), y(b), w(b), h(b)) \), where \( x(b), y(b) \) are the coordinates of the upper - left corner of the bounding box, and \( w(b), h(b) \) are the width and height.
- Calculate the skier's position \( p(t) \) as follows:
\[
x(t)=x(b)+w(b)\cdot0.5
\]
\[
y(t)=y(b)+h(b)\cdot k \quad (\text{where } k\in[0, 1], \text{ usually } k = 0.9)
\]
2. **Camera Motion Tracking**:
- Estimate the camera's motion between consecutive frames by detecting and tracking static key points.
- Use Shi - Tomasi [53] to detect key points and Lucas - Kanade [39] for optical flow tracking.
- Use the RANSAC [23] algorithm to calculate the homography matrix \( H_t \) between homologous key points to describe the change in view between two frames.
3. **Trajectory Calculation**:
- Use the alignment relationship of homologous key points to calculate the homography matrix \( H_t \) by the RANSAC algorithm.
- Map the trajectory points \( p(t - 1)_i \) in the previous frame to the current frame \( F_t \) through \( H_t \) to obtain new trajectory points \( p(t)_i \).
- Add the skier's position \( p(t)_t \) in the current frame to the transformed trajectory to form the final trajectory \( \tau_t \).
### Experimental Setup
To evaluate the performance of SkiTraVis, the authors used a special evaluation dataset, which contains multi - camera broadcast videos from the SkiTB test set. Each video frame was manually annotated with the athlete's bounding box. Since manual trajectory annotation is very time - consuming and it is difficult to identify specific reference points, the authors adopted a semi - automatic annotation method, combining the LOFTR image - matching algorithm and RANSAC to calculate the homography matrix of homologous key points, generating trajectories and performing manual verification.
### Conclusion
The experimental results show that SkiTraVis can effectively calculate and visualize the trajectories of skiers under actual competition conditions, with high accuracy and computational efficiency. This algorithm is not only applicable to alpine skiing, but can also be extended to other winter sports, such as snowboarding, ski jumping and freestyle skiing.