Abstract:The long-standing problem of novel view synthesis has many applications, notably in sports broadcasting. Photorealistic novel view synthesis of soccer actions, in particular, is of enormous interest to the broadcast industry. Yet only a few industrial solutions have been proposed, and even fewer that achieve near-broadcast quality of the synthetic replays. Except for their setup of multiple static cameras around the playfield, the best proprietary systems disclose close to no information about their inner workings. Leveraging multiple static cameras for such a task indeed presents a challenge rarely tackled in the literature, for a lack of public datasets: the reconstruction of a large-scale, mostly static environment, with small, fast-moving elements. Recently, the emergence of neural radiance fields has induced stunning progress in many novel view synthesis applications, leveraging deep learning principles to produce photorealistic results in the most challenging settings. In this work, we investigate the feasibility of basing a solution to the task on dynamic NeRFs, i.e., neural models purposed to reconstruct general dynamic content. We compose synthetic soccer environments and conduct multiple experiments using them, identifying key components that help reconstruct soccer scenes with dynamic NeRFs. We show that, although this approach cannot fully meet the quality requirements for the target application, it suggests promising avenues toward a cost-efficient, automatic solution. We also make our work dataset and code publicly available, with the goal to encourage further efforts from the research community on the task of novel view synthesis for dynamic soccer scenes. For code, data, and video results, please see https://soccernerfs.isach.be.

Soccer on Your Tabletop

Real-time 3D Soccer Ball Tracking from Multiple Cameras

Deep Understanding of Soccer Match Videos

Multi-camera Video Surveillance for Real-Time Analysis and Reconstruction of Soccer Games

Extracting 3D information from broadcast soccer video

Real-time 3D Football Ball Tracking from Multiple Cameras

Real-Time Modeling of 3-D Soccer Ball Trajectories from Multiple Fixed Cameras.

Graph-Based Multi-Camera Soccer Player Tracker

Analyzing In-Game Movements of Soccer Players at Scale

SoccerMap: A Deep Learning Architecture for Visually-Interpretable Analysis in Soccer

Scaling up SoccerNet with multi-view spatial localization and re-identification

SoccerNet Game State Reconstruction: End-to-End Athlete Tracking and Identification on a Minimap

Real-Time Camera Pose Estimation for Sports Fields

Visual Soccer Analytics: Understanding the Characteristics of Collective Team Movement Based on Feature-Driven Analysis and Abstraction

Video-based Analysis of Soccer Matches

Soccer: Who Has the Ball? Generating Visual Analytics and Player Statistics.

Towards Universal Soccer Video Understanding

Dynamic NeRFs for Soccer Scenes

A Monocular Vision System for Playing Soccer in Low Color Information Environments

Multi-camera Temporal Grouping for Play/Break Event Detection in Soccer Games

Monocular 3D Human Pose Estimation for Sports Broadcasts using Partial Sports Field Registration