Tracking one-in-a-million: Large-scale benchmark for microbial single-cell tracking with experiment-aware robustness metrics

J. Seiffarth,L. Blöbaum,R. D. Paul,N. Friederich,A. J. Yamachui Sitcheu,R. Mikut,H. Scharr,A. Grünberger,K. Nöh

2024-11-01

Abstract:Tracking the development of living cells in live-cell time-lapses reveals crucial insights into single-cell behavior and presents tremendous potential for biomedical and biotechnological applications. In microbial live-cell imaging (MLCI), a few to thousands of cells have to be detected and tracked within dozens of growing cell colonies. The challenge of tracking cells is heavily influenced by the experiment parameters, namely the imaging interval and maximal cell number. For now, tracking benchmarks are not widely available in MLCI and the effect of these parameters on the tracking performance are not yet known. Therefore, we present the largest publicly available and annotated dataset for MLCI, containing more than 1.4 million cell instances, 29k cell tracks, and 14k cell divisions. With this dataset at hand, we generalize existing tracking metrics to incorporate relevant imaging and experiment parameters into experiment-aware metrics. These metrics reveal that current cell tracking methods crucially depend on the choice of the experiment parameters, where their performance deteriorates at high imaging intervals and large cell colonies. Thus, our new benchmark quantifies the influence of experiment parameters on the tracking quality, and gives the opportunity to develop new data-driven methods that generalize across imaging and experiment parameters. The benchmark dataset is publicly available at <a class="link-external link-https" href="https://zenodo.org/doi/10.5281/zenodo.7260136" rel="external noopener nofollow">this https URL</a>.

Computer Vision and Pattern Recognition

What problem does this paper attempt to address?

### What problems does this paper attempt to solve?

This paper aims to address several key challenges in microbial single - cell tracking. Specifically:

1. **Lack of large - scale benchmark datasets**: In microbial live - cell imaging (MLCI), there is currently a lack of large - scale, publicly annotated datasets for evaluating and improving cell - tracking algorithms. This has led to insufficient research on the impact of experimental parameters (such as imaging interval and maximum cell number) on tracking performance.

2. **Impact of experimental parameters on tracking performance**: Existing cell - tracking methods perform very differently under different experimental parameters. In particular, in cases of high imaging intervals and large cell populations, the tracking performance will decline significantly. However, these impacts have not been quantified and systematically studied.

3. **Developing highly adaptable tracking methods**: To meet the above challenges, it is necessary to develop new data - driven methods that can adapt to different imaging and experimental parameters and establish corresponding evaluation metrics.

### Main contributions of the paper

To solve these problems, the authors have made the following three main contributions:

1. **Introducing a new large - scale annotated dataset**:
- Provides a large - scale time - series dataset containing more than 1.4 million cell instances and approximately 14,000 cell divisions.
- The dataset records the growth of Corynebacterium glutamicum cells using a low imaging interval (one image per minute).

2. **Introducing experiment - aware evaluation metrics (EATM)**:
- Expands existing tracking evaluation metrics so that they can take into account experimental parameters such as imaging interval and maximum cell number.
- These new metrics can more accurately reflect the performance of tracking algorithms under different experimental conditions.

3. **Evaluating existing tracking methods**:
- Using the newly proposed EATM and robustness metrics (RM), a wide range of state - of - the - art tracking methods are extensively evaluated.
- The results show that the performance of existing methods declines significantly in cases of high imaging intervals and large cell populations, while some deep - learning - based methods (such as Trackastra) show greater robustness.

Through these contributions, the paper not only provides an important benchmark dataset but also provides strong support for the future development of more robust microbial single - cell tracking methods.