Abstract:Average precision (AP) as the area under the Precision – Recall curve is the de facto standard for comparing the quality of algorithms for classification, information retrieval, object detection, etc. However, traditional Precision – Recall curves usually have a zigzag shape, which makes it difficult to calculate the average precision and to compare algorithms. This paper proposes a statistical approach to the construction of Precision – Recall curves when assessing the quality of algorithms for object detection in images. This approach is based on calculating Statistical Precision and Statistical Recall. Instead of the traditional confidence level, a statistical confidence level is calculated for each image as a percentage of objects detected. For each threshold value of the statistical confidence level, the total number of correctly detected objects (Integral TP) and the total number of background objects mistakenly assigned by the algorithm to one of the classes (Integral FP) are calculated for each image. Next, the values of Precision and Recall are calculated. Precision – Recall statistical curves, unlike traditional curves, are guaranteed to be monotonically non-increasing. At the same time, the Statistical Average Precision of object detection algorithms on small test datasets turns out to be less than the traditional Average Precision. On relatively large test image datasets, these differences are smoothed out. The comparison of the use of conventional and statistical Precision – Recall curves is given on a specific example.

Precision and Recall Reject Curves for Classification

Revisiting Precision and Recall Definition for Generative Model Evaluation

Statistical Precision – Recall curves for object detection quality assessment

Decision Curve Analysis: a Technical Note

Tuning model parameters in class‐imbalanced learning with precision‐recall curve

Binormal Precision–Recall Curves for Optimal Classification of Imbalanced Data

Area under Precision-Recall Curves for Weighted and Unweighted Data

The Effect of Class Imbalance on Precision-Recall Curves

Sensitivity and Specificity versus Precision and Recall, and Related Dilemmas

Optimum Reject Options for Prototype-based Classification

Performance measures for classification systems with rejection

Precrec: fast and accurate precision–recall and ROC curve calculations in R

Precision-Recall Curve (PRC) Classification Trees

Population and Empirical PR Curves for Assessment of Ranking Algorithms

The Precision-Recall Plot Is More Informative than the ROC Plot When Evaluating Binary Classifiers on Imbalanced Datasets

Classification performance assessment for imbalanced multiclass data

Classification with rejection: concepts and formal evaluations

Commonly used software tools produce conflicting and overly-optimistic AUPRC values

The MCC-F1 curve: a performance evaluation technique for binary classification

The receiver operating characteristic curve accurately assesses imbalanced datasets

Stacked Confusion Reject Plots (SCORE)