Abstract:The Pascal Visual Object Classes (VOC) challenge consists of two components: (i) a publicly available dataset of images together with ground truth annotation and standardised evaluation software; and (ii) an annual competition and workshop. There are five challenges: classification, detection, segmentation, action classification, and person layout. In this paper we provide a review of the challenge from 2008–2012. The paper is intended for two audiences: algorithm designers, researchers who want to see what the state of the art is, as measured by performance on the VOC datasets, along with the limitations and weak points of the current generation of algorithms; and, challenge designers, who want to see what we as organisers have learnt from the process and our recommendations for the organisation of future challenges. To analyse the performance of submitted algorithms on the VOC datasets we introduce a number of novel evaluation methods: a bootstrapping method for determining whether differences in the performance of two algorithms are significant or not; a normalised average precision so that performance can be compared across classes with different proportions of positive instances; a clustering method for visualising the performance across multiple algorithms so that the hard and easy images can be identified; and the use of a joint classifier over the submitted algorithms in order to measure their complementarity and combined performance. We also analyse the community’s progress through time using the methods of Hoiem et al. (Proceedings of European Conference on Computer Vision, 2012) to identify the types of occurring errors. We conclude the paper with an appraisal of the aspects of the challenge that worked well, and those that could be improved in future challenges.

Proceedings of The 39th Annual Workshop of the Austrian Association for Pattern Recognition (OAGM), 2015

Proceedings of the 37th Annual Workshop of the Austrian Association for Pattern Recognition (ÖAGM/AAPR), 2013

ÖAGM/AAPR 2013 - The 37th Annual Workshop of the Austrian Association for Pattern Recognition

Proceedings of The 38th Annual Workshop of the Austrian Association for Pattern Recognition (ÖAGM), 2014

Pattern Recognition

Proposal For Workshop On Applens: Mining And Learning From Smartphone Apps For Users

AAAI-07 Workshop Reports.

Proceedings Thirteenth Workshop on Quantitative Aspects of Programming Languages and Systems

10 Years of the PCG workshop: Past and Future Trends

FME '22: 2nd Workshop on Facial Micro-Expression: Advanced Techniques for Multi-Modal Facial Expression Analysis

Proceedings Third Workshop on Graphs as Models

Joint Proceedings of the Workshop on AI Problems and Approaches for Intelligent Environments and Workshop on Semantic Cities

The Pascal Visual Object Classes Challenge: A Retrospective

Image Processing, Computer Vision, and Pattern Recognition

Artificial Intelligence : from Research to Application ; the Upper-Rhine Artificial Intelligence Symposium (UR-AI 2019)

Understanding and Shaping Human-Technology Assemblages in the Age of Generative AI

Proceedings 2nd Workshop on GRAPH Inspection and Traversal Engineering

SIGACT News Online Algorithms Column 41: 2023 in review

SIGACT News Online Algorithms Column 40

Proceedings of Principle and practice of data and Knowledge Acquisition Workshop 2022 (PKAW 2022)

Proceedings 14th International Conference on Automated Deduction in Geometry