Let Guidelines Guide You: A Prescriptive Guideline-Centered Data Annotation Methodology

Federico Ruggeri,Eleonora Misino,Arianna Muti,Katerina Korre,Paolo Torroni,Alberto Barrón-Cedeño
2024-07-02
Abstract:We introduce the Guideline-Centered annotation process, a novel data annotation methodology focused on reporting the annotation guidelines associated with each data sample. We identify three main limitations of the standard prescriptive annotation process and describe how the Guideline-Centered methodology overcomes them by reducing the loss of information in the annotation process and ensuring adherence to guidelines. Additionally, we discuss how the Guideline-Centered enables the reuse of annotated data across multiple tasks at the cost of a single human-annotation process.
Computation and Language
What problem does this paper attempt to address?