Abstract:MOTIVATION:The identification of protein and gene names (PGNs) from the scientific literature requires semantic resources: Terminological and lexical resources deliver the term candidates into PGN tagging solutions and the gold standard corpora (GSC) train them to identify term parameters and contextual features. Ideally all three resources, i.e. corpora, lexica and taggers, cover the same domain knowledge, and thus support identification of the same types of PGNs and cover all of them. Unfortunately, none of the three serves as a predominant standard and for this reason it is worth exploring, how these three resources comply with each other. We systematically compare different PGN taggers against publicly available corpora and analyze the impact of the included lexical resource in their performance. In particular, we determine the performance gains through false positive filtering, which contributes to the disambiguation of identified PGNs.RESULTS:In general, machine learning approaches (ML-Tag) for PGN tagging show higher F1-measure performance against the BioCreative-II and Jnlpba GSCs (exact matching), whereas the lexicon based approaches (LexTag) in combination with disambiguation methods show better results on FsuPrge and PennBio. The ML-Tag solutions balance precision and recall, whereas the LexTag solutions have different precision and recall profiles at the same F1-measure across all corpora. Higher recall is achieved with larger lexical resources, which also introduce more noise (false positive results). The ML-Tag solutions certainly perform best, if the test corpus is from the same GSC as the training corpus. As expected, the false negative errors characterize the test corpora and - on the other hand - the profiles of the false positive mistakes characterize the tagging solutions. Lex-Tag solutions that are based on a large terminological resource in combination with false positive filtering produce better results, which, in addition, provide concept identifiers from a knowledge source in contrast to ML-Tag solutions.CONCLUSION:The standard ML-Tag solutions achieve high performance, but not across all corpora, and thus should be trained using several different corpora to reduce possible biases. The LexTag solutions have different profiles for their precision and recall performance, but with similar F1-measure. This result is surprising and suggests that they cover a portion of the most common naming standards, but cope differently with the term variability across the corpora. The false positive filtering applied to LexTag solutions does improve the results by increasing their precision without compromising significantly their recall. The harmonisation of the annotation schemes in combination with standardized lexical resources in the tagging solutions will enable their comparability and will pave the way for a shared standard.

Cross-Domain Evaluation of POS Taggers: From Wall Street Journal to Fandom Wiki

Incorporating External POS Tagger for Punctuation Restoration

Parts-of-Speech Tagger Errors Do Not Necessarily Degrade Accuracy in Extracting Information from Biomedical Text

Rapid Adaptation of POS Tagging for Domain Specific Uses

Cross-Register Projection for Headline Part of Speech Tagging

Revisiting Supertagging for Faster HPSG Pasing

Reliable Part-of-Speech Tagging of Historical Corpora through Set-Valued Prediction

Evaluating Gold Standard Corpora Against Gene/protein Tagging Solutions and Lexical Resources

Exploiting Lexical Information for Function Tag Labeling

Recipe for Zero-shot POS Tagging: Is It Useful in Realistic Scenarios?

Towards Accurate and Efficient Chinese Part-of-Speech Tagging.

Software-specific Part-of-speech Tagging

Character Eyes: Seeing Language through Character-Level Taggers

Exploiting Heterogeneous Annotations for Weibo Word Segmentation and POS Tagging.

Quality Assurance Of Automatic Annotation Of Very Large Corpora: A Study Based On Heterogeneous Tagging Systems

Supertagging the Long Tail with Tree-Structured Decoding of Complex Categories

Developing a corpus of clinical notes manually annotated for part-of-speech

Latin Treebanks in Review: An Evaluation of Morphological Tagging Across Time

CrossWeigh: Training Named Entity Tagger from Imperfect Annotations

Looking into the Operational Modalities Adopted in Some of the POS Tagging Tools in Identification of Contextual Part-of-Speech of Words in Texts

An SVMTool-Based Chinese POS Tagger