Abstract:The cross-institutional secondary use of medical data benefits from structured semantic annotation, which ideally enables the matching and merging of semantically related data items from different sources and sites. While numerous medical terminologies and ontologies, as well as some tooling, exist to support such annotation, cross-institutional data usage based on independently annotated datasets is challenging for multiple reasons: the annotation process is resource intensive and requires a combination of medical and technical expertise since it often requires judgment calls to resolve ambiguities resulting from the non-uniqueness of potential mappings to various levels of ontological hierarchies and relational and representational systems. The divergent resolution of such ambiguities can inhibit joint cross-institutional data usage based on semantic annotation since data items with related content from different sites will not be identifiable based on their respective annotations if different choices were made without further steps such as ontological inference, which is still an active area of research. We hypothesize that a collaborative approach to the semantic annotation of medical data can contribute to more resource-efficient and high-quality annotation by utilizing prior annotational choices of others to inform the annotation process, thus both speeding up the annotation itself and fostering a consensus approach to resolving annotational ambiguities by enabling annotators to discover and follow pre-existing annotational choices. Therefore, we performed a requirements analysis for such a collaborative approach, defined an annotation workflow based on the requirement analysis results, and implemented this workflow in a prototypical Collaborative Annotation Tool (CoAT). We then evaluated its usability and present first inter-institutional experiences with this novel approach to promote practically relevant interoperability driven by use of standardized ontologies. In both single-site usability evaluation and the first inter-institutional application, the CoAT showed potential to improve both annotation efficiency and quality by seamlessly integrating collaboratively generated annotation information into the annotation workflow, warranting further development and evaluation of the proposed innovative approach.

MetaTron: advancing biomedical annotation empowering relation annotation and collaboration

Assisted neuroscience knowledge extraction via machine learning applied to neural reconstruction metadata on NeuroMorpho.Org

TeamTat: a collaborative text annotation tool

MetaMap, an interactive webtool for the exploration of metatranscriptomic reads in human disease-related RNA-seq data

Collaborative Semantic Annotation Tooling (CoAT) to Improve Efficiency and Plug-and-Play Semantic Interoperability in the Secondary Use of Medical Data: Concept, Implementation, and First Cross-Institutional Experiences

Augmented Behavioral Annotation Tools, with Application to Multimodal Datasets and Models: A Systematic Review

GeoAnnotator: A Collaborative Semi-Automatic Platform for Constructing Geo-Annotated Text Corpora

META-BASE: A Novel Architecture for Large-Scale Genomic Metadata Integration

Use of 'off-the-shelf' information extraction algorithms in clinical informatics: a feasibility study of MetaMap annotation of Italian medical notes

Annotating publicly-available samples and studies using interpretable modeling of unstructured metadata

The text2term tool to map free-text descriptions of biomedical terms to ontologies

NERO: a biomedical named-entity (recognition) ontology with a large, annotated corpus reveals meaningful associations through text embedding

MarkIt: A Collaborative Artificial Intelligence Annotation Platform Leveraging Blockchain For Medical Imaging Research

Improving Editorial Workflow and Metadata Quality at Springer Nature

TopicTracker - An advanced software pipeline for text mining on PubMed data: Bridging the gap between off-the-shelf tools and code based approaches

MetaTrinity: Enabling Fast Metagenomic Classification via Seed Counting and Edit Distance Approximation

OMAnnotator: a novel approach to building an annotated consensus genome sequence

Automated Harmonization and Large-Scale Integration of Heterogeneous Biomedical Sample Metadata Using Large Language Models

SemOntoMap: A Hybrid Approach for Semantic Annotation of Clinical Texts

Advancing Italian biomedical information extraction with transformers-based models: Methodological insights and multicenter practical application

ONER: Tool for Organization Named Entity Recognition from Affiliation Strings in PubMed Abstracts