Abstract:Abstract Purpose The explosion of molecular biomarker and treatment information in the precision medicine era drastically exacerbated difficulty in identifying patient-relevant knowledge for clinical researchers and practitioners. Curated knowledgebases, such as the JAX Clinical Knowledgebase (CKB) are tools to organize and display knowledge in a readily accessible format; however, curators face the same challenges in comprehensively identifying clinically relevant information for curation. Natural language processing (NLP) has emerged as a promising direction for accelerating manual curation, but prior applications were often conceived as stand-alone efforts to automate curation, and the scope is often limited to simple entity and relation extraction. In this paper, we study the alternative paradigm of assisted curation and identify key desiderata to scale up knowledge curation with human-computer symbiosis. Methods We chose precision oncology for a case study and introduced self-supervised machine reading, which can automatically generate noisy training examples from unlabeled text. We developed a curation user interface (UI) for precision oncology and through iterative “curathons” (curation hackathons), conducted retrospective and prospective user studies for head-to-head comparison between manual and machine-assisted curation. Results Contrary to the prevailing assumption, we showed that high recall is more important for end-to-end assisted curation. In extensive user studies, we showed that assisted curation can double the curation speed and increase the number of findings by an order of magnitude for previously scarcely curated drugs. Conclusion We demonstrated that an iterative and thoughtful collaboration between professional curators and NLP researchers can facilitate rapid advances in assisted curation for precision medicine. Human-machine reading symbiosis can potentially be applicable to clinical care and research scenarios where curation is a major bottleneck.

Rapidly developing NLP applications for content curation

A Workflow Manager for Complex NLP and Content Curation Pipelines

Development and Evaluation of Task-Specific NLP Framework in China.

QURATOR: Innovative Technologies for Content and Data Curation

NLP Workbench: Efficient and Extensible Integration of State-of-the-art Text Mining Tools

CurateGPT: A flexible language-model assisted biocuration tool

Natural Language Processing in aid of FlyBase curators

Toward Community-Based Natural Language Processing (CBNLP): Cocreating With Communities

Orchestrating NLP Services for the Legal Domain

Integrating NLP Using Linked Data

A Data-Centric Framework for Composable NLP Workflows

Producing NLP-based On-line Contentware

Random matrix ensembles of time-lagged correlation matrices: Derivation of eigenvalue spectra and analysis of financial time-series

Collaborative curation of data from bio-medical texts and abstracts and its integration

Breaching the curation bottleneck with human-machine reading symbiosis

Towards a General-Purpose Linguistic Annotation Backend

NLPExplorer: Exploring the Universe of NLP Papers

NLPContributions: An Annotation Scheme for Machine Reading of Scholarly Contributions in Natural Language Processing Literature

NLP-KG: A System for Exploratory Search of Scientific Literature in Natural Language Processing

Curatr: A Platform for Semantic Analysis and Curation of Historical Literary Texts

STAMP 4 NLP -- An Agile Framework for Rapid Quality-Driven NLP Applications Development