Abstract:Background: With the continuous expansion of available biomedical data, efficient and effective information retrieval has become of utmost importance. Semantic expansion of queries using synonyms may improve information retrieval. Objective: The aim of this study was to automatically construct and evaluate expanded PubMed queries of the form "preferred term"[MH] OR "preferred term"[TIAB] OR "synonym 1"[TIAB] OR "synonym 2"[TIAB] OR …, for each of the 28,313 Medical Subject Heading (MeSH) descriptors, by using different semantic expansion strategies. We sought to propose an innovative method that could automatically evaluate these strategies, based on the three main metrics used in information science (precision, recall, and F-measure). Methods: Three semantic expansion strategies were assessed. They differed by the synonyms used to build the queries as follows: MeSH synonyms, Unified Medical Language System (UMLS) mappings, and custom mappings (Catalogue et Index des Sites Médicaux de langue Française [CISMeF]). The precision, recall, and F-measure metrics were automatically computed for the three strategies and for the standard automatic term mapping (ATM) of PubMed. The method to automatically compute the metrics involved computing the number of all relevant citations (A), using National Library of Medicine indexing as the gold standard ("preferred term"[MH]), the number of citations retrieved by the added terms ("synonym 1"[TIAB] OR "synonym 2"[TIAB] OR …) (B), and the number of relevant citations retrieved by the added terms (combining the previous two queries with an "AND" operator) (C). It was possible to programmatically compute the metrics for each strategy using each of the 28,313 MeSH descriptors as a "preferred term," corresponding to 239,724 different queries built and sent to the PubMed application program interface. The four search strategies were ranked and compared for each metric. Results: ATM had the worst performance for all three metrics among the four strategies. The MeSH strategy had the best mean precision (51%, SD 23%). The UMLS strategy had the best recall and F-measure (41%, SD 31% and 36%, SD 24%, respectively). CISMeF had the second best recall and F-measure (40%, SD 31% and 35%, SD 24%, respectively). However, considering a cutoff of 5%, CISMeF had better precision than UMLS for 1180 descriptors, better recall for 793 descriptors, and better F-measure for 678 descriptors. Conclusions: This study highlights the importance of using semantic expansion strategies to improve information retrieval. However, the performances of a given strategy, relatively to another, varied greatly depending on the MeSH descriptor. These results confirm there is no ideal search strategy for all descriptors. Different semantic expansions should be used depending on the descriptor and the user's objectives. Thus, we developed an interface that allows users to input a descriptor and then proposes the best semantic expansion to maximize the three main metrics (precision, recall, and F-measure).

Comparison of Full-Text Indexing with Metadata Indexing Based Subject Classification Using Graph-Based Index

Using Deep Learning for Title-Based Semantic Subject Indexing to Reach Competitive Performance to Full-Text

Improving short text classification using public search engines

Subject indexing in humanities: a comparison between a local university repository and an international bibliographic service

Large Scale Subject Category Classification of Scholarly Papers with Deep Attentive Neural Networks

A review of ranking approaches for semantic search on Web

Identification of the Best Semantic Expansion to Query PubMed Through Automatic Performance Assessment of Four Search Strategies on All Medical Subject Heading Descriptors: Comparative Study

FullMeSH: Improving Large-Scale MeSH Indexing with Full Text.

Google Scholar vs. dblp vs. Microsoft Academic Search: An indexing comparison for Software Engineering literature

Comparative Study of Long Document Classification

Multi-label classification of research articles using Word2Vec and identification of similarity threshold

Bibliometrics effects of a new item-by-item classification system based on reference reclassification

Measuring Similarity: Computationally Reproducing the Scholar's Interests

Ranking by Relevance and Citation Counts, a Comparative Study: Google Scholar, Microsoft Academic, WoS and Scopus

A new methodology for comparing Google Scholar and Scopus

An item-by-item subject classification of papers published in journals covered by the SSCI database using reference analysis

A Self-Assessing Compilation Based Search Approach for Analytical Research and Data Retrieval

Classifying document types to enhance search and recommendations in digital libraries

How Does Imperfect Automatic Indexing Affect Semantic Search Performance?

Short-text Classification Method Based on Concept Network

Content-based and Algorithmic Classifications of Journals: Perspectives on the Dynamics of Scientific Communication and Indexer Effects.