Abstract:BACKGROUND Question answering (QA) system is widely used in web-based health-care applications. Health consumers likely asked similar questions in various natural language expression due to the lack of medical knowledge. It’s challenging to match a new question to previous similar questions for answering. In health QA system development, question matching (QM) is a task to judge whether a pair of questions express the same meaning and is used to map the answer of matched question in the given question-answering database. BERT (i.e. Bidirectional Encoder Representations from Transformers) is proved to be state-of- the-art model in natural language processing (NLP) tasks, such as binary classification and sentence matching. As a light model of BERT, ALBERT is proposed to address the huge parameters and low training speed problems of BERT. Both of BERT and ALBERT can be used to address the QM problem. OBJECTIVE In this study, we aim to develop an ALBERT based method for Chinese health related question matching. METHODS Our proposed method, named as ALBERT-QM, consists of three components. (1)Data augmenting. Similar health question pairs were augmented for training preparation. (2)ALBERT model training. Given the augmented training pairs, three ALBERT models were trained and fine-tuned. (3)Similarity combining. Health question similarity score were calculated by combining ALBRT model outputs with text similarity. To evaluate our ALBERT-QM performance on similar question identification, we used an open dataset with 20,000 labeled Chinese health question pairs. RESULTS Our ALBERT-QM is able to identify similar Chinese health questions, achieving the precision of 86.69%, recall of 86.70% and F1 of 86.69%. Comparing with baseline method (text similarity algorithm), ALBERT-QM enhanced the F1-score by 20.73%. Comparing with other BERT series models, our ALBERT-QM is much lighter with the files size of 64.8MB which is 1/6 times that other BERT models. We made our ALBERT-QM open accessible at https://github.com/trueto/albert_question_match. CONCLUSIONS In this study, we developed an open source algorithm, ALBERT-QM, contributing to similar Chinese health questions identification in a health QA system. Our ALBERT-QM achieved better performance in question matching with lower memory usage, which is beneficial to the web-based or mobile-based QA applications.

A Deep Transfer Learning Method for Medical Question Matching.

Medical Question Retrieval Based on Siamese Neural Network and Transfer Learning Method.

A Joint Model For Question-Answering Over Traditional Chinese Medicine

IARNN-Based Semantic-Containing Double-Level Embedding Bi-LSTM for Question-and-Answer Matching

Deep scaled dot-product attention based domain adaptation model for biomedical question answering

Improving Medical Q&A Matching by Augmenting Dual-Channel Attention with Global Similarity.

Multi-level Composite Neural Networks for Medical Question Answer Matching

A Review on Medical Textual Question Answering Systems Based on Deep Learning Approaches

An Semantic Similarity Matching Method for Chinese Medical Question Text

Chinese Medical Question Answer Matching Based on Interactive Sentence Representation Learning

A medical question answering system using large language models and knowledge graphs

Multi-task Transfer Learning for Biomedical Machine Reading Comprehension.

Chinese Medical Question Answer Matching Using End-to-End Character-Level Multi-Scale CNNs

Deep learning for named entity recognition on Chinese electronic medical records: Combining deep transfer learning with multitask bi-directional LSTM RNN

Transfer learning for medical images analyses: A survey

Answering medical questions in Chinese using automatically mined knowledge and deep neural networks: an end-to-end solution

ALBERT-QM: An ALBERT Based Method for Chinese Health Related Question Matching (Preprint)

Qilin-Med: Multi-stage Knowledge Injection Advanced Medical Large Language Model

Integration of Multi-Source Medical Data for Medical Diagnosis Question Answering

A Chinese Question Answering System in Medical Domain

Hierarchical Multi-layer Transfer Learning Model for Biomedical Question Answering