GeoNER: Geological Named Entity Recognition with Enriched Domain Pre‐Training Model and Adversarial Training

Kai MA,Xinxin HU,Miao TIAN,Yongjian TAN,Shuai ZHENG,Liufeng TAO,Qinjun QIU

DOI: https://doi.org/10.1111/1755-6724.15213

2024-10-30

Acta Geologica Sinica - English Edition

Abstract:Acta Geologica Sinica - English Edition, Volume 98, Issue 5, Page 1404-1417, October 2024.

geosciences, multidisciplinary

What problem does this paper attempt to address?

Few-shot learning for name entity recognition in geological text based on GeoBERT

Hao Liu,Qinjun Qiu,Liang Wu,Wenjia Li,Bin Wang,Yuan Zhou

DOI: https://doi.org/10.1007/s12145-022-00775-x

2022-03-11

Earth Science Informatics

Abstract:Geological reports are records of the geological elements and survey contents found in geological exploration, but it is difficult to extract useful concepts from such reports. In the process of information extraction, accurately identification of entities in unstructured geotext is a foundational task that is known as geological named entity recognition (Geo-NER). However, the existing methods generally require a large number of annotated corpora, and face problems with long entity recognition. Therefore, this paper proposes a two-stage fine-tuning method. In the first fine-tuning stage, we use a bidirectional encoder representations from transformers language model with geological domain knowledge (GeoBERT), which combines geological domain knowledge, on a pretrained BERT model, and in the second stage, we use a small number of samples to complete the NER task in the geological report based on GeoBERT. Our proposed model achieves a very high F1-score compared to baseline models on the constructed dataset.

geosciences, multidisciplinary,computer science, interdisciplinary applications
A Model for Geographical Named Entity Recognition and Correction Based on Deep Neural Networks and the BERT Model

Wei Zhang,Jingtao Meng,Jianhua Wan,Chengkun Zhang,Jiajun Zhang,Yuanyuan Wang,Liuchang Xu,Fei Li

DOI: https://doi.org/10.3390/ijgi12100394

IF: 3.4

2023-09-29

ISPRS International Journal of Geo-Information

Abstract:Social media is widely used to share real-time information and report accidents during natural disasters. Named entity recognition (NER) is a fundamental task of geospatial information applications that aims to extract location names from natural language text. As a result, the identification of location names from social media information has gradually become a demand. Named entity correction (NEC), as a complementary task of NER, plays a crucial role in ensuring the accuracy of location names and further improving the accuracy of NER. Despite numerous methods having been adopted for NER, including text statistics-based and deep learning-based methods, there has been limited research on NEC. To address this gap, we propose the CTRE model, which is a geospatial named entity recognition and correction model based on the BERT model framework. Our approach enhances the BERT model by introducing incremental pre-training in the pre-training phase, significantly improving the model's recognition accuracy. Subsequently, we adopt the pre-training fine-tuning mode of the BERT base model and extend the fine-tuning process, incorporating a neural network framework to construct the geospatial named entity recognition model and geospatial named entity correction model, respectively. The BERT model utilizes data augmentation of VGI (volunteered geographic information) data and social media data for incremental pre-training, leading to an enhancement in the model accuracy from 85% to 87%. The F1 score of the geospatial named entity recognition model reaches an impressive 0.9045, while the precision of the geospatial named entity correction model achieves 0.9765. The experimental results robustly demonstrate the effectiveness of our proposed CTRE model, providing a reference for subsequent research on location names.

geography, physical,remote sensing,computer science, information systems
Semi-supervised geological disasters named entity recognition using few labeled data

Xinya Lei,Weijing Song,Runyu Fan,Ruyi Feng,Lizhe Wang

DOI: https://doi.org/10.1007/s10707-022-00474-1

IF: 2.7729

2022-10-19

GeoInformatica

Abstract:The geological disasters Named Entity Recognition (NER) method aims to recognize entities reflecting disaster event information in unstructured texts to construct a geohazard knowledge graph that can provide a reference for disaster emergency response. Without training on large-scale labeled data, current NER methods based on deep learning models cannot identify specific geological disaster entities from geological disaster situation reports. However, manually labeling geohazard situation reports is tedious and time-consuming. As a result, we present Semi-GDNER, a semi-supervised geological disasters NER approach that can effectively extract six kinds of geological disaster entities when a few manually labeled and unlabeled in-domain data are available. It is divided into two stages: (1) transferring the parameters of the pre-trained BERT-base model to the BERT layer of the backbone model BERT-BiLSTM-CRF and training the backbone model with a few labeled data; (2) continuing training the backbone model by expanding the training set with unlabeled data using a self-training (ST) strategy. To reduce noise in the second stage, we select the pseudo-labeled samples with high confidence to join the training set in each ST iteration. Experiments on our constructed Geological Disaster NER data show that our approach achieves a higher F1 (0.88) than other NER approaches (including five supervised NER approaches and a semi-supervised NER approach using the ST strategy of expanding the training set with all pseudo-labeled data), demonstrating the effectiveness of our approach. Furthermore, experiments on four general Chinese NER datasets show that the framework of our approach is transferable.

computer science, information systems,geography, physical
EduNER: a Chinese Named Entity Recognition Dataset for Education Research

Xu Li,Chengkun Wei,Zhuoren Jiang,Wenlong Meng,Fan Ouyang,Zihui Zhang,Wenzhi Chen

DOI: https://doi.org/10.1007/s00521-023-08635-5

2023-01-01

Abstract:A high-quality domain-oriented dataset is crucial for the domain-specific named entity recognition (NER) task. In this study, we introduce a novel education-oriented Chinese NER dataset (EduNER). To provide representative and diverse training data, we collect data from multiple sources, including textbooks, academic papers, and education-related web pages. The collected documents span ten years (2012–2021). A team of domain experts is invited to accomplish the education NER schema definition, and a group of trained annotators is hired to complete the annotation. A collaborative labeling platform is built for accelerating human annotation. The constructed EduNER dataset includes 16 entity types, 11k+ sentences, and 35,731 entities. We conduct a thorough statistical analysis of EduNER and summarize its distinctive characteristics by comparing it with eight open-domain or domain-specific NER datasets. Sixteen state-of-the-art models are further utilized for NER tasks validation. The experimental results can enlighten further exploration. To the best of our knowledge, EduNER is the first publicly available dataset for NER task in the education domain, which may promote the development of education-oriented NER models.
Chinese mineral named entity recognition based on BERT model

Yuqing Yu,Yuzhu Wang,Jingqin Mu,Wei Li,Shoutao Jiao,Zhenhua Wang,Pengfei Lv,Yueqin Zhu

DOI: https://doi.org/10.1016/j.eswa.2022.117727

IF: 8.5

2022-11-15

Expert Systems with Applications

Abstract:Mineral named entity recognition (MNER) is the extraction for the specific types of entities from unstructured Chinese mineral text, which is a prerequisite for building a mineral knowledge graph. MNER can also provide important data support for the work related to mineral resources. Chinese mineral text has many types of entities, complex semantics, and a large number of rare characters. To extract entities from Chinese mineral literature, this paper proposes an MNER model based on deep learning. To create word embeddings for mineral text, Bidirectional Encoder Representations from Transformers (BERT) is used. Moreover, the transfer matrix of the Conditional Random Field (CRF) algorithm is combined to improve the accuracy of sequence labeling. Finally, some experiments are conducted on the constructed dataset. The results show that the model can effectively recognize seven mineral entities with an average F1-score of 0.842.

computer science, artificial intelligence,engineering, electrical & electronic,operations research & management science
Unsupervised cross-domain named entity recognition using entity-aware adversarial training

Qi Peng,Changmeng Zheng,Yi Cai,Tao Wang,Haoran Xie,Qing Li

DOI: https://doi.org/10.1016/j.neunet.2020.12.027

IF: 7.8

2021-06-01

Neural Networks

Abstract:<p>The success of neural network based methods in named entity recognition (NER) is heavily relied on abundant manual labeled data. However, these NER methods are unavailable when the data is fully-unlabeled in a new domain. To address the problem, we propose an unsupervised cross-domain model which leverages labeled data from source domain to predict entities in unlabeled target domain. To relieve the distribution divergence when transferring knowledge from source to target domain, we apply adversarial training. Furthermore, we design an entity-aware attention module to guide the adversarial training to reduce the discrepancy of entity features between different domains. Experimental results demonstrate that our model outperforms other methods and achieves state-of-the-art performance.</p>

computer science, artificial intelligence,neurosciences
Enhanced Chinese Domain Named Entity Recognition: An Approach with Lexicon Boundary and Frequency Weight Features

Yan Guo,Shixiang Feng,Fujiang Liu,Weihua Lin,Hongchen Liu,Xianbin Wang,Junshun Su,Qiankai Gao

DOI: https://doi.org/10.3390/app14010354

2023-12-30

Applied Sciences

Abstract:Named entity recognition (NER) plays a crucial role in information extraction but faces challenges in the Chinese context. Especially in Chinese paleontology popular science, NER encounters difficulties, such as low recognition performance for long and nested entities, as well as the complexity of handling mixed Chinese–English texts. This study aims to enhance the performance of NER in this domain. We propose an approach based on the multi-head self-attention mechanism for integrating Chinese lexicon-level features; by integrating Chinese lexicon boundary and domain term frequency weight features, this method enhances the model’s perception of entity boundaries, relative positions, and types. To address training prediction inconsistency, we introduce a novel data augmentation method, generating enhanced data based on the difference set between all and sample entity types. Experiments on four Chinese datasets, namely Resume, Youku, SubDuIE, and our PPOST, show that our approach outperforms baselines, achieving F1-score improvements of 0.03%, 0.16%, 1.27%, and 2.28%, respectively. This research confirms the effectiveness of integrating Chinese lexicon boundary and domain term frequency weight features in NER. Our work provides valuable insights for improving the applicability and performance of NER in other Chinese domain scenarios.

materials science, multidisciplinary,engineering,chemistry,physics, applied
A Toponym Entity Recognition Model for Chinese Social Media

Yuyang Qi,Renjian Zhai,Fang Wu,Jichong Yin,Xianyong Gong,Li Zhu,Haikun Yu

DOI: https://doi.org/10.3390/ijgi13090311

IF: 3.4

2024-09-01

ISPRS International Journal of Geo-Information

Abstract:In the era of information explosion, Chinese social media has become a repository for massive geographic information; however, its unique unstructured nature and diverse expressions are challenging to toponym entity recognition. To address this problem, we propose a Chinese social media named entity recognition (CSMNER) model to improve the accuracy and robustness of toponym recognition in Chinese social media texts. By combining the BERT (Bidirectional Encoder Representations from Transformers) pre-trained model with an improved IDCNN-BiLSTM-CRF (Iterated Dilated Convolutional Neural Network- Bidirectional Long Short-Term Memory- Conditional Random Field) architecture, this study innovatively incorporates a boundary extension module to effectively extract the local boundary features and contextual semantic features of the toponym, successfully addressing the recognition challenges posed by noise interference and language expression variability. To verify the effectiveness of the model, experiments were carried out on three datasets: WeiboNER, MSRA, and the Chinese social named entity recognition (CSNER) dataset, a self-built named entity recognition dataset. Compared with the existing models, CSMNER achieves significant performance improvement in toponym recognition tasks.

geography, physical,remote sensing,computer science, information systems
Coarse-to-Fine Pre-training for Named Entity Recognition

Mengge Xue,Bowen Yu,Zhenyu Zhang,Tingwen Liu,Yue Zhang,Bin Wang

DOI: https://doi.org/10.48550/arXiv.2010.08210

2020-10-16

Abstract:More recently, Named Entity Recognition hasachieved great advances aided by pre-trainingapproaches such as BERT. However, currentpre-training techniques focus on building lan-guage modeling objectives to learn a gen-eral representation, ignoring the named entity-related knowledge. To this end, we proposea NER-specific pre-training framework to in-ject coarse-to-fine automatically mined entityknowledge into pre-trained models. Specifi-cally, we first warm-up the model via an en-tity span identification task by training it withWikipedia anchors, which can be deemed asgeneral-typed entities. Then we leverage thegazetteer-based distant supervision strategy totrain the model extract coarse-grained typedentities. Finally, we devise a self-supervisedauxiliary task to mine the fine-grained namedentity knowledge via <a class="link-external link-http" href="http://clustering.Empiricalstudies" rel="external noopener nofollow">this http URL</a> on three public NER datasets demon-strate that our framework achieves significantimprovements against several pre-trained base-lines, establishing the new state-of-the-art per-formance on three benchmarks. Besides, weshow that our framework gains promising re-sults without using human-labeled trainingdata, demonstrating its effectiveness in label-few and low-resource scenarios

Computation and Language
ERNIE-GeoL: A Geography-and-Language Pre-trained Model and its Applications in Baidu Maps

Jizhou Huang,Haifeng Wang,Yibo Sun,Yunsheng Shi,Zhengjie Huang,An Zhuo,Shikun Feng

DOI: https://doi.org/10.48550/arXiv.2203.09127

2022-03-17

Computation and Language

Abstract:Pre-trained models (PTMs) have become a fundamental backbone for downstream tasks in natural language processing and computer vision. Despite initial gains that were obtained by applying generic PTMs to geo-related tasks at Baidu Maps, a clear performance plateau over time was observed. One of the main reasons for this plateau is the lack of readily available geographic knowledge in generic PTMs. To address this problem, in this paper, we present ERNIE-GeoL, which is a geography-and-language pre-trained model designed and developed for improving the geo-related tasks at Baidu Maps. ERNIE-GeoL is elaborately designed to learn a universal representation of geography-language by pre-training on large-scale data generated from a heterogeneous graph that contains abundant geographic knowledge. Extensive quantitative and qualitative experiments conducted on large-scale real-world datasets demonstrate the superiority and effectiveness of ERNIE-GeoL. ERNIE-GeoL has already been deployed in production at Baidu Maps since April 2021, which significantly benefits the performance of various downstream tasks. This demonstrates that ERNIE-GeoL can serve as a fundamental backbone for a wide range of geo-related tasks.
A deep neural network model for Chinese toponym matching with geographic pre-training model

Qinjun QiuShiyu ZhengMiao TianJiali LiKai MaLiufeng TaoZhong Xiea Key Laboratory of Geological Survey and Evaluation of Ministry of Education,China University of Geosciences,Wuhan,People's Republic of Chinab School of Computer Science,China University of Geosciences,Wuhan,People's Republic of Chinac Hubei Key Laboratory of Intelligent Vision Based Monitoring for Hydroelectric Engineering,China Three Gorges University,Yichang,People's Republic of Chinad Ministry of Nature Resources Key Laboratory of Quantitative Resources Assessment and Information Technology,Wuhan,People's Republic of China

DOI: https://doi.org/10.1080/17538947.2024.2353111

IF: 4.606

2024-05-13

International Journal of Digital Earth

Abstract:Multiple tasks within the field of geographical information retrieval and geographical information sciences necessitate toponym matching, which involves the challenge of aligning toponyms that share a common referent. The multiple string similarity approaches struggle when confronted with the complexities associated with unofficial and/or historical variants of identical toponyms. Also, current state-of-the-art approaches/tools to supervised machine learning rely on labeled samples, and they do not adequately address the intricacies of character replacements either from transliterations or historical shifts in linguistic and cultural norms. To address these issues, this paper proposes a novel matching approach that leverages a deep neural network model empowered by geographic language representation model, known as GeoBERT, which stands for geographic Bidirectional Encoder Representations from Transformers (BERT). This model harnesses the groundbreaking capabilities of the GeoBERT framework by extending a generalized Enhanced Sequential Inference Model architecture and integrating multiple features to enhance the accuracy and robustness of the toponym matching. We present a comprehensive evaluation of the proposed method's performance using three extensive datasets. The findings clearly illustrate that our approach outperforms the individual similarity metrics used in previous studies.

geography, physical,remote sensing
Beyond extraction accuracy: addressing the quality of geographical named entity through advanced recognition and correction models using a modified BERT framework

Liuchang Xu,Jiajun Zhang,Chengkun Zhang,Xinyu Zheng,Zhenhong Du,Xingyu Xue

DOI: https://doi.org/10.1080/10095020.2024.2354229

IF: 4.278

2024-05-30

Geo-spatial Information Science

Abstract:In the realm of geospatial services and applications, the accuracy of address information is of utmost importance. Traditional methods of data collection, being both labor-intensive and costly, have prompted researchers to turn to Volunteered Geographic Information (VGI) for the extraction of Geographical Named Entity (GNE).Notwithstanding, prior studies have predominantly concentrated on enhancing extraction accuracy, while often overlooking the critical aspect of GNE quality. This study addresses this gap by employing a multifaceted approach. Initially, a Geographical Named Entity Semantic Model (GNESM) was constructed by improving the BERT framework and conducting ablation experiments on multiple influencing factors to verify its feasibility. Based on GNESM, a Geographical Named Entity Recognition Model (GNERM) was constructed by incremental pre-training with social media text data and fine-tuning to achieve a recognition accuracy of 90.9%. Subsequently, a Geographical Named Entity Error Correction Model (GNEECM) was constructed by training GNESM with standard GNE data and incorporating error detection and correction modules, achieving a remarkable accuracy of 96.6% in error detection and correction tasks. The experimental results convincingly demonstrate that the proposed identification and correction methods outperform all compared methods. Through the identification and correction process, this study successfully obtained high-quality GNE data, providing a reference for expanding standard address libraries and subsequent research on geographic named entity.

remote sensing
Named entity recognition for Chinese based on global pointer and adversarial training

Hongjun Li,Mingzhe Cheng,Zelin Yang,Liqun Yang,Yansong Chua

DOI: https://doi.org/10.1038/s41598-023-30355-y

IF: 4.6

2023-02-26

Scientific Reports

Abstract:Named entity recognition aims to identify entities from unstructured text and is an important subtask for natural language processing and building knowledge graphs. Most of the existing entity recognition methods use conditional random fields as label decoders or use pointer networks for entity recognition. However, when the number of tags is large, the computational cost of method based on conditional random fields is high and the problem of nested entities cannot be solved. The pointer network uses two modules to identify the first and the last of the entities separately, and a single module can only focus on the information of the first or the last of the entities, but cannot pay attention to the global information of the entities. In addition, the neural network model has the problem of local instability. To solve mentioned problems, a named entity recognition model based on global pointer and adversarial training is proposed. To obtain global entity information, global pointer is used to decode entity information, and rotary relative position information is considered in the model designing to improve the model's perception of position; to solve the model's local instability problem, adversarial training is used to improve the robustness and generalization of the model. The experimental results show that the F1 score of the model are improved on several public datasets of OntoNotes5, MSRA, Resume, and Weibo compared with the existing mainstream models.

multidisciplinary sciences
Neural Chinese Named Entity Recognition via CNN-LSTM-CRF and Joint Training with Word Segmentation

Fangzhao Wu,Junxin Liu,Chuhan Wu,Yongfeng Huang,Xing Xie

DOI: https://doi.org/10.48550/arXiv.1905.01964

2019-04-26

Computation and Language

Abstract:Chinese named entity recognition (CNER) is an important task in Chinese natural language processing field. However, CNER is very challenging since Chinese entity names are highly context-dependent. In addition, Chinese texts lack delimiters to separate words, making it difficult to identify the boundary of entities. Besides, the training data for CNER in many domains is usually insufficient, and annotating enough training data for CNER is very expensive and time-consuming. In this paper, we propose a neural approach for CNER. First, we introduce a CNN-LSTM-CRF neural architecture to capture both local and long-distance contexts for CNER. Second, we propose a unified framework to jointly train CNER and word segmentation models in order to enhance the ability of CNER model in identifying entity boundaries. Third, we introduce an automatic method to generate pseudo labeled samples from existing labeled data which can enrich the training data. Experiments on two benchmark datasets show that our approach can effectively improve the performance of Chinese named entity recognition, especially when training data is insufficient.
MetaNER: Named Entity Recognition with Meta-Learning.

Jing Li,Shuo Shang,Ling Shao

DOI: https://doi.org/10.1145/3366423.3380127

2020-01-01

Abstract:Recent neural architectures in named entity recognition (NER) have yielded state-of-the-art performance on single domain data such as newswires. However, they still suffer from (i) requiring massive amounts of training data to avoid overfitting; (ii) huge performance degradation when there is a domain shift in the data distribution between training and testing. In this paper, we investigate the problem of domain adaptation for NER under homogeneous and heterogeneous settings. We propose MetaNER, a novel meta-learning approach for domain adaptation in NER. Specifically, MetaNER incorporates meta-learning and adversarial training strategies to encourage robust, general and transferable representations for sequence labeling. The key advantage of MetaNER is that it is capable of adapting to new unseen domains with a small amount of annotated data from those domains. We extensively evaluate MetaNER on multiple datasets under homogeneous and heterogeneous settings. The experimental results show that MetaNER achieves state-of-the-art performance against eight baselines. Impressively, MetaNER surpasses the in-domain performance using only 16.17% and 34.76% of target domain data on average for homogeneous and heterogeneous settings, respectively.
Chinese named entity recognition: The state of the art

Pan Liu,Yanming Guo,Fenglei Wang,Guohui Li

DOI: https://doi.org/10.1016/j.neucom.2021.10.101

IF: 6

2022-02-01

Neurocomputing

Abstract:Named Entity Recognition(NER), one of the most fundamental problems in natural language processing, seeks to identify the boundaries and types of entities with specific meanings in natural language text. As an important international language, Chinese has uniqueness in many aspects, and Chinese NER (CNER) is receiving increasing attention. In this paper, we give a comprehensive survey of recent advances in CNER. We first introduce some preliminary knowledge, including the common datasets, tag schemes, evaluation metrics and difficulties of CNER. Then, we separately describe recent advances in traditional research and deep learning research of CNER, in which the CNER with deep learning is our focus. We summarize related works in a basic three-layer architecture, including character representation, context encoder, and context encoder and tag decoder. Meanwhile, the attention mechanism and adversarial-transfer learning methods based on this architecture are introduced. Finally, we present the future research trends and challenges of CNER.

computer science, artificial intelligence
DeepTagger: Knowledge Enhanced Named Entity Recognition for Web-Based Ads Queries

Simiao Zuo,Pengfei Tang,Xinyu Hu,Qiang Lou,Jian Jiao,Denis Charles

DOI: https://doi.org/10.1145/3583780.3615467

2023-01-01

Abstract:Named entity recognition (NER) is a crucial task for online advertisement. State-of-the-art solutions leverage pre-trained language models for this task. However, three major challenges remain unresolved: web queries differ from natural language, on which pretrained models are trained; web queries are short and lack contextual information; and labeled data for NER is scarce. We propose DeepTagger, a knowledge-enhanced NER model for web-based ads queries. The proposed knowledge enhancement framework leverages both model-free and model-based approaches. For model-free enhancement, we collect unlabeled web queries to augment domain knowledge; and we collect web search results to enrich the information of ads queries. We further leverage effective prompting methods to automatically generate labels using large language models such as ChatGPT. Additionally, we adopt a model-based knowledge enhancement method based on adversarial data augmentation. We employ a three-stage training framework to train DeepTagger models.
ATBBC: Named entity recognition in emergency domains based on joint BERT-BILSTM-CRF adversarial training

Buqing Cai,Shengwei Tian,Long Yu,Jun Long,Tiejun Zhou,Bo Wang

DOI: https://doi.org/10.3233/jifs-232385

2023-12-28

Journal of Intelligent & Fuzzy Systems

Abstract:With the rapid growth of Internet penetration, identifying emergency information from network news has become increasingly significant for emergency monitoring and early warning. Although deep learning models have been commonly used in Chinese Named Entity Recognition (NER), they require a significant amount of well-labeled training data, which is difficult to obtain for emergencies. In this paper, we propose an NER model that combines bidirectional encoder representations from Transformers (BERT), bidirectional long-short-term memory (BILSTM), and conditional random field (CRF) based on adversarial training (ATBBC) to address this issue. Firstly, we constructed an emergency dataset (ED) based on the classification and coding specifications of the national emergency platform system. Secondly, we utilized the BERT pre-training model with adversarial training to extract text features. Finally, BILSTM and CRF were used to predict the probability distribution of entity labels and decode the probability distribution into corresponding entity labels.Experiments on the ED show that our model achieves an F1-score of 85.39% on the test dataset, which proves the effectiveness of our model.
Named entity recognition based on semi-supervised ensemble learning with the improved tri-training algorithm

TengFei Ma,QuanSheng Dou,Ping Jiang,Huan Liu

DOI: https://doi.org/10.1145/3446999.3447002

2020-12-25

Abstract:Named entity recognition is one of the hot topics in natural language processing. The purpose is to identify named entities in text and summarize them into corresponding entity types. The deep learning model has achieved good results for the task of named entity recognition. Due to the lack of annotated corpus data for named entity recognition tasks in specific domains, it is difficult for a single deep learning model to achieve excellent performance. To this end, by ensembling Conditional Random Field (CRF) model, Bidirectional Gated Recurrent Unit (BiGRU) network model, and Bidirectional Long Short-Term Memory (BiLSTM) network model, with collaborative training Tri -training algorithm, we proposed a named entity recognition method based on semi-supervised ensemble learning (NER-SSEL). It aims to improve model performance through iterative training with a small amount of labeled data and a large amount of unlabeled data. First, a small amount of labeled data is used for pre-training on the three base learners, and then a collaborative training tri-training algorithm is used to provide reliable labels for the unlabeled raw data. To avoid the introduction of noisy data, we propose a repeated labeling strategy to select high-confidence samples and iteratively expand the training set through the consistency evaluation function. Finally, the model is integrated through the weighted voting method. This article conducts experiments on the ATIS data set. Experimental results show that the model proposed in this paper can effectively use a large amount of unlabeled data and significantly improve the F1 measure in the task of named entity recognition.
Enhancing Chinese Named Entity Recognition with Disentangled Expert Knowledge

Hongkai Wang,Jun Feng,Yidan Wang,Sichen Pan,Shuai Zhao,Yi Xue

DOI: https://doi.org/10.1007/978-981-99-9614-8_6

2024-01-01

Abstract:Chinese Named Entity Recognition (NER) requires model identify entity boundaries in the sentence i.e., entity segmentation, and meanwhile assign entities to pre-defined categories, i.e., entity classification. Current NER tasks follows sequence tagging scheme and assign the characters to different labels by considering both segmentation position and entity categories. In such a scheme, the characters in the same entity will be regarded as different classes in the training process according to different positions. In fact, the knowledge of entity segmentation is shared across different entity categories, while entity category knowledge is relatively independent of entity segmentation. Such labeling scheme will lead to the entanglement of these two objectives, hindering the effective knowledge acquisition by the models. To address the entanglement issue and comprehensively extract useful knowledge of two objectives, we propose a novel framework that disentangle the original NER labels into two additional training labels for entity segmentation and entity classification respectively. Then we introduce two dedicated expert models to effectively extract specific knowledge from the disentangled labels. Afterwards, their predictions will be integrated into the original model as auxiliary knowledge, further enhancing the primary NER model learning process. We conduct experiments on three publicly available datasets to demonstrate the effectiveness of our proposed method.

GeoNER: Geological Named Entity Recognition with Enriched Domain Pre‐Training Model and Adversarial Training

Few-shot learning for name entity recognition in geological text based on GeoBERT

A Model for Geographical Named Entity Recognition and Correction Based on Deep Neural Networks and the BERT Model

Semi-supervised geological disasters named entity recognition using few labeled data

EduNER: a Chinese Named Entity Recognition Dataset for Education Research

Chinese mineral named entity recognition based on BERT model

Unsupervised cross-domain named entity recognition using entity-aware adversarial training

Enhanced Chinese Domain Named Entity Recognition: An Approach with Lexicon Boundary and Frequency Weight Features

A Toponym Entity Recognition Model for Chinese Social Media

Coarse-to-Fine Pre-training for Named Entity Recognition

ERNIE-GeoL: A Geography-and-Language Pre-trained Model and its Applications in Baidu Maps

A deep neural network model for Chinese toponym matching with geographic pre-training model

Beyond extraction accuracy: addressing the quality of geographical named entity through advanced recognition and correction models using a modified BERT framework

Named entity recognition for Chinese based on global pointer and adversarial training

Neural Chinese Named Entity Recognition via CNN-LSTM-CRF and Joint Training with Word Segmentation

MetaNER: Named Entity Recognition with Meta-Learning.

Chinese named entity recognition: The state of the art

DeepTagger: Knowledge Enhanced Named Entity Recognition for Web-Based Ads Queries

ATBBC: Named entity recognition in emergency domains based on joint BERT-BILSTM-CRF adversarial training

Named entity recognition based on semi-supervised ensemble learning with the improved tri-training algorithm

Enhancing Chinese Named Entity Recognition with Disentangled Expert Knowledge