Abstract:The wide adoption of Large language models (LLMs) makes their dependability a pressing concern. Detection of errors is the first step to mitigating their impact on a system and thus, efficient error detection for LLMs is an important issue. In many settings, the LLM is considered as a black box with no access to the internal nodes; this prevents the use of many error detection schemes that need access to the model's internal nodes. An interesting observation is that the output of LLMs in error-free operation should be valid and normal text. Therefore, when the text is not valid or differs significantly from normal text, it is likely that there is an error. Based on this observation we propose to perform Concurrent Linguistic Error Detection (CLED); this scheme extracts some linguistic features of the text generated by the LLM and feeds them to a concurrent classifier that detects errors. Since the proposed error detection mechanism only relies on the outputs of the model, then it can be used on LLMs in which there is no access to the internal nodes. The proposed CLED scheme has been evaluated on the T5 model when used for news summarization and on the OPUS-MT model when used for translation. In both cases, the same set of linguistic features has been used for error detection to illustrate the applicability of the proposed scheme beyond a specific case. The results show that CLED can detect most of the errors at a low overhead penalty. The use of the concurrent classifier also enables a trade-off between error detection effectiveness and its associated overhead, so providing flexibility to a designer.

Improving Low-Resource Chinese Event Detection with Multi-task Learning.

Cross-Lingual Text Image Recognition Via Multi-Task Sequence to Sequence Learning.

ELCA: Enhanced boundary location for Chinese named entity recognition via contextual association

Pre-trained Language Model Augmented Adversarial Training Network for Chinese Clinical Event Detection.

Neural Cross-Lingual Event Detection with Minimal Parallel Resources.

Event Detection with Trigger-Aware Lattice Neural Network

Adversarial Multi-Task Learning for Efficient Chinese Named Entity Recognition

Event Detection via Gated Multilingual Attention Mechanism.

A syntax-enhanced parameter generation network for multi-source cross-lingual event extraction

Extracting Chinese events with a joint label space model

Concurrent Linguistic Error Detection (CLED) for Large Language Models

CSMA-CNER:Multi-modal Chinese NER Task with Cross- and Self-Modality Attention

Collaborate SLM and LLM with latent answers for event detection

Adversarial training based lattice LSTM for Chinese clinical named entity recognition

MCL: Multi-Granularity Contrastive Learning Framework for Chinese NER

Multi-task Learning for Chinese Word Usage Errors Detection

A scene-dependent sound event detection approach using multi-task learning

Chinese Named Entity Recognition Method Combining ALBERT and a Local Adversarial Training and Adding Attention Mechanism

Leveraging Multi-Head Attention Mechanism To Improve Event Detection

Neural Chinese Named Entity Recognition via CNN-LSTM-CRF and Joint Training with Word Segmentation

Multi-task learning for Chinese clinical named entity recognition with external knowledge