Abstract:Data-to-text generation plays an important role in natural language processing by processing structured data and helping people understand those data by generating user-friendly descriptive text. It can be applied to news generation, financial report generation, customer service, etc. However, in practice, it needs to adapt to different domains that may lack an annotated training corpus. To alleviate this dataset scarcity problem, distantly-supervised data-to-text generation has emerged, which constructs a training corpus automatically and is more practical to apply to new domains when well-aligned data is expensive to obtain. However, this distant supervision method of training induces an over-generation problem since the automatically aligned text includes hallucination. These expressions cannot be inferred from the data, misguiding the model to produce unfaithful text. To exploit the noisy dataset while maintaining faithfulness, we empower the neural data-to-text model by dynamically increasing the weights of those well-aligned training instances and reducing the weights of the low-quality ones via meta learning. To our best knowledge, we are the first to alleviate the noise in distantly-supervised data-to-text generation via meta learning. In addition, we rewrite those low-quality texts to provide better training instances. Finally, we construct a new distantly-supervised dataset, DIST-ToTTo (abbreviation for Distantly-supervised Table-To-Text), and conduct experiments on both the benchmark WITA (abbreviation for the data source Wikipedia and Wikidata) and DIST-ToTTo datasets. The evaluation results show that our model can improve the state-of-the-art DSG (abbreviation for Distant Supervision Generation) model across all automatic evaluation metrics, with an improvement of 3.72% on the WITA dataset and 3.82% on the DIST-ToTTo dataset in terms of the widely used metric BLEU (abbreviation for BiLingual Evaluation Understudy). Furthermore, based on human evaluation, our model can generate more grammatically correct and more faithful text compared to the state-of-the-art DSG model.

Biomedical Data-to-Text Generation via Fine-Tuning Transformers

Biomedical Data-to-Text Generation via Fine-Tuning Transformers

A Survey on Neural Data-to-Text Generation

Have Your Text and Use It Too! End-to-End Neural Data-to-Text Generation with Semantic Fidelity

BioGPT: Generative Pre-trained Transformer for Biomedical Text Generation and Mining

Fine-Tuning Transformers For Genomic Tasks

Training Optimus Prime, M.D.: Generating Medical Certification Items by Fine-Tuning OpenAI's gpt2 Transformer Model

Data Mining in Clinical Trial Text: Transformers for Classification and Question Answering Tasks

Evaluation of ChatGPT on Biomedical Tasks: A Zero-Shot Comparison with Fine-Tuned Generative Transformers

Better Synthetic Data by Retrieving and Transforming Existing Datasets

Quality Control for Distantly-Supervised Data-to-Text Generation Via Meta Learning

DiffuD2T: Empowering Data-to-Text Generation with Diffusion

Transformers and the Representation of Biomedical Background Knowledge

Reshaping Free-Text Radiology Notes Into Structured Reports With Generative Transformers

Towards Fine-Dining Recipe Generation with Generative Pre-trained Transformers

Understanding Transformers for Bot Detection in Twitter

BioMedGPT: Open Multimodal Generative Pre-trained Transformer for BioMedicine

BioMedGPT: Open Multimodal Generative Pre-trained Transformer for BioMedicine