Abstract:We propose a novel framework for modeling the interaction between graphical structures and the natural language text associated with their nodes and edges. Existing approaches typically fall into two categories. On group ignores the relational structure by converting them into linear sequences and then utilize the highly successful Seq2Seq models. The other side ignores the sequential nature of the text by representing them as fixed-dimensional vectors and apply graph neural networks. Both simplifications lead to information loss. Our proposed method utilizes both the graphical structure as well as the sequential nature of the texts. The input to our model is a set of text segments associated with the nodes and edges of the graph, which are then processed with a transformer encoder-decoder model, equipped with a self-attention mechanism that is aware of the graphical relations between the nodes containing the segments. This also allows us to use BERT-like models that are already trained on large amounts of text. While the proposed model has wide applications, we demonstrate its capabilities on data-to-text generation tasks. Our approach compares favorably against state-of-the-art methods in four tasks without tailoring the model architecture. We also provide an early demonstration in a novel practical application -- generating clinical notes from the medical entities mentioned during clinical visits.

Refining Data for Text Generation.

Step-by-Step: Separating Planning from Realization in Neural Data-to-Text Generation

Data-To-Text Generation With Attention Recurrent Unit

Beyond Word for Word: Fact Guided Training for Neural Data-to-Document Generation

Point Precisely: Towards Ensuring the Precision of Data in Generated Texts Using Delayed Copy Mechanism

Neural data-to-text generation with dynamic content planning

Data-to-text Generation with Entity Modeling

Data-to-Text Generation with Iterative Text Editing

A Semi-Supervised Approach for Low-Resourced Text Generation.

Operation-guided Neural Networks for High Fidelity Data-To-Text Generation

Order-Planning Neural Text Generation from Structured Data

An Encoder with non-Sequential Dependency for Neural Data-to-Text Generation.

Have Your Text and Use It Too! End-to-End Neural Data-to-Text Generation with Semantic Fidelity

Repetition In Repetition Out: Towards Understanding Neural Text Degeneration from the Data Perspective

Improving Neural Question Generation using Deep Linguistic Representation

Table-to-Text Generation with Effective Hierarchical Encoder on Three Dimensions (Row, Column and Time)

Enhancing Text Generation in Joint NLG/NLU Learning Through Curriculum Learning, Semi-Supervised Training, and Advanced Optimization Techniques

Select and Attend: Towards Controllable Content Selection in Text Generation

Neural Text Generation from Rich Semantic Representations

Data2Text Studio: Automated Text Generation from Structured Data.

R2D2: Relational Text Decoding with Transformers