Abstract:Headline generation is an important problem in natural language processing, which aims to describe a document by a compact and informative headline. Some recent successes on this task have been achieved by advanced graph-based neural models, which marry the representational power of deep neural networks with the structural modeling ability of the relational sentence graphs. The advantages of graph-based neural models over traditional Seq2Seq models lie in that they can encode long-distance relationship between sentences beyond the surface linear structure. However, since documents are typically weakly-structured data, modern graph-based neural models usually rely on manually designed rules or some heuristics to construct the sentence graph a prior. This may largely limit the power and increase the cost of the graph-based methods. In this paper, therefore, we propose to incorporate structure learning into the graph-based neural models for headline generation. That is, we want to automatically learn the sentence graph using a data-driven way, so that we can unveil the document structure flexibly without prior heuristics or rules. To achieve this goal, we employ a deep & wide network to encode rich relational information between sentences for the sentence graph learning. For the deep component, we leverage neural matching models, either representation-focused or interaction-focused model, to learn semantic similarity between sentences. For the wide component, we encode a variety of discourse relations between sentences. A Graph Convolutional Network (GCN) is then applied over the sentence graph to generate high-level relational representations for headline generation. The whole model could be optimized end-to-end so that the structure and representation could be learned jointly. Empirical studies show that our model can significantly outperform the state-of-the-art headline generation models.

Syntax-guided Text Generation Via Graph Neural Network

Explicit Syntactic Guidance for Neural Text Generation

Top-Down Tree Structured Text Generation.

Augmenting Paraphrase Generation with Syntax Information Using Graph Convolutional Networks.

Syntax-type-aware Graph Convolutional Networks for Natural Language Understanding.

Tree-Structured Neural Machine for Linguistics-Aware Sentence Generation

Sentence Modeling via Graph Construction and Graph Neural Networks for Semantic Textual Similarity

Reinforcement Learning Based Graph-to-Sequence Model for Natural Question Generation

Text Generation with Syntax - Enhanced Variational Autoencoder.

Graph Transformer Networks Based Text Representation.

Enriched Entity Representation of Knowledge Graph for Text Generation

Dependency-based N-Gram Models for General Purpose Sentence Realisation

GraphSpeech: Syntax-Aware Graph Attention Network For Neural Speech Synthesis

Syntax-guided question generation using prompt learning

GraphSeq2Seq: Graph-Sequence-to-Sequence for Neural Machine Translation

GraphTTS: graph-to-sequence modelling in neural text-to-speech

Text Generation Based on Generative Adversarial Nets with Latent Variable

Structure Learning for Headline Generation

A Semi-Autoregressive Graph Generative Model for Dependency Graph Parsing

Improving Chinese Story Generation via Awareness of Syntactic Dependencies and Semantics

Embedding Text-Rich Graph Neural Networks with Sequence and Topical Semantic Structures