Deep Learning Models for Automatic Summarization

Pirmin Lemberger

DOI: https://doi.org/10.48550/arXiv.2005.11988

2020-05-25

Abstract:Text summarization is an NLP task which aims to convert a textual document into a shorter one while keeping as much meaning as possible. This pedagogical article reviews a number of recent Deep Learning architectures that have helped to advance research in this field. We will discuss in particular applications of pointer networks, hierarchical Transformers and Reinforcement Learning. We assume basic knowledge of Seq2Seq architecture and Transformer networks within NLP.

Computation and Language,Machine Learning

What problem does this paper attempt to address?

The problem that this paper attempts to solve is the challenge of text summarization. Specifically, the author reviews the progress in recent years in the field of natural language processing (NLP), especially in deep - learning models, for improving text summarization techniques. The paper mainly focuses on how to use deep - learning architectures, such as Pointer Networks, Hierarchical Transformers and Reinforcement Learning, to improve the quality and efficiency of automatic summarization.

### Specific problems solved by the paper include:

1. **Generating high - quality summaries**:
- How to generate summaries that are both faithful to the original text and highly generalizable.
- Diversity of summaries: For the same document, different summaries may all be reasonable.

2. **Evaluating summary quality**:
- There is a lack of an objective and comprehensive evaluation standard. Although existing evaluation metrics such as ROUGE scores are commonly used, they have limitations.
- How to more accurately evaluate the similarity between the generated summary and the manually - written reference summary.

3. **Limitations of datasets**:
- High - quality training data is scarce and costly to collect.
- Existing datasets (such as the CNN/Daily Mail dataset) have some biases, which affect the training effect of the model.

4. **Model innovation**:
- Propose new model architectures, such as Pointer - Generator Networks and Coverage Mechanism, to solve the problems of factual errors and repetitions in summary generation.
- Use Hierarchical Bidirectional Transformers (HIBERT) for document - level pre - training to improve the quality of extractive summaries.
- Use Reinforcement Learning to optimize summary generation so that it can better handle the problem of non - unique best summaries.

### Main contributions of the paper:

- **Reviews multiple deep - learning models**: Introduces the applications of Pointer Networks, Hierarchical Transformers and Reinforcement Learning in automatic summarization.
- **Proposes new solutions**: In view of the shortcomings of existing methods, some innovative techniques, such as Pointer - Generator Networks and Coverage Mechanism, are proposed.
- **Discusses future research directions**: Explores how to combine multi - modal data and domain knowledge to further improve the effect of automatic summarization.

Through these methods, the paper aims to promote the development of automatic summarization technology, making it more effective and reliable in practical applications.