Abstract:Multi-document summarization (MDS) generates a summary from a document set. Each document in a set describes topic-relevant concepts, while per document also has its unique contents. However, the document specificity receives little attention from existing MDS approaches. Neglecting specific information for each document limits the comprehensiveness of the generated summaries. To solve this problem, in this paper, we propose to disentangle the specific content from documents in one document set. The document-specific representations, which are encouraged to be distant from each other via a proposed orthogonal constraint, are learned by the specific representation learner. We provide extensive analysis and have interesting findings that specific information and document set representations contribute distinctive strengths and their combination yields a more comprehensive solution for the MDS. Also, we find that the common (i.e. shared) information could not contribute much to the overall performance under the MDS settings. Implemetation codes are available at <a class="link-external link-https" href="https://github.com/congboma/DisentangleSum" rel="external noopener nofollow">this https URL</a>.
What problem does this paper attempt to address?
This paper attempts to solve the problem of ignoring document - specific information in multi - document summarization (MDS). Existing methods often focus on the overall information of the document collection when generating summaries, while ignoring the unique content in each document. This neglect limits the comprehensiveness of the generated summaries because each document contains not only common information but also unique information that can distinguish it from other documents. These specific information include unique facts, opinions and details, which are crucial for obtaining a comprehensive understanding of the entire document set. Therefore, the paper proposes a new model - DisentangleSum, aiming to decouple the unique content of each document from the document set, thereby generating more comprehensive summaries.
### Main contributions of the paper:
1. **Propose the DisentangleSum model**: This model can decouple the specific information of each document in the multi - document summarization task, thereby generating more comprehensive summaries. To the best of the authors' knowledge, this is the first time that specific information has been considered in the deep - learning - based MDS task.
2. **Introduce the orthogonal constraint**: In order to encourage the document - specific representer to retain the document - specific information, an orthogonal constraint is proposed. This constraint promotes the perpendicularity between the document - specific representation vectors, ensuring their semantic separation.
3. **Experimental verification**: Experiments were carried out on two MDS datasets, and the results show the effectiveness of DisentangleSum. In addition, the author also conducted a comprehensive analysis from multiple perspectives to explore the working mechanism of the model and its applicable conditions.
### Method overview:
- **Problem definition**: Each document set contains multiple documents, and each document has both common information and specific information. DisentangleSum generates the specific representation \( S_i \) of each document through a specific encoder, and generates the overall representation \( F \) of the document set through a document - set encoder.
- **Specific representation learner**: Use a specific encoder with shared parameters to generate the specific representation of each document, and concatenate these representations to form the overall specific representation \( S \) of the document set.
- **Orthogonal constraint**: In order to ensure the distinguishability between specific representations, an orthogonal constraint is introduced. Calculate the squared norm of the inner product between specific representations, and reduce its complexity from quadratic to linear through a cyclic pairing loss function.
- **Overall objective function**: Combine the supervised cross - entropy loss of the generated summary and the orthogonal constraint loss of the specific representation to form the overall objective function \( L_{\text{total}}=L_{\text{gen}}+\beta\cdot L_{\text{spec}} \).
### Experimental results:
- **Coverage score**: DisentangleSum has the highest coverage score on the two datasets, indicating that the summaries it generates contain more information from the source documents.
- **Overall performance**: On multiple evaluation metrics (such as ROUGE - 1, ROUGE - 2, ROUGE - SU and BERTScore), the performance of DisentangleSum is better than that of other baseline models.
- **Manual evaluation**: In the manual evaluation of four aspects, namely concreteness, comprehensiveness, coherence and relevance, DisentangleSum also performs the best.
### Conclusion:
By decoupling the specific information of documents, DisentangleSum can generate more comprehensive and specific summaries, significantly improving the quality of multi - document summarization.