An End-to-End Workflow using Topic Segmentation and Text Summarisation Methods for Improved Podcast Comprehension

Andrew Aquilina,Sean Diacono,Panagiotis Papapetrou,Maria Movin

2023-07-25

Abstract:The consumption of podcast media has been increasing rapidly. Due to the lengthy nature of podcast episodes, users often carefully select which ones to listen to. Although episode descriptions aid users by providing a summary of the entire podcast, they do not provide a topic-by-topic breakdown. This study explores the combined application of topic segmentation and text summarisation methods to investigate how podcast episode comprehension can be improved. We have sampled 10 episodes from Spotify's English-Language Podcast Dataset and employed TextTiling and TextSplit to segment them. Moreover, three text summarisation models, namely T5, BART, and Pegasus, were applied to provide a very short title for each segment. The segmentation part was evaluated using our annotated sample with the $P_k$ and WindowDiff ($WD$) metrics. A survey was also rolled out ($N=25$) to assess the quality of the generated summaries. The TextSplit algorithm achieved the lowest mean for both evaluation metrics ($\bar{P_k}=0.41$ and $\bar{WD}=0.41$), while the T5 model produced the best summaries, achieving a relevancy score only $8\%$ less to the one achieved by the human-written titles.

Information Retrieval

What problem does this paper attempt to address?

### Problems the Paper Attempts to Solve

This paper aims to improve the understanding of podcast episodes by combining topic segmentation and text summarization methods. Specifically:

1. **Research Background**:
- The consumption of podcast media is rapidly increasing, but podcast episodes are usually long, requiring users to carefully choose before deciding to listen.
- Although episode descriptions provide an overview of the entire episode, they do not offer a breakdown of individual topics.

2. **Main Objectives**:
- Explore how to enhance user understanding and navigation by automatically segmenting podcast episodes and generating short titles for each segment.
- Evaluate the effectiveness of different algorithms (such as TextTiling and TextSplit) for topic segmentation and compare the quality of summaries generated by different text summarization models (such as T5, BART, and Pegasus).

3. **Specific Methods**:
- Use TextTiling and TextSplit for topic segmentation and use T5, BART, and Pegasus to generate summaries.
- Evaluate the performance of segmentation algorithms using a manually annotated dataset and assess the relevance of generated summaries through surveys.

4. **Experimental Results**:
- The TextSplit algorithm performed the best in topic segmentation, with the lowest average error.
- The T5 model produced the highest quality summaries, closely matching the quality of human-written titles.

In summary, this paper aims to improve the readability and information retrieval efficiency of podcast episodes through automated tools.