SegRewardGraph: unsupervised teaching video story segmentation method based on subtitle length-rewarding strategy and semantic relatedness graphs
Zhi Liu,Hao Chen,Xi Kong,Chaodong Wen,Jia Chen,Sannyuya Liu,Zongkai Yang
DOI: https://doi.org/10.1007/s11042-023-17523-y
IF: 2.577
2023-11-05
Multimedia Tools and Applications
Abstract:Story segmentation plays an important role in helping students quickly locate key knowledge points within massive and fragmented teaching video resources. It also serves as the basis for accurate searching of valuable teaching video clips. While keyframe-based story segmentation has achieved remarkable results in news videos, achieving accurate story segmentation in teaching videos is challenging. Therefore, we propose an unsupervised teaching video story segmentation method called SegRewardGraph , which utilizes a subtitle length-rewarding strategy and semantic relatedness graphs. SegRewardGraph employs subtitle semantic similarity to divide video stories, ensuring semantic integrity and improving the accuracy of story segmentation. Specifically, SegRewardGraph first uses the Bidirectional Encoder Representation from Transformers (BERT) model combined with first-last-avg pooling to encode semantic vectors for sentences within video stories. Then, it computes similarities between all sentence vectors and utilizes the associations to build semantic relatedness graphs. A subtitle length-rewarding strategy is formulated to evaluate the segmentation effect. Additionally, boundary detection and boundary merging algorithms are designed based on the subtitle length-rewarding strategy to generate effective segmentation suggestions. Finally, the selected boundaries are mapped to keyframes, enabling semantic content-based segmentation of teaching videos. This study verifies the effectiveness of the proposed method using massive open online courses (MOOC) teaching video datasets. Experimental results demonstrate that the model proposed in this paper outperforms existing methods and achieves state-of-the-art results for teaching video segmentation tasks.
computer science, information systems, theory & methods,engineering, electrical & electronic, software engineering