Latent representation discretization for unsupervised text style generation
Yang Gao,Qianhui Liu,Yizhe Yang,Ke Wang
DOI: https://doi.org/10.1016/j.ipm.2024.103643
IF: 7.466
2024-01-31
Information Processing & Management
Abstract:Language models , such as BART and GPT, have been shown to be highly effective at producing quality headlines. However, without clear guidelines for what constitutes a particular writing style, they may generate text that does not meet the desired style criteria (i.e., attention-grabbing), even if the resulting text is grammatically correct and semantically coherent. In this study, we introduce a novel approach called Discretized Style Transfer (DST) for unsupervised style transfer. We argue that the textual style signal is inherently abstract and separate from the text itself. Therefore, we discretize the style representation into a discrete space, where each discrete point corresponds to a particular category of style that can be elicited by the syntactic structure. To evaluate the effectiveness of our approach, we propose two new automatic evaluation metrics along with several conventional criteria, especially STR metric is nearly 0.9 in TechST, 0.87 in GYAFC datasets, and the best PPL metrics. Furthermore, we conduct thorough human evaluations by directly measuring click-through rates as an indicator of attractiveness, showing our model receives the most popularity. Our results demonstrate that DST achieves competitive performance on style transfer and can effectively capture the written structure of specified styles. This approach has the potential to significantly enhance its relevance and is capable of generating appealing content.
computer science, information systems,information science & library science