CopyFormer: A Transformer Text Generation model combined with Pointer Network.
Xingyu Ma,Songfeng Lu,Hu Liu,Bingyan Feng
DOI: https://doi.org/10.1145/3594315.3594368
2023-01-01
Abstract:Industrial Internet Research Institute Wuhan Huazhong Numerical Control Co., Ltd.Wuhan, 430074, China In the Pointer Generation Network, the model uses the attention mechanism in Seq2Seq [1]to selectively point to the text that needs to be copied, thus effectively alleviating the problem of out of vocabulary (OOV). However, Seq2Seq has some defects, so Google proposed a new Encoder-Decoder architecture Transformer [2], to replace the traditional Seq2Seq. This paper proposes a Pointer Generation Network model based on Transformer, which can effectively improve the generation ability of the model by using the powerful model of Transformer combined with Pointer Generation Network. In addition, considering that the Pointer Generation Network does not effectively use the relevant information in the input sequence, this paper selectively combines the information in the input sequence to the Decoder, so as to improve the effect of the model. Finally, experiments on Similar Text Generation tasks verify the effectiveness of the method.