PMAN: Progressive Multi-Attention Network for Human Pose Transfer

Baoyu Chen,Yi Zhang,Hongchen Tan,Baocai Yin,Xiuping Liu
DOI: https://doi.org/10.1109/tcsvt.2021.3059706
IF: 5.859
2021-01-01
IEEE Transactions on Circuits and Systems for Video Technology
Abstract:This paper presents a novel approach for human pose transfer, progressive multi-attention network (PMAN), which generates a new human image by transferring the pose of a given person to a target pose. The network gradually updates the pose feature and the image feature through a series of multi-attention transfer blocks (MATBs). Each MATB consists of two attention mechanisms: pose-conditioned batch normalization (PCBN) and cooperative attention mechanism (CAM). Specifically, in low-level feature space, the PCBN layer with pose information is used to replace the BN layer of the image channel to realize the preliminary guidance of pose to image. The CAM is implemented as two gated mechanisms in high-level feature space, which reveals the essence of human pose transfer, that is, mutual guidance and dynamic control between pose and image. Gated memory writing is used to calculate the pixel-wise weight of the pose by using global image information to guide the update of the pose. Gated response utilizes an adaptive gating mechanism to dynamically control the pose information flow so as to update the image. A large number of subjective and objective experiments on DeepFashion and Market-1501 demonstrate the superiority of our method. The proposed multi-attention mechanism is well adapted to the human pose transfer task and provides a possible new idea for other cross-domain generation tasks.
What problem does this paper attempt to address?