Gaze estimation via bilinear pooling-based attention networks

Dakai Ren,Jiazhong Chen,Jian Zhong,Zhaoming Lu,Tao Jia,Zongyi Li

DOI: https://doi.org/10.1016/j.jvcir.2021.103369

IF: 2.887

2021-01-01

Journal of Visual Communication and Image Representation

Abstract:Attention mechanism has been found effective for human gaze estimation, and the attention and diversity of learned features are two important aspects of attention mechanism. However, the traditional attention mechanism used in existing gaze model is more prone to utilize first-order information that is attentive but not diverse. Though the existing bilinear pooling-based attention could overcome the shortcoming of traditional attention, it is limited to extract high-order contextual information. Thus we introduce a novel bilinear poolingbased attention mechanism, which could extract the second-order contextual information by the interaction between local deep learned features. To make the gaze-related features robust for spatial misalignment, we further propose an attention-in-attention method, which consists of a global average pooling and an inner attention on the second-order features. For the purpose of gaze estimation, a new bilinear pooling-based attention networks with attention-in-attention is further proposed. Extensive evaluation shows that our method surpasses the state-of-the-art by a big margin.