Abstract:Gaze estimation, which has a wide range of applications in many scenarios, is a challenging task due to various unconstrained conditions. As information from both full-face and eye images is instrumental in improving gaze estimation, many multiregion gaze estimation models have been proposed in recent studies. However, most of them simply use the same regression method on both eye and face images, overlooking that the eye region may contribute more fine-grained features than the full-face region, and the variation in the left and right eyes of an individual caused by head pose, illumination, and partially occluded eye may lead to inconsistent estimations. To address these issues, we propose an appearance-based end-to-end learning network architecture with an attention mechanism, named efficient gaze network (EG-Net), which employs a two-branch network for gaze estimation. Specifically, a base CNN is utilized for full-face images, while an efficient eye network (EE-Net), which is scaled up from the base CNN, is used for left- and right-eye images. EE-Net uniformly scales up the depth, width and resolution of the base CNN with a set of constant coefficients for eye feature extraction and adaptively weights the left- and right-eye images via an attention network according to its "image quality". Finally, features from the full-face image, two individual eye images and head pose vectors are fused to regress the eye gaze vectors. We evaluate our approach on 3 public datasets, the proposed EG-Net model achieves much better performance. In particular, our EG-Net-v4 model outperforms state-of-the-art approaches on the MPIIFaceGaze dataset, with prediction errors of 2.41 cm and 2.76 degrees in 2D and 3D gaze estimation, respectively. It also yields a performance improvement to 1.58 cm on GazeCapture and 4.55 degrees on EyeDIAP dataset, with 23.4 % and 14.2 % improvement over prior arts on the two datasets respectively. The code related to this project is open-source and available at https://github.com/wuxinmei/EE_Net.git .

An end-to-end convolutional network for estimating the essential matrix

Estimating the fundamental matrix based on the end-to-end convolutional network

End-to-End Spatial Transform Face Detection and Recognition

MCNet: Rethinking the Core Ingredients for Accurate and Efficient Homography Estimation

An End to End Network Architecture for Fundamental Matrix Estimation

Consensus Learning with Deep Sets for Essential Matrix Estimation

EANet: Towards Lightweight Human Pose Estimation With Effective Aggregation Network

E$^3$-Net: Efficient E(3)-Equivariant Normal Estimation Network

Optimized S2E Attention Block based Convolutional Network for Human Pose Estimation

EG-Net: Appearance-based eye gaze estimation using an efficient gaze network with attention mechanism

A Deconvolutional Bottom-up Deep Network for Multi-Person Pose Estimation.

Efficient Dynamic Correspondence Network

FM-Net: Deep Learning Network for the Fundamental Matrix Estimation from Biplanar Radiographs.

Self-supervised Monocular Depth Estimation Based on Combining Convolution and Multilayer Perceptron

PT-Net: Pyramid Transformer Network for Feature Matching Learning

Human Pose Estimation Based on Efficient and Lightweight High-Resolution Network (EL-HRNet)

ExpNet: Landmark-Free, Deep, 3D Facial Expressions

Effective Local-Global Transformer for Natural Image Matting

EPT-Net: Edge Perception Transformer for 3D Medical Image Segmentation

EM-Net: Gaze Estimation with Expectation Maximization Algorithm

CMLP-Net: A Convolution-Multilayer Perceptron Network for EEG-based Emotion Recognition