Abstract:Gaze estimation, which has a wide range of applications in many scenarios, is a challenging task due to various unconstrained conditions. As information from both full-face and eye images is instrumental in improving gaze estimation, many multiregion gaze estimation models have been proposed in recent studies. However, most of them simply use the same regression method on both eye and face images, overlooking that the eye region may contribute more fine-grained features than the full-face region, and the variation in the left and right eyes of an individual caused by head pose, illumination, and partially occluded eye may lead to inconsistent estimations. To address these issues, we propose an appearance-based end-to-end learning network architecture with an attention mechanism, named efficient gaze network (EG-Net), which employs a two-branch network for gaze estimation. Specifically, a base CNN is utilized for full-face images, while an efficient eye network (EE-Net), which is scaled up from the base CNN, is used for left- and right-eye images. EE-Net uniformly scales up the depth, width and resolution of the base CNN with a set of constant coefficients for eye feature extraction and adaptively weights the left- and right-eye images via an attention network according to its "image quality". Finally, features from the full-face image, two individual eye images and head pose vectors are fused to regress the eye gaze vectors. We evaluate our approach on 3 public datasets, the proposed EG-Net model achieves much better performance. In particular, our EG-Net-v4 model outperforms state-of-the-art approaches on the MPIIFaceGaze dataset, with prediction errors of 2.41 cm and 2.76 degrees in 2D and 3D gaze estimation, respectively. It also yields a performance improvement to 1.58 cm on GazeCapture and 4.55 degrees on EyeDIAP dataset, with 23.4 % and 14.2 % improvement over prior arts on the two datasets respectively. The code related to this project is open-source and available at https://github.com/wuxinmei/EE_Net.git .

NGDNet: Nonuniform Gaussian-label distribution learning for infrared head pose estimation and on-task behavior understanding in the classroom

GVGNet: Gaze-Directed Visual Grounding for Learning Under-Specified Object Referring Intention

PerimetryNet: A Multiscale Fine Grained Deep Network for Three-Dimensional Eye Gaze Estimation Using Visual Field Analysis

AGCNNs: Attention-guided Convolutional Neural Networks for Infrared Head Pose Estimation in Assisted Driving System

Context-Guided Adaptive Network for Efficient Human Pose Estimation.

FDN: Feature Decoupling Network for Head Pose Estimation.

Classroom Student Posture Recognition Based on an Improved High-Resolution Network.

ARHPE: Asymmetric Relation-aware Representation Learning for Head Pose Estimation in Industrial Human-machine Interaction

KGNet: Knowledge-Guided Networks for Category-Level 6D Object Pose and Size Estimation.

An Attention Detection System Based on Gaze Estimation Using Self-supervised Learning

2DHeadPose: A simple and effective annotation method for the head pose in RGB images and its dataset

EG-Net: Appearance-based eye gaze estimation using an efficient gaze network with attention mechanism

Different gaze direction (DGNet) collaborative learning for iris segmentation

LDCNet: Limb Direction Cues-aware Network for Flexible Human Pose Estimation in Industrial Behavioral Biometrics Systems

Gaze Estimation Approach Using Deep Differential Residual Network

R-Net: A Novel Fully Convolutional Network–based Infrared Image Segmentation Method for Intelligent Human Behavior Analysis

EG-HRNet: an Efficient High-Resolution Network Using Ghost-Modules for Human Pose Estimation

A Novel Convolutional Neural Network for Head Detection and Pose Estimation in Complex Environments from Single-Depth Images

CDGNet: Class Distribution Guided Network for Human Parsing

GazeGaussian: High-Fidelity Gaze Redirection with 3D Gaussian Splatting

Oriented Object Detector with Gaussian Distribution Cost Label Assignment and Task-Decoupled Head