Abstract:Highlights • The performance of identification algorithms is susceptible to many factors. • Head rotation beyond a 15-degree angle pose challenges to automated systems. • An upward tilt of 10° and higher leads to a notable degradation in performance. • Only a smiling face produced consistently positive log-likelihood ratios. • There is evidence of sex-related bias favoring males in image processing. In today's biometric and commercial settings, state-of-the-art image processing relies solely on artificial intelligence and machine learning which provides a high level of accuracy. However, these principles are deeply rooted in abstract, complex "black-box systems". When applied to forensic image identification, concerns about transparency and accountability emerge. This study explores the impact of two challenging factors in automated facial identification: facial expressions and head poses. The sample comprised 3D faces with nine prototype expressions, collected from 41 participants (13 males, 28 females) of European descent aged 19.96 to 50.89 years. Pre-processing involved converting 3D models to 2D color images (256 × 256 px). Probes included a set of 9 images per individual with head poses varying by 5° in both left-to-right (yaw) and up-and-down (pitch) directions for neutral expressions. A second set of 3,610 images per individual covered viewpoints in 5° increments from −45° to 45° for head movements and different facial expressions, forming the targets. Pair-wise comparisons using ArcFace, a state-of-the-art face identification algorithm yielded 54,615,690 dissimilarity scores. Results indicate that minor head deviations in probes have minimal impact. However, the performance diminished as targets deviated from the frontal position. Right-to-left movements were less influential than up and down, with downward pitch showing less impact than upward movements. The lowest accuracy was for upward pitch at 45°. Dissimilarity scores were consistently higher for males than for females across all studied factors. The performance particularly diverged in upward movements, starting at 15. Among tested facial expressions, happiness and contempt performed best, while disgust exhibited the lowest AUC values.

The Criminality From Face Illusion

Automated Inference on Criminality Using Face Images.

Social Judgments from Faces Are Universal

Automated Inference on Sociopsychological Impressions of Attractive Female Faces

A set of distinct facial traits learned by machines is not predictive of appearance bias in the wild

A Data-Theoretic Approach to Identifying Violent Facial Expressions in Social Crime Contexts

Accuracy, Bias, and Overgeneralization: Perceived Aggression Guides Threat Detection and Punishment of Female Criminal Offenders

An Evaluation of Forensic Facial Recognition

Foundations for Unfairness in Anomaly Detection -- Case Studies in Facial Imaging Data

Understanding Fairness of Gender Classification Algorithms Across Gender-Race Groups

Head poses and grimaces: Challenges for automated face identification algorithms?

On the advantages of using AI-generated images of filler faces for creating fair lineups

Synthetic Counterfactual Faces

Responses to Critiques on Machine Learning of Criminality Perceptions (Addendum of <a class="link-https" data-arxiv-id="1611.04135" href="https://arxiv.org/abs/1611.04135">arXiv:1611.04135</a>)

AI Hyperrealism: Why AI Faces Are Perceived as More Real Than Human Ones

Your Face Mirrors Your Deepest Beliefs-Predicting Personality and Morals through Facial Emotion Recognition

Role of human physiology and facial biomechanics towards building robust deepfake detectors: A comprehensive survey and analysis

Responses to Critiques on Machine Learning of Criminality Perceptions (addendum of Arxiv:1611.04135)

Image Counterfactual Sensitivity Analysis for Detecting Unintended Bias

Evading Forensic Classifiers with Attribute-Conditioned Adversarial Faces

DL MODEL FOR IMITATION FALSE FACE, AGE, GENDER AND SCENARIO DISPARITIES