Robust Optical Flow Estimation Method Based on Structure-Texture Aware Retinex Model and Its Application on Face Anti-Spoofing
Cai Zemin,Liao Xiaoxin,Lai Jianhuang,Chen Jun
DOI: https://doi.org/10.11834/jig.220778
2023-01-01
Journal of Image and Graphics
Abstract:Objective Optical flow estimation is essential for computer vision and image processing, which is focused on pixel-wise motions between consecutive images. It is beneficial for multiple research domains like target tracking, crowd flow segmentation, and human behavior analysis. In addition, optical flow can be as an effective tool for video-based motion extraction. Hence, it was investigated to find out the face spoofing-relevant motion clues. It is still challenged for accurate optical flow estimation under complex illumination circumstances although optical flow estimation has attracted wide attention in the field of computer vision. Most of dense optical flow estimation methods are based on variational iteration in terms of Horn and Schunck’s framework. The variational framework consists of data and regularization terms in common. Due to the uncontrolled illumination conditions are required to be clarified, the gradient constancy hypothesis can be adopted as the data term for variational model. The brightness constancy assumption can be optimized to a certain extent.However, neither brightness constancy assumption nor gradient constancy hypothesis effective in representing the complex illumination variations. As a result, the calculation of optical flow is mutual-benefited to illumination changes. Optical flow estimations are required to be regularized because of the inherent ill-posedness. The Tikhonov regularization is often based on the L2 norm, and it is minimized in terms of small-amplitude coefficients-distributed preservation, which can capture global patterns well in optical flow calculation. And, the minimization of the L1 norm is focused on more zero-or smallamplitude coefficients, and less large-amplitude ones. For discrete signals, L1 norm can give out better results than L2.L1 norm-based optical flow computation can be more precisely for mathematical modeling. To deal with regularization problems, conventional L1 and L2 norm is still related to current variational optical flow models. Robust penalty functionrelevant L1 or L2 norm regularization can be used to generate smooth flow field and preserve motion inconsistency. However, it could lose the fine-scale motion structures and produce excessive segmentation artifacts. The image pre-processing methods can also be carried out in estimation of optical flow while illumination changes cannot be ignored between two consecutive frames. Optical flow calculation can provide motion features in facial biometric systems as well, which is concerned for such domains like surveillance, access control and forensic investigations. However, one of the challenges of facial biometric systems is the high possibility of the systems being deceived or spoofed by non-real faces. Recent face antispoofing technique can be as an effective pathway for facial biometric systems.Method to strengthen video-based targets motion in relevance with illumination changes, our research is focused on structure-texture-perceptive retinex model and optical flow-robust estimation for human-facial anti-spoofing application. For retinex theory, to improve robustness of optical flow against non-uniform illumination, the components of illumination and reflectance in the image are separated by decoupling. Reflection and illumination components are separated from the image base on a stronger reflectivity constancy assumption. After that, the illumination component is filtered through a low pass filter and it is then integrated into the new optical flow model. Additionally, a smooth-sparse regularization constraint is adopted to preserve edges and improve the accuracy of optical flow estimation. Furthermore, the numerical implementation of the model is demonstrated.Result Comparative analysis is carried out with some state-of-the-art optical flow estimation methods, including the variational based methods and deep learning based approaches on 3 publicity datasets of Middleburry, MPI Sintel and KITTI 2015. The quantitative analysis is carried out in terms of average angular error（AAE）, average end-point error（EPE） and the Fl scores in comparison with other related optical flow computations. Moreover, to evaluate the robustness of optical flow with respect to variations of illumination using the datasets, we consider conducting simulations of illumination to the source images. To render synthetic images, illumination patterns of linear, sinusoidal, Gaussian and mixture of Gaussian are involved in. To simulate the regularity and variability of real-world illumination patterns, such parameters like additive factor, multiplicative factor and gamma correlation are used as well. The experimental results show that our model outperforms all other evaluated methods on the three public datasets and their synthetic versions with different illumination patterns. To verify the feasibility of illumination-invariant method, the calculation is applied to obtain human-facial optical flow motion features, and face liveness-detected experiments are conducted on the Institute of Automation, Chinese Academy of Sciences（CASIA） face anti-spoofing database. The proposed STARFlow method is compared to some popular anti-spoofing methods related to optical flow like weighted regularization transform（WRT） and ARFlow. The quantitative and comparative evaluation metrics is composed of the accuracy of face anti-spoofing classification and the half total error rate（HTER）.Similarly, to validate the illumination robustness of the proposed method under challenging illumination changes, four synthetic illumination patterns are also appended to the dataset. Experiments are carried out on the basis of the completed CASIA database and the four illumination environments.Conclusion In this study, a new variational optical flow estimation model is facilitated in terms of structure-texture aware retinex theory. Experimental results validate that the proposed model outperforms several state-of-the-art optical flow estimation approaches, including some variational based methods and deep learning based methods. Meanwhile, the proposed STARFlow method can achieve a potential illuminationinvariance in terms of face anti-spoofing under different illumination changes circumstances. The source code of this project is available at:https://github. com/Xiaoxin-Liao/STARFlow.