Triterpenes from Sebastiania schottiana.

R. Yunes,J. Calixto,O. Miguel,F. Monache

DOI: https://doi.org/10.1055/S-2006-960942

IF: 3.007

1990-04-01

Planta Medica

Abstract:

What problem does this paper attempt to address?

Micro-Expression Recognition by Motion Feature Extraction based on Pre-training

Ruolin Li,Lu Wang,Tingting Yang,Lisheng Xu,Bingyang Ma,Yongchun Li,Hongchao Wei

2024-07-10

Abstract:Micro-expressions (MEs) are spontaneous, unconscious facial expressions that have promising applications in various fields such as psychotherapy and national security. Thus, micro-expression recognition (MER) has attracted more and more attention from researchers. Although various MER methods have emerged especially with the development of deep learning techniques, the task still faces several challenges, e.g. subtle motion and limited training data. To address these problems, we propose a novel motion extraction strategy (MoExt) for the MER task and use additional macro-expression data in the pre-training process. We primarily pretrain the feature separator and motion extractor using the contrastive loss, thus enabling them to extract representative motion features. In MoExt, shape features and texture features are first extracted separately from onset and apex frames, and then motion features related to MEs are extracted based on the shape features of both frames. To enable the model to more effectively separate features, we utilize the extracted motion features and the texture features from the onset frame to reconstruct the apex frame. Through pre-training, the module is enabled to extract inter-frame motion features of facial expressions while excluding irrelevant information. The feature separator and motion extractor are ultimately integrated into the MER network, which is then fine-tuned using the target ME data. The effectiveness of proposed method is validated on three commonly used datasets, i.e., CASME II, SMIC, SAMM, and CAS(ME)3 dataset. The results show that our method performs favorably against state-of-the-art methods.

Computer Vision and Pattern Recognition
Facial Micro-Expressions: an Overview

Guoying Zhao,Xiaobai Li,Yante Li,Matti Pietikainen

DOI: https://doi.org/10.1109/jproc.2023.3275192

IF: 20.6

2023-01-01

Proceedings of the IEEE

Abstract:Micro-expression (ME) is an involuntary, fleeting, and subtle facial expression. It may occur in high-stake situations when people attempt to conceal or suppress their true feelings. Therefore, MEs can provide essential clues to people's true feelings and have plenty of potential applications, such as national security, clinical diagnosis, and interrogations. In recent years, ME analysis has gained much attention in various fields due to its practical importance, especially automatic ME analysis in computer vision as MEs are difficult to process by naked eyes. In this survey, we provide a comprehensive review of ME development in the field of computer vision, from the ME studies in psychology and early attempts in computer vision to various computational ME analysis methods and future directions. Four main tasks in ME analysis are specifically discussed, including ME spotting, ME recognition, ME action unit detection, and ME generation in terms of the approaches, advance developments, and challenges. Through this survey, readers can understand MEs in both aspects of psychology and computer vision, and apprehend the future research direction in ME analysis.
Extended Local Binary Patterns for Efficient and Robust Spontaneous Facial Micro-Expression Recognition

Chengyu Guo,Jingyun Liang,Geng Zhan,Zhong Liu,Matti Pietikäinen,Li Liu

DOI: https://doi.org/10.48550/arXiv.1907.09160

2019-07-22

Computer Vision and Pattern Recognition

Abstract:Facial Micro-Expressions (MEs) are spontaneous, involuntary facial movements when a person experiences an emotion but deliberately or unconsciously attempts to conceal his or her genuine emotions. Recently, ME recognition has attracted increasing attention due to its potential applications such as clinical diagnosis, business negotiation, interrogations, and security. However, it is expensive to build large scale ME datasets, mainly due to the difficulty of inducing spontaneous MEs. This limits the application of deep learning techniques which require lots of training data. In this paper, we propose a simple, efficient yet robust descriptor called Extended Local Binary Patterns on Three Orthogonal Planes (ELBPTOP) for ME recognition. ELBPTOP consists of three complementary binary descriptors: LBPTOP and two novel ones Radial Difference LBPTOP (RDLBPTOP) and Angular Difference LBPTOP (ADLBPTOP), which explore the local second order information along the radial and angular directions contained in ME video sequences. ELBPTOP is a novel ME descriptor inspired by unique and subtle facial movements. It is computationally efficient and only marginally increases the cost of computing LBPTOP, yet is extremely effective for ME recognition. In addition, by firstly introducing Whitened Principal Component Analysis (WPCA) to ME recognition, we can further obtain more compact and discriminative feature representations, then achieve significantly computational savings. Extensive experimental evaluation on three popular spontaneous ME datasets SMIC, CASME II and SAMM show that our proposed ELBPTOP approach significantly outperforms the previous state-of-the-art on all three single evaluated datasets and achieves promising results on cross-database recognition.Our code will be made available.
Revealing concealed spontaneous facial micro-expression: Are we a step closer to unveil real-life behavioral expressions?

Y.S. Gan,Gen-Bing Liong,Kun-Hong Liu,Sze-Teng Liong

DOI: https://doi.org/10.1016/j.neucom.2023.03.055

IF: 6

2023-04-13

Neurocomputing

Abstract:Facial micro-expression (ME) expression analysis has aroused interdisciplinary attention from the realms of computer vision and psychology due to its advantages of divulging nonverbal emotion via subtle involuntary facial muscle changes. However, developing and commercializing ME recognition relevant products has been hindered in current progress due to the scarcity of databases depicting real-world conditions. This paper aims to give a strong impetus to facilitating the advancement of ME recognition systems, particularly in in-the-wild applications. Succinctly, an efficient recognition system is introduced herein, which incorporates the core processes like 3D facial reconstruction, apex spotting, and emotion recognition tasks. Concretely, all faces are first rendered into 3D point cloud form for the brevity of face alignment along the video. Then, the optical flow-guided components are employed as the primary features to represent the motion details. Subsequently, a 3-Stream channel based on 3-Dimensional faces Network (3T3D-Net) with skip connection via multiplication is tailored to cope with the small-size input image. As a result, pleasing recognition performance is yielded via the suite of novel techniques devised, by producing a UAR of 69.44% and UF1 of 70.71%, when evaluated on the in-the-wild ME dataset, viz, MEVIEW. In addition, extensive studies and analyses are presented to verify the contribution of each component of the proposed pipeline. It is envisioned that the remarkable result attained by casting an invigorating perspective will shed light and pave a new prospect for analysis in ME applications.

computer science, artificial intelligence
Boosting Micro-Expression Recognition Via Self-Expression Reconstruction and Memory Contrastive Learning

Yongtang Bao,Chenxi Wu,Peng Zhang,Caifeng Shan,Yue Qi,Xianye Ben

DOI: https://doi.org/10.1109/taffc.2024.3397701

IF: 13.99

2024-01-01

IEEE Transactions on Affective Computing

Abstract:Micro-expression (ME) is an instinctive reaction that is not controlled by thoughts. It reveals one's inner feelings, which is significant in sentiment analysis and lie detection. Since micro-expression is expressed as subtle facial changes within particular facial action units, learning discriminative and generalized features for Micro-expression Recognition (MER) is challenging. To achieve the purpose, this paper proposes a novel MER framework that simultaneously integrates supervised Prototype-based Memory Contrastive Learning (PMCL) for discriminative feature mining and adds Self-expression Reconstruction (SER) as an auxiliary task and regularization for better generalization. In particular, the proposed SER module is forced as a regularization by reconstructing input ME from the randomly dropped patch- wise features in the bottleneck. And, the PMCL module globally compares historical and current cluster agents learned from training instances to enhance intra-class compactness and inter-class separability. Extensive experiments are conducted on three benchmarks, e.g., SMIC, CASME II, and SAMM, under evaluation criteria of both Composite Database Evaluation (CDE) and Single Database Evaluation (SDE) protocols. The results show our method surpasses other state-of-the-art approaches under various evaluation metrics, achieving overall 86.30% unweighed F1-score and 88.30% unweighed average recall on the composite dataset. Furthermore, the ablation studies verify the effectiveness of our SER for better generalization and PMCL for better discrimination in learning feature representation from limited micro-expression samples.
A Two-Stage 3D CNN Based Learning Method for Spontaneous Micro-Expression Recognition

Sirui Zhao,Hanqing Tao,Yangsong Zhang,Tong Xu,Kun Zhang,Zhongkai Hao,Enhong Chen

DOI: https://doi.org/10.1016/j.neucom.2021.03.058

IF: 6

2021-01-01

Neurocomputing

Abstract:Micro-expressions (MEs) are spontaneous and involuntary facial subtle reactions which often reveal the genuine emotions within human beings. Recognizing MEs automatically is becoming increasingly crucial for many areas such as diagnosis and security. However, the short time duration and low spatial intensity of MEs pose great challenges for accurately recognizing them. Additionally, the lack of sufficient and balanced spontaneous MEs data also makes this problem even harder to solve, and some adaptive modeling strategies have been quite urgent recently. To this end, this paper draws inspirations from few-shot learning to propose a novel two-stage learning (i.e., prior learning and target learning) method based on a siamese 3D convolutional neural network for MEs recognition (MERSiamC3D). Specifically, in the prior learning stage, the proposed MERSiamC3D is used to extract the generic features of MEs. In the target learning stage, the structure and parameters of the MERSiamC3D will be carefully adjusted and the Focal Loss is adopted for high-level features learning. Afterwards, in order to effectively retain the spatiotemporal information of the original MEs video, an adaptive construction method based on adaptive convolutional neural network is proposed to construct the key-frames sequence to summarize the original MEs video, which is able to help drop the redundant frames and relatively highlight the movement of the apex frame. Then, the new key-frames are taken as the input of the two-stage learning method. Finally, through extensive evaluations and experiments on three publically available MEs datasets, the proposed method in this work could outperform traditional methods and other deep learning baselines, which provides a novel insight on how to leverage scarce data for MEs recognition. (c) 2021 Elsevier B.V. All rights reserved.
MMNet: Muscle motion-guided network for micro-expression recognition

Hanting Li,Mingzhe Sui,Zhaoqing Zhu,Feng Zhao

DOI: https://doi.org/10.48550/arXiv.2201.05297

2022-08-19

Abstract:Facial micro-expressions (MEs) are involuntary facial motions revealing peoples real feelings and play an important role in the early intervention of mental illness, the national security, and many human-computer interaction systems. However, existing micro-expression datasets are limited and usually pose some challenges for training good classifiers. To model the subtle facial muscle motions, we propose a robust micro-expression recognition (MER) framework, namely muscle motion-guided network (MMNet). Specifically, a continuous attention (CA) block is introduced to focus on modeling local subtle muscle motion patterns with little identity information, which is different from most previous methods that directly extract features from complete video frames with much identity information. Besides, we design a position calibration (PC) module based on the vision transformer. By adding the position embeddings of the face generated by PC module at the end of the two branches, the PC module can help to add position information to facial muscle motion pattern features for the MER. Extensive experiments on three public micro-expression datasets demonstrate that our approach outperforms state-of-the-art methods by a large margin.

Computer Vision and Pattern Recognition
Micro-expression recognition based on depth map to point cloud

Ren Zhang,Jianqin Yin,Chao Qi,Zehao Wang,Zhicheng Zhang,Yonghao Dang

2024-06-12

Abstract:Micro-expressions are nonverbal facial expressions that reveal the covert emotions of individuals, making the micro-expression recognition task receive widespread attention. However, the micro-expression recognition task is challenging due to the subtle facial motion and brevity in duration. Many 2D image-based methods have been developed in recent years to recognize MEs effectively, but, these approaches are restricted by facial texture information and are susceptible to environmental factors, such as lighting. Conversely, depth information can effectively represent motion information related to facial structure changes and is not affected by lighting. Motion information derived from facial structures can describe motion features that pixel textures cannot delineate. We proposed a network for micro-expression recognition based on facial depth information, and our experiments have demonstrated the crucial role of depth maps in the micro-expression recognition task. Initially, we transform the depth map into a point cloud and obtain the motion information for each point by aligning the initiating frame with the apex frame and performing a differential operation. Subsequently, we adjusted all point cloud motion feature input dimensions and used them as inputs for multiple point cloud networks to assess the efficacy of this representation. PointNet++ was chosen as the ultimate outcome for micro-expression recognition due to its superior performance. Our experiments show that our proposed method significantly outperforms the existing deep learning methods, including the baseline, on the $CAS(ME)^3$ dataset, which includes depth information.

Image and Video Processing
More is Better: A Database for Spontaneous Micro-Expression with High Frame Rates

Sirui Zhao,Huaying Tang,Xinglong Mao,Shifeng Liu,Hanqing Tao,Hao Wang,Tong Xu,Enhong Chen

DOI: https://doi.org/10.48550/arXiv.2301.00985

2023-01-03

Abstract:As one of the most important psychic stress reactions, micro-expressions (MEs), are spontaneous and transient facial expressions that can reveal the genuine emotions of human beings. Thus, recognizing MEs (MER) automatically is becoming increasingly crucial in the field of affective computing, and provides essential technical support in lie detection, psychological analysis and other areas. However, the lack of abundant ME data seriously restricts the development of cutting-edge data-driven MER models. Despite the recent efforts of several spontaneous ME datasets to alleviate this problem, it is still a tiny amount of work. To solve the problem of ME data hunger, we construct a dynamic spontaneous ME dataset with the largest current ME data scale, called DFME (Dynamic Facial Micro-expressions), which includes 7,526 well-labeled ME videos induced by 671 participants and annotated by more than 20 annotators throughout three years. Afterwards, we adopt four classical spatiotemporal feature learning models on DFME to perform MER experiments to objectively verify the validity of DFME dataset. In addition, we explore different solutions to the class imbalance and key-frame sequence sampling problems in dynamic MER respectively on DFME, so as to provide a valuable reference for future research. The comprehensive experimental results show that our DFME dataset can facilitate the research of automatic MER, and provide a new benchmark for MER. DFME will be published via <a class="link-external link-https" href="https://mea-lab-421.github.io" rel="external noopener nofollow">this https URL</a>.

Computer Vision and Pattern Recognition
4DME: A Spontaneous 4D Micro-Expression Dataset with Multimodalities

Xiaobai Li,Shiyang Cheng,Yante Li,Muzammil Behzad,Jie Shen,Stefanos Zafeiriou,Maja Pantic,Guoying Zhao

DOI: https://doi.org/10.1109/taffc.2022.3182342

IF: 13.99

2023-01-01

IEEE Transactions on Affective Computing

Abstract:Micro-expressions (ME) are a special form of facial expressions which may occur when people try to hide their true feelings for some reasons. MEs are important clues to reveal people's true feelings, but are difficult or impossible to be captured by ordinary persons with naked-eyes as they are very short and subtle. It is expected that robust computer vision methods can be developed to automatically analyze MEs which requires lots of ME data. The current ME datasets are insufficient, and mostly contain only one single form of 2D color videos. Researches on 4D data of ordinary facial expressions have prospered, but so far no 4D data is available in ME study. In the current study, we introduce the 4DME dataset: a new spontaneous ME dataset which includes 4D data along with three other video modalities. Both micro- and macro-expression clips are labeled out in 4DME, and 22 AU labels and five categories of emotion labels are annotated. Experiments are carried out using three 2D-based methods and one 4D-based method to provide baseline results. The results indicate that the 4D data can potentially benefit ME recognition. The 4DME dataset could be used for developing 4D-based approaches, or exploring fusion of multiple video sources (e.g., texture and depth) for the task of ME analysis in future. Besides, we also emphasize the importance of forming a clear and unified criteria of ME annotation for future ME data collection studies. Several key questions related with ME annotation are listed and discussed in depth, especially about the relationship between AUs and ME emotion categories. A preliminary AU-Emo mapping table is proposed with justified explanations and supportive experimental results. Several unsolved issues are also summarized for future work.
Geometric Graph Representation with Learnable Graph Structure and Adaptive AU Constraint for Micro-Expression Recognition

Jinsheng Wei,Wei Peng,Guanming Lu,Yante Li,Jingjie Yan,Guoying Zhao

DOI: https://doi.org/10.48550/arXiv.2205.00380

2023-03-03

Abstract:Micro-expression recognition (MER) is valuable because micro-expressions (MEs) can reveal genuine emotions. Most works take image sequences as input and cannot effectively explore ME information because subtle ME-related motions are easily submerged in unrelated information. Instead, the facial landmark is a low-dimensional and compact modality, which achieves lower computational cost and potentially concentrates on ME-related movement features. However, the discriminability of facial landmarks for MER is unclear. Thus, this paper explores the contribution of facial landmarks and proposes a novel framework to efficiently recognize MEs. Firstly, a geometric two-stream graph network is constructed to aggregate the low-order and high-order geometric movement information from facial landmarks to obtain discriminative ME representation. Secondly, a self-learning fashion is introduced to automatically model the dynamic relationship between nodes even long-distance nodes. Furthermore, an adaptive action unit loss is proposed to reasonably build the strong correlation between landmarks, facial action units and MEs. Notably, this work provides a novel idea with much higher efficiency to promote MER, only utilizing graph-based geometric features. The experimental results demonstrate that the proposed method achieves competitive performance with a significantly reduced computational cost. Furthermore, facial landmarks significantly contribute to MER and are worth further study for high-efficient ME analysis.

Computer Vision and Pattern Recognition,Artificial Intelligence
A Boost in Revealing Subtle Facial Expressions: A Consolidated Eulerian Framework

Wei Peng,Xiaopeng Hong,Yingyue Xu,Guoying Zhao

DOI: https://doi.org/10.1109/FG.2019.8756541

2019-01-23

Abstract:Facial Micro-expression Recognition (MER) distinguishes the underlying emotional states of spontaneous subtle facialexpressions. Automatic MER is challenging because that 1) the intensity of subtle facial muscle movement is extremely lowand 2) the duration of ME is <a class="link-external link-http" href="http://transient.Recent" rel="external noopener nofollow">this http URL</a> works adopt motion magnification or time interpolation to resolve these issues. Nevertheless, existing works dividethem into two separate modules due to their non-linearity. Though such operation eases the difficulty in implementation, itignores their underlying connections and thus results in inevitable losses in both accuracy and speed. Instead, in this paper, weexplore their underlying joint formulations and propose a consolidated Eulerian framework to reveal the subtle facial <a class="link-external link-http" href="http://movements.It" rel="external noopener nofollow">this http URL</a> expands the temporal duration and amplifies the muscle movements in micro-expressions simultaneously. Compared toexisting approaches, the proposed method can not only process ME clips more efficiently but also make subtle ME movementsmore distinguishable. Experiments on two public MER databases indicate that our model outperforms the state-of-the-art inboth speed and accuracy.

Computer Vision and Pattern Recognition
MEGC2022

Jingting Li,Moi Hoon Yap,Wen-Huang Cheng,John See,Xiaopeng Hong,Xiaobai Li,Sujing Wang,Adrian K. Davison,Yante Li,Zizhao Dong

DOI: https://doi.org/10.1145/3503161.3551601

2022-01-01

Abstract:Facial micro-expressions (MEs) are involuntary movements of the face that occur spontaneously when a person experiences an emotion but attempts to suppress or repress the facial expression, typically found in a high-stakes environment. Unfortunately, the small sample problem severely limits the automation of ME analysis. Furthermore, due to the brief and subtle nature of ME, ME spotting is a challenging task, and the performance is still not satisfactory yet. This challenge focuses on two tasks, i.e., the micro- and macro-expression spotting task, and the ME Generation task.
Feature Representation Learning with Adaptive Displacement Generation and Transformer Fusion for Micro-Expression Recognition

Zhijun Zhai,Jianhui Zhao,Chengjiang Long,Wenju Xu,Shuangjiang He,Huijuan Zhao

2023-04-10

Abstract:Micro-expressions are spontaneous, rapid and subtle facial movements that can neither be forged nor suppressed. They are very important nonverbal communication clues, but are transient and of low intensity thus difficult to recognize. Recently deep learning based methods have been developed for micro-expression (ME) recognition using feature extraction and fusion techniques, however, targeted feature learning and efficient feature fusion still lack further study according to the ME characteristics. To address these issues, we propose a novel framework Feature Representation Learning with adaptive Displacement Generation and Transformer fusion (FRL-DGT), in which a convolutional Displacement Generation Module (DGM) with self-supervised learning is used to extract dynamic features from onset/apex frames targeted to the subsequent ME recognition task, and a well-designed Transformer Fusion mechanism composed of three Transformer-based fusion modules (local, global fusions based on AU regions and full-face fusion) is applied to extract the multi-level informative features after DGM for the final ME prediction. The extensive experiments with solid leave-one-subject-out (LOSO) evaluation results have demonstrated the superiority of our proposed FRL-DGT to state-of-the-art methods.

Computer Vision and Pattern Recognition
Inceptr: micro-expression recognition integrating inception-CBAM and vision transformer

Haoliang Zhou,Shucheng Huang,Yuqiao Xu

DOI: https://doi.org/10.1007/s00530-023-01164-0

IF: 3.9

2023-09-01

Multimedia Systems

Abstract:Micro-Expressions (MEs) are the instantaneous and subtle facial movement that conveys crucial emotional information. However, traditional neural networks face difficulties in accurately capturing the delicate features of MEs due to the limited amount of available data. To address this issue, a dual-branch attention network is proposed for ME recognition, called IncepTR, which can capture attention-aware local and global representations. The network takes optical flow features as input and performs feature extraction using a dual-branch network. First, the Inception model based on the Convolutional Block Attention Module (CBAM) attention mechanism is maintained for multi-scale local feature extraction. Second, the Vision Transformer (ViT) is employed to capture subtle motion features and robustly model global relationships among multiple local patches. Additionally, to enhance the rich relationships between different local patches in ViT, Multi-head Self-Attention Dropping (MSAD) is introduced to drop an attention map randomly, effectively preventing overfitting to specific regions. Finally, the two types of features could be used to learn ME representations effectively through similarity comparison and feature fusion. With such combination, the model is forced to capture the most discriminative multi-scale local and global features while reducing the influence of affective-irrelevant features. Extensive experiments show that the proposed IncepTR achieves UF1 and UAR of 0.753 and 0.746 on the composite dataset MEGC2019-CD, demonstrating better or competitive performance compared to existing state-of-the-art methods for ME recognition.

computer science, information systems, theory & methods
MFDAN: Multi-level Flow-Driven Attention Network for Micro-Expression Recognition

Wenhao Cai,Junli Zhao,Ran Yi,Minjing Yu,Fuqing Duan,Zhenkuan Pan,Yong-Jin Liu

DOI: https://doi.org/10.1109/tcsvt.2024.3437481

IF: 5.859

2024-01-01

IEEE Transactions on Circuits and Systems for Video Technology

Abstract:Facial expressions are an essential part of human emotional communication, and micro-expressions (MEs), as transient and imperceptible non-verbal signals, can potentially reveal real human emotions. However, subtle motion variations, limited and unbalanced samples make micro-expression recognition (MER) challenging. In this paper, we design a novel dual-branch learning framework of multi-level flow-driven attention for micro-expression recognition (MFDAN), which innovatively integrates optical flow prior to guide the attention learning in the image encoding branch, enabling the model to focus on the most discriminative facial regions for subtle motion patterns. Firstly, we extract optical flow information by an optical flow encoding module. Then, in the image coding module, we construct a Transformer structure containing an optical flow-driven attention mechanism, which can effectively locate the interest region of micro-expressions in the image according to the position information of optical flow to capture more sensitive and fine-grained micro-expressions. By interoperating prior knowledge with data learning, and introducing the Dropkey operation and Focal Loss, our method can handle subtle micro-expression features on small imbalanced datasets. Through extensive experiments on three independent datasets and a composite database, including SMIC-HS, SAMM, and CASME II, robust leave-one-subject-out (LOSO) evaluation results show that our method outperforms state-of-the-art methods especially on the composite database.
The heterogeneous ensemble of deep forest and deep neural networks for micro-expressions recognition

Meng-Xin Sun,Sze-Teng Liong,Kun-Hong Liu,Qing-Qiang Wu

DOI: https://doi.org/10.1007/s10489-022-03284-y

IF: 5.3

2022-03-24

Applied Intelligence

Abstract:Micro-Expressions (MEs) are a kind of short-lived and uncontrollable facial expressions. Therefore, the MEs recognition task poses a great challenge to both the psychological and computer vision research communities. In this study, a new ensemble algorithm is proposed by fusing two different deep learning frameworks: Deep Forest (DF) and Convolutional Neural Networks (CNN) (DFN for short). A modified DF structure is deployed to extract features through the multi-grained scanning technique, along with three different sliding windows to gain diverse motion features. In addition, two shallow CNNs are deployed to extract the features from the three-dimensional optical flow vector and the apex frame. In this way, the fusion of DF and CNNs forms DFN to extract the static and dynamic features for MEs, to generate diverse features with high-level abstraction. Consequently, this heterogeneous ensemble deploys the high diversity in these two models to promote the overall discriminative ability. Comprehensive experiments have confirmed the robustness and effectiveness of the proposed DFN with relatively less computational consumption. Related theoretical analysis has been given to further provide evidence and insights into the proposed method. Our source code is publicly available for non-commercial or research use at https://github.com/MLDMXM2017/DFN_ME.

computer science, artificial intelligence
Highly Effective End-to-End Single-to-Multichannel Feature Fusion and Ensemble Classification to Decode Emotional Secretes from Small-Scale Spontaneous Facial Micro-Expressions

Sajjatul Islam,Yongsheng Sang,Adam A. Q. Mohammed,Lei Yuan,Jiancheng Lv,MD Sajjatul Islam

DOI: https://doi.org/10.1016/j.jksuci.2023.101653

IF: 9.006

2023-07-19

Journal of King Saud University - Computer and Information Sciences

Abstract:Facial Micro-Expression (ME) is one of the pre-dominating non-verbal clues to demystify the true emotional states that people try to conceal cautiously. But emotion recognition from spontaneous ME images limits the high accuracy due to short duration and low intensity, lack of sufficient samples and consistencies among publicly available ME datasets. In this study, we have proposed two highly effective, lightweight, and generalized single-channel DLRRF-MER, and multi-channel DLH-3C-FUSION fusion models inspired by deep dense convolutional models and texture-based feature descriptors Local Binary Pattern (LBP) and Histogram of Oriented Gradients (HOG) to recognize ME from apex frame by constructing a composite dataset from five publicly available ME datasets CASME, CASMEII, CAS(ME) 2 , SAMM and MMEW. Pre-training has been done on a new composition of five facial macro expressions datasets CK+, MUGFE, OuluCasia, SFEW, and RAF-DB. The proposed models are fine-tuned on the target ME dataset rigorously with Stratified 5-Fold and 10-Fold, Leave-One-Subject-Out, and Leave-One-Dataset-Out(LODO) cross-validations(CV). In all evaluations, our proposed algorithms show remarkable improvement in effectiveness which surpasses the state-of-the-art accuracies and results in higher generalization capacity. To the best of our knowledge, it is the first attempt to use LODO CV in ME recognition from five ME datasets.

computer science, information systems
FME '22: 2ndWorkshop on Facial Micro-Expression: Advanced Techniques for Multi-Modal Facial Expression Analysis

Jingting Li,Moi Hoon Yap,Wen-Huang Cheng,John See,Xiaopeng Hong,Xiaobai Li,Su-Jing Wang

DOI: https://doi.org/10.1145/3503161.3554777

2022-01-01

Abstract:Micro-expressions are facial movements that are extremely short and not easily detected, which often reflect the genuine emotions of individuals. Micro-expressions are important cues for understanding real human emotions and can be used for non-contact non-perceptual deception detection, or abnormal emotion recognition. It has broad application prospects in national security, judicial practice, health prevention, clinical practice, etc. However, micro-expression feature extraction and learning are highly challenging because micro-expressions have the characteristics of short duration, low intensity, and local asymmetry. In addition, the intelligent micro-expression analysis combined with deep learning technology is also plagued by the problem of small samples. Not only is micro-expression elicitation very difficult, micro-expression annotation is also very time-consuming and laborious. More importantly, the micro-expression generation mechanism is not yet clear, which shackles the application of micro-expressions in real scenarios. FME'22 is the inaugural workshop in this area of research, with the aim of promoting interactions between researchers and scholars from within this niche area of research and also including those from broader, general areas of expression and psychology research. The complete FME'22 workshop proceedings are available at: https://dl.acm.org/doi/proceedings/10.1145/3552465.
An Overview of Facial Micro-Expression Analysis: Data, Methodology and Challenge

Hong-Xia Xie,Ling Lo,Hong-Han Shuai,Wen-Huang Cheng

DOI: https://doi.org/10.48550/arXiv.2012.11307

2020-12-21

Abstract:Facial micro-expressions indicate brief and subtle facial movements that appear during emotional communication. In comparison to macro-expressions, micro-expressions are more challenging to be analyzed due to the short span of time and the fine-grained changes. In recent years, micro-expression recognition (MER) has drawn much attention because it can benefit a wide range of applications, e.g. police interrogation, clinical diagnosis, depression analysis, and business negotiation. In this survey, we offer a fresh overview to discuss new research directions and challenges these days for MER tasks. For example, we review MER approaches from three novel aspects: macro-to-micro adaptation, recognition based on key apex frames, and recognition based on facial action units. Moreover, to mitigate the problem of limited and biased ME data, synthetic data generation is surveyed for the diversity enrichment of micro-expression data. Since micro-expression spotting can boost micro-expression analysis, the state-of-the-art spotting works are also introduced in this paper. At last, we discuss the challenges in MER research and provide potential solutions as well as possible directions for further investigation.

Computer Vision and Pattern Recognition,Machine Learning

Triterpenes from Sebastiania schottiana.

Micro-Expression Recognition by Motion Feature Extraction based on Pre-training

Facial Micro-Expressions: an Overview

Extended Local Binary Patterns for Efficient and Robust Spontaneous Facial Micro-Expression Recognition

Revealing concealed spontaneous facial micro-expression: Are we a step closer to unveil real-life behavioral expressions?

Boosting Micro-Expression Recognition Via Self-Expression Reconstruction and Memory Contrastive Learning

A Two-Stage 3D CNN Based Learning Method for Spontaneous Micro-Expression Recognition

MMNet: Muscle motion-guided network for micro-expression recognition

Micro-expression recognition based on depth map to point cloud

More is Better: A Database for Spontaneous Micro-Expression with High Frame Rates

4DME: A Spontaneous 4D Micro-Expression Dataset with Multimodalities

Geometric Graph Representation with Learnable Graph Structure and Adaptive AU Constraint for Micro-Expression Recognition

A Boost in Revealing Subtle Facial Expressions: A Consolidated Eulerian Framework

MEGC2022

Feature Representation Learning with Adaptive Displacement Generation and Transformer Fusion for Micro-Expression Recognition

Inceptr: micro-expression recognition integrating inception-CBAM and vision transformer

MFDAN: Multi-level Flow-Driven Attention Network for Micro-Expression Recognition

The heterogeneous ensemble of deep forest and deep neural networks for micro-expressions recognition

Highly Effective End-to-End Single-to-Multichannel Feature Fusion and Ensemble Classification to Decode Emotional Secretes from Small-Scale Spontaneous Facial Micro-Expressions

FME '22: 2ndWorkshop on Facial Micro-Expression: Advanced Techniques for Multi-Modal Facial Expression Analysis

An Overview of Facial Micro-Expression Analysis: Data, Methodology and Challenge