Abstract:Macro-expression spotting is an important prior step in many dynamic facial expression analysis applications. It automatically detects the onset and offset image frames of a macro-expression in the video. The state-of-the-art methods of macro-expression spotting characterize the movement of facial muscle through explicit analysis of the optical flow map and have achieved promising results. However, optical flow map estimation and expression spotting in these methods are performed in two separate and successive stages. In this paper, we propose a new dual-branch network to achieve unified optimization for expression spotting and optical flow estimation tasks. The proposed dual-branch network implicitly learns optical flow during training and enriches the feature representation with motion information. During inference, we use only the encoder of the optical flow estimation network for motion feature extraction and integrate it with expression spotting into a one-stage framework. The proposed method eliminates the need to construct optical flow maps explicitly during inference and significantly reduces the computational cost. We also apply a consistency constraint on the global- and local-level semantic features of the clip to guide the model to focus on the category-consistent regions of the video. We evaluate the proposed methods extensively on two popular facial expression spotting datasets, CAS(ME) and SAMM Long Videos. The experimental results show that compared with the state-of-the-art methods, the proposed method improves the F1-scores for MaE spotting by 5.81 and 1.57 on the CAS(ME) and SAMM Long Videos datasets respectively.

Spatio-temporal Fusion for Macro- and Micro-expression Spotting in Long Video Sequences

Efficient Micro-Expression Spotting Based on Main Directional Mean Optical Flow Feature

Integrating VideoMAE based model and Optical Flow for Micro- and Macro-expression Spotting

SpotFormer: Multi-Scale Spatio-Temporal Transformer for Facial Expression Spotting

Micro-Expression Spotting with Face Alignment and Optical Flow

A Multi-scale Feature Learning Network with Optical Flow Correction for Micro- and Macro-expression Spotting

Research on Micro-Expression Spotting Method Based on Optical Flow Features

A Magnitude and Angle Combined Optical Flow Feature for Microexpression Spotting

Spotting Micro-Expressions on Long Videos Sequences

Micro-expression Spotting with Multi-scale Local Transformer in Long Videos

A Unique M-pattern for Micro-Expreesion Spotting in Long Videos

3D Feature Extraction Network Based on Self-supervision for Micro-expression Spotting

3D-CNN for Facial Micro- and Macro-expression Spotting on Long Video Sequences using Temporal Oriented Reference Frame

Bidirectional Cross-scale Feature Fusion for Long Video Micro-Expression 3D Spotting Network

Spotting Macro- and Micro-expression Intervals in Long Video Sequences

Micro-expression Detection based on Action Units and Multi-region Feature Fusion.

Real-Time Micro-Expression Detection In Unlabeled Long Videos Using Optical Flow And Lstm Neural Network

Duration-aware and Mode-Aware Micro-Expression Spotting for Long Video Sequences

Deep Learning Pipeline for Spotting Macro- and Micro-expressions in Long Video Sequences Based on Action Units and Optical Flow

A dual-branch network based on optical flow learning and semantic consistency for macro-expression spotting

ABPN: Apex and Boundary Perception Network for Micro- and Macro-Expression Spotting