Abstract:Video-based facial expression recognition has received substantial attention over the past decade, while early expression detection (EED) is still a relatively new and challenging problem. The goal of EED is to identify an expression as quickly as possible after the expression starts and before it ends. This timely ability has many potential applications, ranging from human-computer interaction to security. The max-margin early event detector (MMED) is a well-known ranking model for early event detection. It can achieve competitive EED performance but suffers from several critical limitations: 1) MMED lacks flexibility in extracting useful information for segment comparison, which leads to poor performance in exploring the ranking relation between segment pairs; 2) the training process is slow due to the large number of constraints, and the memory requirement is also usually hard to satisfy; and 3) MMED is linear in nature, and hence may not be appropriate for data in a nonlinear feature space. To overcome these limitations, we propose an online multi-instance learning (MIL) framework for EED. In particular, the MIL technique is first introduced to generalize MMED, resulting in the proposed MIL-based EED (MIED), which is more general and flexible than MMED, since various instance construction and combination strategies can be adopted. To accelerate the training process, we reformulate MIED in the online setting and develop online multi-instance learning framework for EED (OMIED). To further exploit the nonlinear structure of the data distribution, we incorporate the kernel methods in OMIED, which results in the proposed online kernel multi-instance learning for early expression detection. Experiments on two popular and one challenging video-based expression data sets demonstrate both the efficiency and effectiveness of the proposed methods.

Scalable Misbehavior Detection in Online Video Chat Services.

Deep Learning Based Abnormal Behavior Detection for Elderly Healthcare Using Consumer Network Cameras

Multi Frame Obscene Video Detection with ViT

Video stream interception and erotic content filtering

Online Anomaly Detection over Live Social Video Streaming

Online Kernel-Based Structured Output SVM for Early Expression Detection

Reinforcement Learning-based Mobile Edge Computing and Transmission Scheduling for Video Surveillance

Early Expression Detection Via Online Multi-Instance Learning with Nonlinear Extension

Modeling Live Video Streaming: Real-Time Classification, QoE Inference, and Field Evaluation

Securing Behavior-based Opinion Spam Detection

An English video teaching classroom attention evaluation model incorporating multimodal information

An Empirical Study of Spam and Prevention Mechanisms in Online Video Chat Services

A Streaming Machine Learning Framework for Online Aggression Detection on Twitter

EmoChat: Bringing Multimodal Emotion Detection to Mobile Conversation

Detecting Abnormal Behaviors in Surveillance Videos Based on Fuzzy Clustering and Multiple Auto-Encoders

Online Social Network Data-Driven Early Detection on Short-Form Video Addiction

Combating Online Misinformation Videos: Characterization, Detection, and Future Directions

Ultra Low-Cost Two-Stage Multimodal System for Non-Normative Behavior Detection

Impact of Social Network Structure on Multimedia Fingerprinting Misbehavior Detection and Identification

Deep Convolution Network Based Emotion Analysis for Automatic Detection of Mild Cognitive Impairment in the Elderly

Adaptive Big Data Analytics for Deceptive Review Detection in Online Social Media.