Abstract:This paper proposes an advanced multi-instance learning method with multi-features engineering and conservative optimization for engagement intensity prediction. It was applied to the EmotiW Challenge 2020 and the results demonstrated the proposed method's good performance. The task is to predict the engagement level when a subject-student is watching an educational video under a range of conditions and in various environments. As engagement intensity has a strong correlation with facial movements, upper-body posture movements and overall environmental movements in a given time interval, we extract and incorporate these motion features into a deep regression model consisting of layers with a combination of long short-term memory(LSTM), gated recurrent unit (GRU) and a fully connected layer. In order to precisely and robustly predict the engagement level in a long video with various situations such as darkness and complex backgrounds, a multi-features engineering function is used to extract synchronized multi-model features in a given period of time by considering both short-term and long-term dependencies. Based on these well-processed engineered multi-features, in the 1st training stage, we train and generate the best models covering all the model configurations to maximize validation accuracy. Furthermore, in the 2nd training stage, to avoid the overfitting problem attributable to the extremely small engagement dataset, we conduct conservative optimization by applying a single Bi-LSTM layer with only 16 units to minimize the overfitting, and split the engagement dataset (train + validation) with 5-fold cross validation (stratified k-fold) to train a conservative model. The proposed method, by using decision-level ensemble for the two training stages' models, finally win the second place in the challenge (MSE: 0.061110 on the testing set).

Moment-to-moment Engagement Prediction through the Eyes of the Observer: PUBG Streaming on Twitch

FaceEngage: Robust Estimation of Gameplay Engagement from User-Contributed (youtube) Videos

Detecting Video Game Player Burnout With the Use of Sensor Data and Machine Learning

AI-enabled prediction of video game player performance using the data from heterogeneous sensors

Analyzing Viewer Motivations and Engagement in Game Live Streaming Through Eye Tracking

E-Sports Talent Scouting Based on Multimodal Twitch Stream Data

Delving Deep into Engagement Prediction of Short Videos

Modelling Early User-Game Interactions for Joint Estimation of Survival Time and Churn Probability

Multi-source Data Multi-task Learning for Profiling Players in Online Games

Seeker: Topic-Aware Viewing Pattern Prediction in Crowdsourced Interactive Live Streaming

Predicting Churn in Online Games by Quantifying Diversity of Engagement

A Machine Learning Approach to Detect Strategic Behavior from Large-Population Observational Data Applied to Game Mode Prediction on a Team-Based Video Game

Interaction-Aware Watching Duration Prediction on Live Streaming Platforms.

Video Highlight Prediction Using Audience Chat Reactions

Scalable Psychological Momentum Forecasting in Esports

Predicting Outcomes in Video Games with Long Short Term Memory Networks

Eyes on the Game: Deciphering Implicit Human Signals to Infer Human Proficiency, Trust, and Intent

Advanced Multi-Instance Learning Method with Multi-features Engineering and Conservative Optimization for Engagement Intensity Prediction

Gambling engagement mechanisms in Twitch live streaming

Do I Have Your Attention: A Large Scale Engagement Prediction Dataset and Baselines

AI-enabled Prediction of eSports Player Performance Using the Data from Heterogeneous Sensors