Abstract:Sentiment analysis can be used to study an individual or a group’s emotions and attitudes towards other people and entities like products, services, or social events. With the advancements in the field of deep learning, the enormity of available information on internet, chiefly on social media, combined with powerful computing machines, it’s just a matter of time before artificial intelligence (AI) systems make their presence in every aspect of human life, making our lives more introspective. In this paper, we propose to implement a multimodal sentiment prediction system that can analyze the emotions predicted from different modal sources such as video, audio and text and integrate them to recognize the group emotions of the students in a classroom. Our experimental setup involves a digital video camera with microphones to capture the live video and audio feeds of the students during a lecture. The students are advised to provide their digital feedback on the lecture as ‘tweets’ on their twitter account addressed to the lecturer’s official twitter account. The audio and video frames are separated from the live streaming video using tools such as lame and ffmpeg. A twitter API was used to access and extract messages from twitter platform. The audio and video features are extracted using Mel-Frequency Cepstral Co-efficients (MFCC) and Haar Cascades classifier respectively. The extracted features are then passed to the Convolutional Neural Network (CNN) model trained on the FER2013 facial images database to generate the feature vector for classification of video-based emotions. A Recurrent Neural Network (RNN) with Long Short-Term Memory (LSTM), trained on speech emotion corpus database was used to train on the audio features. A lexicon-based approach with senti-word dictionary and learning based approach with custom dataset trained by Support Vector Machines (SVM) was used in the twitter-texts based approach. A decision-level fusion algorithm was applied on these three different modal schemes to integrate the classification results and deduce the overall group emotions of the students. The use-case of this proposed system will be in student emotion recognition, employee performance feedback, monitoring or surveillance-based systems. The implemented system framework was tested in a classroom environment during a live lecture and the predicted emotions demonstrated the classification accuracy of our approach.

Multi-dimensional Sentiment Classification in Online Learning Environment

SBTM: A Joint Sentiment and Behaviour Topic Model for Online Course Discussion Forums

Multi-Modal and Multi-Task Depression Detection with Sentiment Assistance

Educational Big Data Analytics Using Sentiment Analysis for Student Requirement Analysis on Courses

The Challenges of Online Teacher Education - Sentiment Analysis of Course Reviews

Research on MOOC Reviews Oriented Sentiment Analysis by Awareness of Emotional Distinctions

Sentiment analysis on massive open online course evaluations: A text mining and deep learning approach

Multimodal Decision-level Group Sentiment Prediction of Students in Classrooms

Bloom-epistemic and sentiment analysis hierarchical classification in course discussion forums

Sentiment analysis and opinion mining on educational data: A survey

Multi-class Emotion AI by reconstructing linguistic context of words

Summative Student Course Review Tool Based on Machine Learning Sentiment Analysis to Enhance Life Science Feedback Efficacy

Multi-aspect and Multi-class Based Document Sentiment Analysis of Educational Data Catering Accreditation Process

Student Sentiment Analysis and Classroom Feedback Prediction Using Deep Learning

Sentiment-enhanced learning model for online language learning system

Enhancing the Value of Large-Enrollment Course Evaluation Data Using Sentiment Analysis

Sentiment Analysis and Topic Mining Using a Novel Deep Attention-Based Parallel Dual-Channel Model for Online Course Reviews

A Hybrid Classification Algorithm to Classify Engineering Students' Problems and Perks

Understanding Student Sentiment on Mental Health Support in Colleges Using Large Language Models

Sentiment Analysis in Online Product Reviews: Mining Customer Opinions for Sentiment Classification

A Study on Emotion Analysis for Online Learning Based on Students' Feedback via Social Networks