Abstract:Sentiment and emotion detection from textual communication records of developers have various application scenarios in software engineering (SE). However, commonly used off-the-shelf sentiment/emotion detection tools cannot obtain reliable results in SE tasks and misunderstanding of technical knowledge is demonstrated to be the main reason. Then researchers start to create labeled SE-related datasets manually and customize SE-specific methods. However, the scarce labeled data can cover only very limited lexicon and expressions. In this article, we employ emojis as an instrument to address this problem. Different from manual labels that are provided by annotators, emojis are self-reported labels provided by the authors themselves to intentionally convey affective states and thus are suitable indications of sentiment and emotion in texts. Since emojis have been widely adopted in online communication, a large amount of emoji-labeled texts can be easily accessed to help tackle the scarcity of the manually labeled data. Specifically, we leverage Tweets and GitHub posts containing emojis to learn representations of SE-related texts through emoji prediction. By predicting emojis containing in each text, texts that tend to surround the same emoji are represented with similar vectors, which transfers the sentiment knowledge contained in emoji usage to the representations of texts. Then we leverage the sentiment-aware representations as well as manually labeled data to learn the final sentiment/emotion classifier via transfer learning. Compared to existing approaches, our approach can achieve significant improvement on representative benchmark datasets, with an average increase of 0.036 and 0.049 in macro-F1 in sentiment and emotion detection, respectively. Further investigations reveal that the large-scale Tweets make a key contribution to the power of our approach. This finding informs future research not to unilaterally pursue the domain-specific resource but try to transform knowledge from the open domain through ubiquitous signals such as emojis. Finally, we present the open challenges of sentiment and emotion detection in SE through a qualitative analysis of texts misclassified by our approach.

Emojilization

SEntiMoji: An Emoji-Powered Learning Approach for Sentiment Analysis in Software Engineering

Odor Emoticon: an Olfactory Application That Conveys Emotions.

Emojinize: Enriching Any Text with Emoji Translations

VoiceMoji: A Novel On-Device Pipeline for Seamless Emoji Insertion in Dictation

Laugh Now Cry Later: Controlling Time-Varying Emotional States of Flow-Matching-Based Zero-Shot Text-to-Speech

Emo-Tts:Parallel Transformer-based Text-to-Speech Model with Emotional Awareness

iEmoTTS: Toward Robust Cross-Speaker Emotion Transfer and Control for Speech Synthesis Based on Disentanglement Between Prosody and Timbre

EmoSpeech: A Corpus of Emotionally Rich and Contextually Detailed Speech Annotations

Facial Expression-Enhanced TTS: Combining Face Representation and Emotion Intensity for Adaptive Speech

Cross-Platform Emoji Interpretation: Analysis, a Solution, and Applications

Emoji-powered Sentiment and Emotion Detection from Software Developers’ Communication Data

Toward Synthesizing Expressive Mandarin Speech

Emojifying Chatbot Interactions: an Exploration of Emoji Utilization in Human-Chatbot Communications

The expression of emotions by text and speech

Emojis Decoded: Leveraging ChatGPT for Enhanced Understanding in Social Media Communications

EmoKnob: Enhance Voice Cloning with Fine-Grained Emotion Control

EmojiLM: Modeling the New Emoji Language

Joint Emoji Classification And Embedding Learning

MsEmoTTS: Multi-scale emotion transfer, prediction, and control for emotional speech synthesis

An Empirical Study on the Effect of Emoticon Stickers Usage on Information Communication in Social Media: Based on Speech Act Theory