Abstract:Increasing use of social media has resulted in many detrimental effects in youth. With very little control over multimodal content consumed on these platforms and the false narratives conveyed by these multimodal social media postings, such platforms often impact the mental well-being of the users. To reduce these negative effects of multimodal social media content, an important step is to understand creators' intent behind sharing content and to educate their social network of this intent. Towards this goal, we propose I ntent-o-meter , a perceived human intent prediction model for multimodal (image and text) social media posts. I ntent-o-meter models ideas from psychology and cognitive modeling literature, in addition to using the visual and textual features for an improved perceived intent prediction model. I ntent-o-meter leverages Theory of Reasoned Action (TRA) factoring in (i) the creator's attitude towards sharing a post, and (ii) the social norm or perception towards the multimodal post in determining the creator's intention. We also introduce I ntentgram , a dataset of 55K social media posts scraped from public Instagram profiles. We compare I ntent-o-meter with state-of-the-art intent prediction approaches on four perceived intent prediction datasets, Intentonomy, MDID, MET-Meme, and I ntentgram . We observe that leveraging TRA in addition to visual and textual features—as opposed to using only the latter–results in improved prediction accuracy by up to in Top-1 accuracy and in AUC on I ntentgram . In summary, we also develop a web browser application mimicking a popular social media platform and show users social media content overlaid with these intent labels. From our analysis, around users confirmed that tagging posts with intent labels helped them become more aware of the content consumed, and they would be open to experimenting with filtering content based on these labels. However, more extensive user evaluation is required to understand how adding such perceived intent labels mitigate the negative effects of social media.

What problem does this paper attempt to address?

### Problems the Paper Attempts to Solve This paper aims to address the negative impact of multimodal content (images and text) on social media platforms on adolescent mental health. Specifically, the researchers focus on understanding the intentions of content creators when sharing content and educating the audience about these intentions to reduce negative effects. #### Background Issues 1. **Impact of Social Media on Adolescents**: - The widespread use of social media has led to numerous mental health issues among adolescents, such as body image anxiety and depression. - Users often post content to elevate their social status, which can lead to the spread of false narratives. - When these false narratives are consumed by other users, it can trigger unnecessary negative social comparisons, thereby affecting mental health. 2. **Limitations of Existing Methods**: - Existing intention prediction models primarily rely on visual and textual features, lacking psychological modeling of human cognition and behavior. - There is a lack of a standard intention classification system; different studies have proposed various classification methods, but there is no consensus. - Existing models are mostly black-box neural networks, lacking interpretability and being susceptible to domain shift. #### Solutions 1. **Proposing the Intent-o-meter Model**: - Intent-o-meter is a multimodal (image and text) intention prediction model for social media posts. - The model not only utilizes visual and textual features but also incorporates the Theory of Reasoned Action (TRA), considering the creator's attitudes and social norms. - By designing new input features, the model can more accurately predict the creator's intentions. 2. **Developing the Intentgram Dataset**: - Intentgram is a dataset collected from 55,000 social media posts from public Instagram profiles. - The dataset includes images, captions, and tags, and uses a 7-label intention classification system. 3. **User Feedback and Application**: - A web application similar to a social media platform was developed, displaying posts with intention labels to collect user feedback. - Experimental results show that about 70% of users found the intention labels helpful in better understanding the content they consumed and were willing to try filtering content based on these labels. ### Summary The main goal of the paper is to reduce the negative impact of multimodal content on adolescent mental health by understanding and predicting the intentions of social media content creators. By introducing the Intent-o-meter model and the Intentgram dataset, the researchers provide a more comprehensive approach to addressing this issue.

Towards determining perceived audience intent for multimodal social media posts using the theory of reasoned action

Multimodal Marketing Intent Analysis for Effective Targeted Advertising

The Unappreciated Role of Intent in Algorithmic Moderation of Social Media Content

Intentonomy: a Dataset and Study towards Human Intent Understanding

Integrating Text and Image: Determining Multimodal Document Intent in Instagram Posts

Multimodal Post Attentive Profiling for Influencer Marketing.

Moral judgment of objectionable online content: Reporting decisions and punishment preferences on social media

MIKO: Multimodal Intention Knowledge Distillation from Large Language Models for Social-Media Commonsense Discovery

A Comprehensive Artificial Intelligence Based User Intention Assessment Model from Online Reviews and Social Media

A Multimodal Analysis of Influencer Content on Twitter

A multi-analytic approach to predict social media marketing influence on consumer purchase intention

Effects of Automated Misinformation Warning Labels on the Intents to Like, Comment and Share Posts

Characterizing Anti-Vaping Posts for Effective Communication on Instagram Using Multimodal Deep Learning

Predicting Potential School Shooters from Social Media Posts

Multimodal analysis of user behavior and browsed content under different image search intents

Modeling Political Orientation of Social Media Posts: An Extended Analysis

A Multidimensional Approach for Evaluating Reality in Social Media: Mixed Methods Study

Characterizing and Classifying Developer Forum Posts with their Intentions

A STUDY ON THE MEDIATING EFFECT OF FoMO ON SOCIAL MEDIA (INSTAGRAM ) INDUCED TRAVEL ADDICTION AND RISK TAKING TRAVEL BEHAVIORAL INTENTION IN YOUTH

Why Misinformation is Created? Detecting them by Integrating Intent Features

A Novel Hybrid Multi-Modal Deep Learning for Detecting Hashtag Incongruity on Social Media