Abstract:Micro-videos, a new form of user-generated content, have been spreading widely across various social platforms, such as Vine, Kuaishou, and TikTok. Different from traditional long videos, micro-videos are usually recorded by smart mobile devices at any place within a few seconds. Due to their brevity and low bandwidth cost, micro-videos are gaining increasing user enthusiasm. The blossoming of micro-videos opens the door to the possibility of many promising applications, ranging from network content caching to online advertising. Thus, it is highly desirable to develop an effective scheme for high-order micro-video understanding. Micro-video understanding is, however, non-trivial due to the following challenges: (1) how to represent micro-videos that only convey one or few high-level themes or concepts; (2) how to utilize the hierarchical structure of venue categories to guide micro-video analysis; (3) how to alleviate the influence of low quality caused by complex surrounding environments and camera shake; (4) how to model multimodal sequential data, i.e. textual, acoustic, visual, and social modalities to enhance micro-video understanding; and (5) how to construct large-scale benchmark datasets for analysis. These challenges have been largely unexplored to date. In this book, we focus on addressing the challenges presented above by proposing some state-of-the-art multimodal learning theories. To demonstrate the effectiveness of these models, we apply them to three practical tasks of micro-video understanding: popularity prediction, venue category estimation, and micro-video routing. Particularly, we first build three large-scale real-world micro-video datasets for these practical tasks. We then present a multimodal transductive learning framework for micro-video popularity prediction. Furthermore, we introduce several multimodal cooperative learning approaches and a multimodal transfer learning scheme for micro-video venue category estimation. Meanwhile, we develop a multimodal sequential learning approach for micro-video recommendation. Finally, we conclude the book and figure out the future research directions in multimodal learning toward micro-video understanding. Table of Contents: Preface / Acknowledgments / Introduction / Data Collection / Multimodal Transductive Learning for Micro-Video Popularity Prediction / Multimodal Cooperative Learning for Micro-Video Venue Categorization / Multimodal Transfer Learning in Micro-Video Analysis / Multimodal Sequential Learning for Micro-Video Recommendation / Research Frontiers / Bibliography / Authors' Biographies

Hashtag Our Stories: Hashtag Recommendation for Micro-Videos Via Harnessing Multiple Modalities.

Sentiment Enhanced Multi-Modal Hashtag Recommendation for Micro-Videos

A Hybrid Filtering for Micro-video Hashtag Recommendation using Graph-based Deep Neural Network

An Intelligent Video Tag Recommendation Method for Improving Video Popularity in Mobile Computing Environment

Personalized Hashtag Recommendation for Micro-videos

Implicit semantic-based personalized micro-videos recommendation

Hashtag Recommendation for Multimodal Microblog Using Co-Attention Network.

Research and Implementation of Short Video Product Recommendation Algorithm Based on Multimodal Features

Co-Attention Memory Network for Multimodal Microblog's Hashtag Recommendation

Hashtag Recommendation Using End-To-End Memory Networks with Hierarchical Attention.

Personalized Micro-Video Recommendation via Hierarchical User Interest Modeling

A Hybrid Deep Neural Network for Multimodal Personalized Hashtag Recommendation

Attention-based Multimodal Feature Representation Model for Micro-video Recommendation

Multimodal Topic Learning for Video Recommendation

Hashtag Recommendation Using Attention-Based Convolutional Neural Network.

User-Video Co-Attention Network for Personalized Micro-video Recommendation

LLM-MHR: A LLM-Augmented Multimodal Hashtag Recommendation Algorithm

Time-aware Personalized Hashtag Recommendation on Social Media.

Personalized Micro-video Recommendation Based on Multi-modal Features and User Interest Evolution

Multimodal Learning toward Micro-Video Understanding

Micro Tells Macro: Predicting The Popularity Of Micro-Videos Via A Transductive Model