Abstract:This paper provides an analysis of the latest research trends of theories and technologies in cross-media intelligent correlation analysis and semantic understanding. The main content of this report includes a unified representation of cross-media information, knowledge-guided data fusion, cross-media correlation analysis, cross-media knowledge graph, and intelligent applications for multi-modal. Unified representations are preconditions for analyzing and inference about multi-modal information. The semantic consistency between multi-modal information is utilized to eliminate redundant information and achieve unified representation through cross-modal interconversion to learn more comprehensive feature representation. The cross-media association analysis focuses on image-language, video-language, and audio-video-language,aiming to bridge the semantic gap between visual, auditory, language, and fully establish the semantic association between different modalities. By introducing the construction of cross-media knowledge graph,cross-media knowledge graph construction, cross-media knowledge graph embedding, and cross-media knowledge inference, the cross-media representation based on knowledge graph enhances the reliability and improves the efficiency and accuracy of subsequent inference tasks. With the rapid development of cross-modal analysis, intelligent applications for multi-modal are supported by more technologies. According to the required domain knowledge, this paper selects cross-modal applications such as multi-modal visual question answering, multi-modal video summarization, multi-modal visual pattern mining, multi-modal recommendation, cross-modal intelligent inference, and cross-modal medical image prediction, their research progress is compared and reviewed in terms of multi-modal fusion and cross-media inference.

Multi-modal Tensor Data Mining Algorithms and Applications

Deep Vision Multimodal Learning: Methodology, Benchmark, and Trend

Multi-way Spectral Analysis Methods Based on Support Tensor Machines

Unraveling Complexity: an Exploration into the Large-Scale Multi-Modal Signal Processing

Tensor Methods in High Dimensional Data Analysis: Opportunities and Challenges

Multi-modal Deep Analysis for Multimedia

Multi-modal Multi-view Topic-opinion Mining for Social Event Analysis.

Tensor Multi-Clustering Parallel Intelligent Computing Method Based on Tensor Chain Decomposition

Many Could Be Better Than All: A Novel Instance-Oriented Algorithm for Multi-modal Multi-label Problem.

A Discriminative Vectorial Framework for Multi-modal Feature Representation

Extensible Topic Modeling and Analysis Framework for Multisource Data*

Survey on Deep Multi-modal Data Analytics: Collaboration, Rivalry and Fusion

Artificial intelligence accelerates multi-modal biomedical process: A Survey

Multi-view clustering: A survey

Recent Advances and Trends in Multimodal Deep Learning: A Review

Advances in Theory and Technology of Cross-Media Intelligent Association Analysis

Applications of tensor (multiway array) factorizations and decompositions in data mining

A survey of multi-modal learning theory

Multi-modal visual tracking: Review and experimental comparison

A Survey of Multi-object Video Tracking Algorithms

Multi-modal discrete tensor decomposition hashing for efficient multimedia retrieval