Abstract:Cross-media 3D model recognition is an important and challenging task in computer vision, which can be utilized in many applications such as landmark detection, image set classification, etc. In recent years, with the development of deep learning, many approaches have been proposed to handle the 3D model recognition problem. However, all of these methods focus on the structure information representation and the multi-view information fusion, and ignore the spatial and temporal information. So that it is not suitable for the cross-media 3D model recognition. In this paper, we utilize the sequence views to represent each 3D model and propose a novel Multi-view Convolutional LSTM Network (MVCLN), which utilizes the LSTM structure to extract temporal information and applies the convolutional operation to extract spatial information. More especially, the spatial and temporal information both are considered during the training process, which can effectively utilize the differences between the view's spatial information to improve the final performance. Meanwhile, we also introduce the classic attention model to define the weight of each view, which can reduce the redundant information of view's spatial information in the information fusion step. We evaluate the proposed method on the ModelNet40 for 3D model classification and retrieval task. We also construct a dataset utilizing the overlap categories of MV-RED, ShapenetCore and ModelNet to demonstrate the effectiveness of our approach for the cross-media 3D model recognition. Experimental results and comparisons with the state-of-the-art methods demonstrate that our framework can achieve superior performance.

Multi-View Deep Metric Learning for Volumetric Image Recognition.

Multi-view Deep Metric Learning for Image Classification.

Multi-View Analysis Dictionary Learning for Image Classification.

Sharable and Individual Multi-View Metric Learning.

Image classification with multi-view multi-instance metric learning

Multi-view metric learning for visual classification

Deep Variational Metric Learning

Collaborative Multi-View Metric Learning For Visual Classification

Multiview Metric Learning with Global Consistency and Local Smoothness

Multi-View Metric Learning For Multi-Label Image Classification

A Multi-View Deep Metric Learning Approach for Categorical Representation on Mixed Data

Multi-Manifold Deep Metric Learning For Image Set Classification

Beyond Shared Subspace: A View-Specific Fusion for Multi-View Multi-Label Learning.

Multi-View Stereo with Learnable Cost Metric

Multi-view metric learning based on KL-divergence for similarity measurement.

MV-C3D: A Spatial Correlated Multi-View 3D Convolutional Neural Networks

MVCLN: Multi-View Convolutional LSTM Network for Cross-Media 3D Shape Recognition

Multi-view Domain Generalization for Visual Recognition

Multi-View Stereo Network Based on Attention Mechanism and Neural Volume Rendering

Exponential Discriminative Metric Embedding in Deep Learning

Deep Metric Multi-View Hashing for Multimedia Retrieval