Abstract:Existing multi-view classification algorithms usually assume that all examples have observations on all views, and the data in different views are clean. However, in real-world applications, we are often provided with data that have missing representations or contain noise on some views (i.e., missing or noise views). This may lead to significant performance degeneration, and thus many algorithms are proposed to address the incomplete view or noisy view issues. However, most of existing algorithms deal with the two issues separately, and hence may fail when both missing and noisy views exist. They are also usually not flexible in that the view or feature significance cannot be adaptively identified. Besides, the view missing patterns may vary in the training and test phases, and such difference is often ignored. To remedy these drawbacks, we propose a novel multi-view classification framework that is simultaneously robust to both incomplete and noisy views. This is achieved by integrating early fusion and late fusion in a single framework. Specifically, in our early fusion module, we propose a view-aware transformer to mask the missing views and adaptively explore the relationships between views and target tasks to deal with missing views. Considering that view missing patterns may change from the training to the test phase, we also design single-view classification and category-consistency constraints to reduce the dependence of our model on view-missing patterns. In our late fusion module, we quantify the view uncertainty in an ensemble way to estimate the noise level of that view. Then the uncertainty and prediction logits of different views are integrated to make our model robust to noisy views. The framework is trained in an end-to-end manner. Experimental results on diverse datasets demonstrate the robustness and effectiveness of our model for both incomplete and noisy views. Codes are available at https://github.com/li-yapeng/UVaT.

Effective and Efficient Multi-View Imputation with Optimal Transport

Jointly Imputing Multi-View Data with Optimal Transport.

Exploring and Exploiting Uncertainty for Incomplete Multi-View Classification

VIGAN: Missing View Imputation with Generative Adversarial Networks

Missing Value Imputation for Multi-view Urban Statistical Data via Spatial Correlation Learning

Unified View Imputation and Feature Selection Learning for Incomplete Multi-view Data

Generative Partial Multi-View Clustering

Incomplete Multi-view Clustering via Prototype-based Imputation

Imputation of missing values in multi-view data

Incomplete Multi-view Fuzzy Inference System with Missing View Imputation and Cooperative Learning

Efficient Unsupervised Dimension Reduction for Streaming Multiview Data.

URRL-IMVC: Unified and Robust Representation Learning for Incomplete Multi-View Clustering

Missing Road Condition Imputation Using a Multi-View Heterogeneous Graph Network from GPS Trajectory

A matrix completion-based multiview learning method for imputing missing values in buoy monitoring data

UVaT: Uncertainty Incorporated View-Aware Transformer for Robust Multi-View Classification

Improved generative adversarial imputation networks for missing data

Task-Augmented Cross-View Imputation Network for Partial Multi-View Incomplete Multi-Label Classification

Uncertainty-guided Optimal Transport in Depth Supervised Sparse-View 3D Gaussian

Multi-View Representation Learning via Dual Optimal Transportation

Uniform Projection for Multi-View Learning.

Multi-View Learning with Incomplete Views