Abstract:Multiway data analysis aims to uncover patterns in data structured as multi-indexed arrays, and the covariance of such data plays a crucial role in various machine learning applications. However, the intrinsically high dimension of multiway covariance presents significant challenges. To address these challenges, factorized covariance models have been proposed that rely on a separability assumption: the multiway covariance can be accurately expressed as a sum of Kronecker products of mode-wise covariances. This paper is concerned with the accuracy of such separable models for representing multiway covariances. We reduce the question of whether a given covariance can be represented as a separable multiway covariance to an equivalent question about separability of quantum states. Based on this equivalence, we establish that generic multiway covariances tend to be not separable. Moreover, we show that determining the best separable approximation of a generic covariance is NP-hard. Our results suggest that factorized covariance models might not accurately approximate covariance, without additional assumptions ensuring separability. To balance these negative results, we propose an iterative Frank-Wolfe algorithm for computing Kronecker-separable covariance approximations with some additional side information. We establish an oracle complexity bound and empirically observe its consistent convergence to a separable limit point, often close to the ``best'' separable approximation. These results suggest that practical methods may be able to find a Kronecker-separable approximation of covariances, despite the worst-case NP hardness results.

Univariate Gaussian Model For Multimodal Inseparable Problems

Learning Gaussian Networks

An iterative local updating ensemble smoother for high-dimensional inverse modeling with multimodal distributions

Scalable Semisupervised GMM for Big Data Quality Prediction in Multimode Processes.

A Multiobjective Evolutionary Algorithm Using Gaussian Process-Based Inverse Modeling.

Analysis of multivariate Gaussian mixture model based on DRJMCMC method

Partial separability and functional graphical models for multivariate Gaussian processes

Gaussian model-based partitioning using iterated local search

Exploration of the search space of Gaussian graphical models for paired data

A Genetic Algorithm for Multivariate Isotonic Regression

Optimal Multi-model Detection with Application to Gaussian Problems

Sparse Multimodal Gaussian Processes

Heterogeneous Ensemble-Based Infill Criterion for Evolutionary Multiobjective Optimization of Expensive Problems

A multi-manifold semi-supervised Gaussian mixture model for pattern classification

Multivariate Bernoulli distribution

Infinite mixtures of multivariate Gaussian processes

An adaptive Gaussian process method for multi-modal Bayesian inverse problems

On Separability of Covariance in Multiway Data Analysis

A Two-round Variant of EM for Gaussian Mixtures

Multiview Learning with Variational Mixtures of Gaussian Processes

Joint estimation of multiple dependent Gaussian graphical models with applications to mouse genomics