Abstract:In this paper, the problem of identifying s correlated components from a p−dimensional Gaussian vector is considered. Instead of having to take a full-vector observation at each time index, the observer is allowed to observe any subset or full set of components in the vector, and he has the freedom to design his sampling strategies over time. The observer aims to find an optimal sampling strategy and a decision rule to maximize the error exponent (per sample). Both nonadaptive strategies, in which the sampling actions are predetermined, and adaptive strategies, in which the sampling actions can depend on the observations, are considered. We first characterize the optimal non-adaptive sampling strategy. We show that the strategy of taking full-vector observation is only optimal when the strength of the signal is weak. When the strength of the signal is strong, the optimal strategy is to observe only l∗ components, where l∗ is an implicit function of the signal strength, in each time slot, and all possible combinations of l∗-component strategies appear with the same frequency in the optimal strategy. We show that this optimal non-adaptive sampling strategy increases the detection error exponent at least 25% compared with the classic full-vector observation strategy. Furthermore, we derive performance bounds for the adaptive sampling strategies. We show that a low complexity adaptive procedure, namely the sequential diagonal method, substantially outperforms the optimal non-adaptive strategy when the strength of the signal is strong. The work of J. Geng was supported by the Fundamental Research Funds for the Central Universities under grant AUGA5710013915. The work of L. Lai was supported by the National Science Foundation under grant CNS-1457076. The results in this paper were presented in part at IEEE Information Theory Workshop, Jeju Island, Korea, Oct., 2015 [1]. J. Geng is with School of Electronics and Information Engineering, Harbin Institute of Technology, Harbin, 150001, China (Email: jgeng@hit.edu.cn) W. Xu is with the Department of Electrical and Computer Engineering, University of Iowa, Iowa City, IA, 52242, USA (Email: weiyu-xu@uiowa.edu). L. Lai is with the Department of Electrical and Computer Engineering, Worcester Polytechnic Institute, Worcester, MA, 01609, USA (Email: llai@wpi.edu). June 14, 2016 DRAFT Page 1 of 38 https://mc.manuscriptcentral.com/t-it IEEE Transactions on Information Theory, For Peer Review Only

Gaussian Universal Features, Canonical Correlations, and Common Information.

An Information-Theoretic Approach to Universal Feature Selection in High-Dimensional Inference.

A Local Characterization for Wyner Common Information

An Information-Theoretic Approach to Unsupervised Feature Selection for High-Dimensional Data

Identifying correlated components in high-dimensional multivariate gaussian models

TCIC_FS: Total Correlation Information Coefficient-Based Feature Selection Method for High-Dimensional Data

Gacs-Korner Common Information Variational Autoencoder

Generalized Common Informations: Measuring Commonness by the Conditional Maximal Correlation

Discovering Common Information in Multi-view Data

Generalized Canonical Correlation Analysis Using GSVD

Generalized Information Bottleneck for Gaussian Variables

D-GCCA: Decomposition-based Generalized Canonical Correlation Analysis for Multi-view High-dimensional Data

Estimation of Correlated Components in Multivariate Gaussian Models

Detection of Correlated Components in Multivariate Gaussian Models

Canonical Correlation Analysis based on Hilbert-Schmidt Independence Criterion and Centered Kernel Target Alignment.

A Novel Generalized Fuzzy Canonical Correlation Analysis Framework for Feature Fusion and Recognition

Kernel Generalized Canonical Correlation and a New Feature Fusion Strategy.

Mutual Information Criteria for Feature Selection

Revisiting Deep Generalized Canonical Correlation Analysis

Covariance descriptors on a Gaussian manifold and their application to image set classification