Abstract:Matrix completion tackles the task of predicting missing values in a low-rank matrix based on a sparse set of observed entries. It is often assumed that the observation pattern is generated uniformly at random or has a very specific structure tuned to a given algorithm. There is still a gap in our understanding when it comes to arbitrary sampling patterns. Given an arbitrary sampling pattern, we introduce a matrix completion algorithm based on network flows in the bipartite graph induced by the observation pattern. For additive matrices, the particular flow we used is the electrical flow and we establish error upper bounds customized to each entry as a function of the observation set, along with matching minimax lower bounds. Our results show that the minimax squared error for recovery of a particular entry in the matrix is proportional to the effective resistance of the corresponding edge in the graph. Furthermore, we show that our estimator is equivalent to the least squares estimator. We apply our estimator to the two-way fixed effects model and show that it enables us to accurately infer individual causal effects and the unit-specific and time-specific confounders. For rank-$1$ matrices, we use edge-disjoint paths to form an estimator that achieves minimax optimal estimation when the sampling is sufficiently dense. Our discovery introduces a new family of estimators parametrized by network flows, which provide a fine-grained and intuitive understanding of the impact of the given sampling pattern on the relative difficulty of estimation at an entry-specific level. This graph-based approach allows us to quantify the inherent complexity of matrix completion for individual entries, rather than relying solely on global measures of performance.

Sequential And Adaptive Sampling For Matrix Completion In Network Monitoring Systems

Poster: online adaptive sampling for network delay measurement via matrix completion

Adaptive Coherent Sampling for Network Delay Measurement

Entry-Specific Matrix Estimation under Arbitrary Sampling Patterns through the Lens of Network Flows

Efficient Performance Monitoring for Ubiquitous Virtual Networks Based on Matrix Completion

Data Reconstructing Algorithm in Unreliable Links Based on Matrix Completion for Heterogeneous Wireless Sensor Networks

Increasing Energy Efficiency in Sensor Networks: Blue Noise Sampling and Non-Convex Matrix Completion

A Compressive Sensing-Based Approach to End-to-End Network Traffic Reconstruction

Matrix Completion with Nonuniform Sampling: Theories and Methods.

A sparsity adaptive measurement algorithm for network traffic matrix

Network Latency Estimation for Personal Devices: A Matrix Completion Approach.

Information-Guided Sampling for Low-Rank Matrix Completion

Subspace projection matrix recovery from incomplete information

RW-MC: Self-Adaptive Random Walk Based Matrix Completion Algorithm

Matrix Completion with Deterministic Sampling: Theories and Methods

A Parallel Low Rank Matrix Optimization Method for Recovering Internet Traffic Network Data Via Link Flow Measurement.

Space-Time Sampling for Network Observability

Low-rank Traffic Matrix Completion with Marginal Information

Adaptive Load-Aware Sampling for Network Monitoring on Multicore Commodity Hardware

A Hybrid Sampling Approach for Network Flow Monitoring.

Joint Time-Frequency Sparse Estimation of Large-Scale Network Traffic