Abstract:The von Neumann graph entropy (VNGE) is a measure of graph complexity based on the Laplacian spectrum. It has recently found applications in various learning tasks driven by networked data. However, it is computationally demanding and hard to interpret using simple structural patterns. Due to the close relation between Lapalcian spectrum and degree sequence, we conjecture that the structural information, defined as the Shannon entropy of the normalized degree sequence, might approximate VNGE well. In this work, we thereby study the difference between the structural information and VNGE named as entropy gap. Based on the knowledge that the degree sequence is majorized by the Laplacian spectrum, we for the first time prove the entropy gap is between 0 and log 2e in any undirected unweighted graphs. Consequently we certify that the structural information is a good approximation of VNGE that achieves provable accuracy, scalability, and interpretability simultaneously. This approximation is further applied to two entropy-related tasks: network design and graph similarity measure, where novel graph similarity measure and fast algorithms are proposed. Our experimental results on graphs of various scales and types show that the very small entropy gap readily applies to a wide range of graphs and weighted graphs. As an approximation of VNGE, the structural information is the only one that achieves both high efficiency and high accuracy among the prominent methods. It is at least two orders of magnitude faster than SLaQ [40] with comparable accuracy. Our structural information based methods also exhibit superior performance in two entropy-related tasks.

Varentropy Estimation via Nearest Neighbor Graphs

On stochastic properties of past varentropy with applications

Varentropy of past lifetimes

Efficient multivariate entropy estimation via $k$-nearest neighbour distances

Weighted (residual) varentropy and its applications

Nearest neighbor empirical processes

Stochastic properties of varentropy based on residual lifetime

Optimizing Distributions for Associated Entropic Vectors via Generative Convolutional Neural Networks

Concentration of information on discrete groups

Contribution to Transfer Entropy Estimation Via the K-Nearest-neighbors Approach

On Spectral Portraits of Incidence Matrices of Nearest Neighbor Graphs

Random Forests for Adaptive Nearest Neighbor Estimation of Information-Theoretic Quantities

On the estimation of varextropy under complete data

On the rate of change of Varentropy for Markov diffusion processes

Bridging the Gap Between Von Neumann Graph Entropy and Structural Information: Theory and Applications

On the Von Neumann Entropy of Graphs

Entropy Estimation via Uniformization

$V$-statistics and Variance Estimation

Varextropy of doubly truncated random variable

Results on Varextropy Measure of Random Variables

Closely estimating the entropy of sparse graph models