A sampling construction of graphon 1-norm convergence

Xingchao Jian,Feng Ji,Wee Peng Tay

2024-03-31

Abstract:In the short note, we describe a sampling construction that yields a sequence of graphons converging to a prescribed limit graphon in 1-norm. This convergence is stronger than the convergence in the cut norm, usually used to study graphon sequences. The note also contains errata of the previous version of the note.

Signal Processing

What problem does this paper attempt to address?

On the Rate of Convergence by Generalized Baskakov Operators

Yi Gao,Wenshuai Wang,Shigang Yue

DOI: https://doi.org/10.1155/2015/564854

2015-01-01

Advances in Mathematical Physics

Abstract:We firstly construct generalized Baskakov operators Vn,α,q(f;x) and their truncated sum Bn,α,q(f;γn,x). Secondly, we study the pointwise convergence and the uniform convergence of the operators Vn,α,q(f;x), respectively, and estimate that the rate of convergence by the operators Vn,α,q(f;x) is 1/nq/2. Finally, we study the convergence by the truncated operators Bn,α,q(f;γn,x) and state that the finite truncated sum Bn,α,q(f;γn,x) can replace the operators Vn,α,q(f;x) in the computational point of view provided that limn→∞nγn=∞.
A Supplement to the Complete Convergence

WD Liu,ZY Lin

DOI: https://doi.org/10.1016/j.spl.2005.10.006

IF: 0.718

2006-01-01

Statistics & Probability Letters

Abstract:Let X,X1,X2,… be independent identically distributed random variables. Then, Baum and Katz [1965. Convergence rates in the law of large numbers. Trans. Amer. Math. Soc. 120, 108–123] proved that for p>1,∑k=1∞kp-2P(|Sk|⩾k)<∞,if and only if E|X|p<∞ and EX=0. We prove thatf(n)≔∑k=1nkp-2P(|Sk|⩾k)is slowly varying asn→∞,if and only if EX=0 and E|X|pI{|X|⩽x} is slowly varying as x→∞.
Sampling and Uniqueness Sets in Graphon Signal Processing

Alejandro Parada-Mayorga,Alejandro Ribeiro

2024-01-12

Abstract:In this work, we study the properties of sampling sets on families of large graphs by leveraging the theory of graphons and graph limits. To this end, we extend to graphon signals the notion of removable and uniqueness sets, which was developed originally for the analysis of signals on graphs. We state the formal definition of a $\Lambda-$removable set and conditions under which a bandlimited graphon signal can be represented in a unique way when its samples are obtained from the complement of a given $\Lambda-$removable set in the graphon. By leveraging such results we show that graphon representations of graphs and graph signals can be used as a common framework to compare sampling sets between graphs with different numbers of nodes and edges, and different node labelings. Additionally, given a sequence of graphs that converges to a graphon, we show that the sequences of sampling sets whose graphon representation is identical in $[0,1]$ are convergent as well. We exploit the convergence results to provide an algorithm that obtains approximately close to optimal sampling sets. Performing a set of numerical experiments, we evaluate the quality of these sampling sets. Our results open the door for the efficient computation of optimal sampling sets in graphs of large size.

Machine Learning,Signal Processing
Generalized Graphon Process: Convergence of Graph Frequencies in Stretched Cut Distance

Xingchao Jian,Feng Ji,Wee Peng Tay

2023-09-11

Abstract:Graphons have traditionally served as limit objects for dense graph sequences, with the cut distance serving as the metric for convergence. However, sparse graph sequences converge to the trivial graphon under the conventional definition of cut distance, which make this framework inadequate for many practical applications. In this paper, we utilize the concepts of generalized graphons and stretched cut distance to describe the convergence of sparse graph sequences. Specifically, we consider a random graph process generated from a generalized graphon. This random graph process converges to the generalized graphon in stretched cut distance. We use this random graph process to model the growing sparse graph, and prove the convergence of the adjacency matrices' eigenvalues. We supplement our findings with experimental validation. Our results indicate the possibility of transfer learning between sparse graphs.

Signal Processing,Machine Learning
Graphons of Line Graphs

Sevvandi Kandanaarachchi,Cheng Soon Ong

2024-09-11

Abstract:We consider the problem of estimating graph limits, known as graphons, from observations of sequences of sparse finite graphs. In this paper we show a simple method that can shed light on a subset of sparse graphs. The method involves mapping the original graphs to their line graphs. We show that graphs satisfying a particular property, which we call the square-degree property are sparse, but give rise to dense line graphs. This enables the use of results on graph limits of dense graphs to derive convergence. In particular, star graphs satisfy the square-degree property resulting in dense line graphs and non-zero graphons of line graphs. We demonstrate empirically that we can distinguish different numbers of stars (which are sparse) by the graphons of their corresponding line graphs. Whereas in the original graphs, the different number of stars all converge to the zero graphon due to sparsity. Similarly, superlinear preferential attachment graphs give rise to dense line graphs almost surely. In contrast, dense graphs, including Erdos-Renyi graphs make the line graphs sparse, resulting in the zero graphon.

Machine Learning,Discrete Mathematics,Combinatorics
Modeling Sparse Graph Sequences and Signals Using Generalized Graphons

Feng Ji,Xingchao Jian,Wee Peng Tay

2024-03-23

Abstract:Graphons are limit objects of sequences of graphs and are used to analyze the behavior of large graphs. Recently, graphon signal processing has been developed to study signal processing on large graphs. A major limitation of this approach is that any sparse sequence of graphs inevitably converges to the zero graphon, rendering the resulting signal processing theory trivial and inadequate for sparse graph sequences. To overcome this limitation, we propose a new signal processing framework that leverages the concept of generalized graphons and introduces the stretched cut distance as a measure to compare these graphons. Our framework focuses on the sampling of graph sequences from generalized graphons and explores the convergence properties of associated operators, spectra, and signals. Our signal processing framework provides a comprehensive approach to analyzing and processing signals on graph sequences, even if they are sparse. Finally, we discuss the practical implications of our theory for real-world large networks through numerical experiments.

Signal Processing
Graph Signal Sampling Under Smoothness Priors: A Difference-of-Convex Approach

Shunsuke Ono,Kazuki Naganuma,Keitaro Yamashita

2023-06-26

Abstract:This paper proposes a method for properly sampling graph signals under smoothness priors. Unlike many existing approaches that assume bandlimited graph signals, our method designs a sampling operator for graph signals that are not necessarily bandlimited based on the generalized sampling theory. First, we formulate the sampling operator design under smoothness priors as a feasibility problem. Then, we transform this problem into a Difference-of-Convex (DC) problem by relaxing a certain invertibility constraint in the original problem using the nuclear norm. Furthermore, we develop an efficient algorithm to solve this DC problem based on the proximal linearized difference-of-convex algorithm and guarantee its convergence to a critical point of the problem. Finally, we demonstrate the effectiveness of our method for several graph signal models through sampling-and-reconstruction experiments.

Signal Processing
A Poincaré Inequality and Consistency Results for Signal Sampling on Large Graphs

Thien Le,Luana Ruiz,Stefanie Jegelka

2024-10-10

Abstract:Large-scale graph machine learning is challenging as the complexity of learning models scales with the graph size. Subsampling the graph is a viable alternative, but sampling on graphs is nontrivial as graphs are non-Euclidean. Existing graph sampling techniques require not only computing the spectra of large matrices but also repeating these computations when the graph changes, e.g., grows. In this paper, we introduce a signal sampling theory for a type of graph limit -- the graphon. We prove a Poincaré inequality for graphon signals and show that complements of node subsets satisfying this inequality are unique sampling sets for Paley-Wiener spaces of graphon signals. Exploiting connections with spectral clustering and Gaussian elimination, we prove that such sampling sets are consistent in the sense that unique sampling sets on a convergent graph sequence converge to unique sampling sets on the graphon. We then propose a related graphon signal sampling algorithm for large graphs, and demonstrate its good empirical performance on graph machine learning tasks.

Machine Learning
Graph sequences sampled from Robinson graphons

Mahya Ghandehari,Jeannette Janssen

DOI: https://doi.org/10.1016/j.ejc.2023.103859

2023-09-12

Abstract:The function $\Gamma$ on the space of graphons, introduced in [CGH$^+$15], aims to measure the extent to which a graphon $w$ exhibits the Robinson property: for all $x<y<z$, $w(x,z)\leq \min\{ w(x,y),w(y,z)\}$. Robinson graphons form a model for graphs with a natural line embedding so that most edges are local. Function $\Gamma$ is compatible with the cut-norm $\|\cdot \|_\Box$, in the sense that graphons close in cut-norm have similar $\Gamma $-values. Here we show the converse, by proving that every graphon $w$ can be approximated by a Robinson graphon $R_w$ so that $\|w-R_w\|_\Box$ is bounded in terms of $\Gamma (w)$. We then use classical techniques from functional analysis to show that a converging graph sequence $\{G_n\}$ converges to a Robinson graphon if and only if $\Gamma (G_n)\rightarrow 0$. Finally, using probabilistic techniques we show that the rate of convergence of $\Gamma$ for graph sequences sampled from a Robinson graphon can differ substantially depending on how strongly $w$ exhibits the Robinson property.

Combinatorics
A Note on the Convergence Rate of the Spectral Distributions of Large Sample Covariance Matrices

JIN Bai-suo,MIAO Bai-qi,PAN Guang-ming

DOI: https://doi.org/10.3969/j.issn.0253-2778.2005.06.012

2005-01-01

Journal of University of Science and Technology of China

Abstract:In this note, we improve the convergence rates of spectral distributions that large-dimensional sample covariance matrices of size p×n tend to the Marenko-Pastur distribution, under the assumption that the entries have a finite eighth moment. Especially, we show that the expected spectral distributions of large-dimensional sample covariance matrices of size p×n tends to the limiting distribution with the dimension sample size ratio y=y_n=p/n at the rate of O(n~ -1/6 ) if y is close to 1 . Similar results for both the convergence in probability and the almost sure convergence are shown to be O_p(n~ -1/6 ) and O_ a.s. (n~ -1/6 ), under the condition that y is close to 1.
Probability graphons: the right convergence point of view

Giulio Zucal

2024-07-08

Abstract:We extend the theory of probability graphons, continuum representations of edge-decorated graphs arising in graph limits theory, to the 'right convergence' point of view. First of all, we generalise the notions of overlay functionals and quotient sets to the case of probability graphons. Furthermore, we characterise the convergence of probability graphons in terms of these global quantities. In particular, we show the equivalence of these two notions of convergence with the unlabelled cut-metric convergence (and thus also with the homomorphism densities convergence and the subgraph sampling convergence). In other words, we prove the equivalence of the 'left convergence' and the 'right convergence' views on probability graphons convergence, generalising the corresponding result for (real-valued) graphons (the classical continuum representation for simple graphs).

Probability,Combinatorics,Functional Analysis
Bi-stochastically normalized graph Laplacian: convergence to manifold Laplacian and robustness to outlier noise

Xiuyuan Cheng,Boris Landa

2024-07-18

Abstract:Bi-stochastic normalization provides an alternative normalization of graph Laplacians in graph-based data analysis and can be computed efficiently by Sinkhorn-Knopp (SK) iterations. This paper proves the convergence of bi-stochastically normalized graph Laplacian to manifold (weighted-)Laplacian with rates, when $n$ data points are i.i.d. sampled from a general $d$-dimensional manifold embedded in a possibly high-dimensional space. Under certain joint limit of $n \to \infty$ and kernel bandwidth $\epsilon \to 0$, the point-wise convergence rate of the graph Laplacian operator (under 2-norm) is proved to be $ O( n^{-1/(d/2+3)})$ at finite large $n$ up to log factors, achieved at the scaling of $\epsilon \sim n^{-1/(d/2+3)} $. When the manifold data are corrupted by outlier noise, we theoretically prove the graph Laplacian point-wise consistency which matches the rate for clean manifold data plus an additional term proportional to the boundedness of the inner-products of the noise vectors among themselves and with data vectors. Motivated by our analysis, which suggests that not exact bi-stochastic normalization but an approximate one will achieve the same consistency rate, we propose an approximate and constrained matrix scaling problem that can be solved by SK iterations with early termination. Numerical experiments support our theoretical results and show the robustness of bi-stochastically normalized graph Laplacian to high-dimensional outlier noise.

Statistics Theory,Machine Learning
Graphon Particle Systems, Part I: Spatio-Temporal Approximation and Law of Large Numbers

Yan Chen,Tao Li

2024-07-03

Abstract:We study a class of graphon particle systems with time-varying random coefficients. In a graphon particle system, the interactions among particles are characterized by the coupled mean field terms through an underlying graphon and the randomness of the coefficients comes from the stochastic processes associated with the particle labels. By constructing two-level approximated sequences converging in 2-Wasserstein distance, we prove the existence and uniqueness of the solution to the system. Besides, by constructing two-level approximated functions converging to the graphon mean field terms, we establish the law of large numbers, which reveals that if the number of particles tends to infinity and the discretization step tends to zero, then the discrete-time interacting particle system over a large-scale network converges to the graphon particle system. As a byproduct, we discover that the graphon particle system can describe the limiting dynamics of the distributed stochastic gradient descent algorithm over the large-scale network and prove that if the gradients of the local cost functions are Lipschitz continuous, then the graphon particle system can be regarded as the spatio-temporal approximation of the discrete-time distributed stochastic gradient descent algorithm as the number of network nodes tends to infinity and the algorithm step size tends to zero.

Systems and Control,Probability
A Short Note on Concentration Inequalities for Random Vectors with SubGaussian Norm

Chi Jin,Praneeth Netrapalli,Rong Ge,Sham M. Kakade,Michael I. Jordan

DOI: https://doi.org/10.48550/arXiv.1902.03736

2019-02-11

Abstract:In this note, we derive concentration inequalities for random vectors with subGaussian norm (a generalization of both subGaussian random vectors and norm bounded random vectors), which are tight up to logarithmic factors.

Probability,Machine Learning
The Sample Complexity of One-Hidden-Layer Neural Networks

Gal Vardi,Ohad Shamir,Nathan Srebro

DOI: https://doi.org/10.48550/arXiv.2202.06233

2022-09-22

Abstract:We study norm-based uniform convergence bounds for neural networks, aiming at a tight understanding of how these are affected by the architecture and type of norm constraint, for the simple class of scalar-valued one-hidden-layer networks, and inputs bounded in Euclidean norm. We begin by proving that in general, controlling the spectral norm of the hidden layer weight matrix is insufficient to get uniform convergence guarantees (independent of the network width), while a stronger Frobenius norm control is sufficient, extending and improving on previous work. Motivated by the proof constructions, we identify and analyze two important settings where (perhaps surprisingly) a mere spectral norm control turns out to be sufficient: First, when the network's activation functions are sufficiently smooth (with the result extending to deeper networks); and second, for certain types of convolutional networks. In the latter setting, we study how the sample complexity is additionally affected by parameters such as the amount of overlap between patches and the overall number of patches.

Machine Learning
Uniform spectral gap and orthogeodesic counting for strong convergence of Kleinian groups

Beibei Liu,Franco Vargas Pallete

2023-07-11

Abstract:We show convergence of small eigenvalues for geometrically finite hyperbolic $n$-manifolds under strong limits. For a class of convergent convex sets in a strongly convergent sequence of Kleinian groups, we use the spectral gap of the limit manifold and the exponentially mixing property of the geodesic flow along the strongly convergent sequence to find asymptotically uniform counting formulas for the number of orthogeodesics between the convex sets. In particular, this provides asymptotically uniform counting formulas (with respect to length) for orthogeodesics between converging Margulis tubes, geodesic loops based at converging basepoints, and primitive closed geodesics.

Differential Geometry,Geometric Topology
Graphon Filters: Graph Signal Processing in the Limit

Matthew W. Morency,Geert Leus

DOI: https://doi.org/10.1109/TSP.2021.3061575

2020-03-04

Abstract:Graph signal processing is an emerging field which aims to model processes that exist on the nodes of a network and are explained through diffusion over this structure. Graph signal processing works have heretofore assumed knowledge of the graph shift operator. Our approach is to investigate the question of graph filtering on a graph about which we only know a model. To do this we leverage the theory of graphons proposed by L. Lovasz and B. Szegedy. We make three key contributions to the emerging field of graph signal processing. We show first that filters defined over the scaled adjacency matrix of a random graph drawn from a graphon converge to filters defined over the Fredholm integral operator with the graphon as its kernel. Second, leveraging classical findings from the theory of the numerical solution of Fredholm integral equations, we define the Fourier-Galerkin shift operator. Lastly, using the Fourier-Galerkin shift operator, we derive a graph filter design algorithm which only depends on the graphon, and thus depends only on the probabilistic structure of the graph instead of the particular graph itself. The derived graphon filtering algorithm is verified through simulations on a variety of random graph models.

Signal Processing
Almost sure convergence of extreme order statistics

Zuoxiang Peng,Jiaona Li,Saralees Nadarajah

DOI: https://doi.org/10.1214/08-ejs303

2009-01-01

Electronic Journal of Statistics

Abstract:Let Mn(k) denote the kth largest maximum of a sample (X1,X2,…,Xn) from parent X with continuous distribution. Assume there exist normalizing constants an>0, bn∈ℝ and a nondegenerate distribution G such that . Then for fixed k∈ℕ, the almost sure convergence of is derived if the positive weight sequence (dn) with DN=∑n=1Ndn satisfies conditions provided by Hörmann. Some practical issues of this result are also discussed.

statistics & probability
The cut norm and Sampling Lemmas for unbounded kernels

Panna Tímea Fekete,Dávid Kunszenti-Kovács

2024-02-13

Abstract:Generalizing the bounded kernel results of Borgs, Chayes, Lovász, Sós and Vesztergombi (2008), we prove two Sampling Lemmas for unbounded kernels with respect to the cut norm. On the one hand, we show that given a (symmetric) kernel $U\in L^p([0,1]^2)$ for some $3<p<\infty$, the cut norm of a random $k$-sample of $U$ is with high probability within $O(k^{-\frac14+\frac{1}{4p}})$ of the cut norm of $U$. The cut norm of the sample has a strong bias to being larger than the original, allowing us to actually obtain a stronger high probability bound of order $O(k^{-\frac 12+\frac1p+\varepsilon})$ for how much smaller it can be (for any $p>2$ here). These results are then partially extended to the case of vector valued kernels. On the other hand, we show that with high probability, the $k$-samples are also close to $U$ in the cut metric, albeit with a weaker bound of order $O((\ln k)^{-\frac12+\frac1{2p}})$ (for any appropriate $p>2$). As a corollary, we obtain that whenever $U\in L^p$ with $p>4$, the $k$-samples converge almost surely to $U$ in the cut metric as $k\to\infty$.

Probability,Combinatorics,Functional Analysis
Convergence Rate of Gibbs Sampler and Its Application

Li Kaican,Geng Zhi

DOI: https://doi.org/10.1360/02ys0013

2005-01-01

Science in China Series A Mathematics

Abstract:Based on the convergence rate defined by the Pearson-X 2 distance, this paper discusses properties of different Gibbs sampling schemes. Under a set of regularity conditions, it is proved in this paper that the rate of convergence on systematic scan Gibbs samplers is the norm of a forward operator. We also discuss that the collapsed Gibbs sampler has a faster convergence rate than the systematic scan Gibbs sampler as proposed by Liu et al. Based on the definition of convergence rate of the Pearson- X 2 distance, this paper proved this result quantitatively. According to Theorem 2, we also proved that the convergence rate defined with the spectral radius of matrix by Robert and Shau is equivalent to the corresponding radius of the forward operator.

A sampling construction of graphon 1-norm convergence

On the Rate of Convergence by Generalized Baskakov Operators

A Supplement to the Complete Convergence

Sampling and Uniqueness Sets in Graphon Signal Processing

Generalized Graphon Process: Convergence of Graph Frequencies in Stretched Cut Distance

Graphons of Line Graphs

Modeling Sparse Graph Sequences and Signals Using Generalized Graphons

Graph Signal Sampling Under Smoothness Priors: A Difference-of-Convex Approach

A Poincaré Inequality and Consistency Results for Signal Sampling on Large Graphs

Graph sequences sampled from Robinson graphons

A Note on the Convergence Rate of the Spectral Distributions of Large Sample Covariance Matrices

Probability graphons: the right convergence point of view

Bi-stochastically normalized graph Laplacian: convergence to manifold Laplacian and robustness to outlier noise

Graphon Particle Systems, Part I: Spatio-Temporal Approximation and Law of Large Numbers

A Short Note on Concentration Inequalities for Random Vectors with SubGaussian Norm

The Sample Complexity of One-Hidden-Layer Neural Networks

Uniform spectral gap and orthogeodesic counting for strong convergence of Kleinian groups

Graphon Filters: Graph Signal Processing in the Limit

Almost sure convergence of extreme order statistics

The cut norm and Sampling Lemmas for unbounded kernels

Convergence Rate of Gibbs Sampler and Its Application