Abstract:Graph convolution operator of the GCN model is originally motivated from a localized first-order approximation of spectral graph convolutions. This work stands on a different view; establishing a \textit{mathematical connection between graph convolution and graph-regularized PCA} (GPCA). Based on this connection, GCN architecture, shaped by stacking graph convolution layers, shares a close relationship with stacking GPCA. We empirically demonstrate that the \textit{unsupervised} embeddings by GPCA paired with a 1- or 2-layer MLP achieves similar or even better performance than GCN on semi-supervised node classification tasks across five datasets including Open Graph Benchmark \footnote{\url{<a class="link-external link-https" href="https://ogb.stanford.edu/" rel="external noopener nofollow">this https URL</a>}}. This suggests that the prowess of GCN is driven by graph based regularization. In addition, we extend GPCA to the (semi-)supervised setting and show that it is equivalent to GPCA on a graph extended with "ghost" edges between nodes of the same label. Finally, we capitalize on the discovered relationship to design an effective initialization strategy based on stacking GPCA, enabling GCN to converge faster and achieve robust performance at large number of layers. Notably, the proposed initialization is general-purpose and applies to other GNNs.

What problem does this paper attempt to address?

The main problem that this paper attempts to solve is to explore the mathematical connection between Graph Convolutional Network (GCN) and Graph - regularized Principal Component Analysis (GPCA), and through this connection, gain in - depth understanding of the effectiveness and limitations of GCN. Specifically, the researchers hope to solve the problem through the following points: 1. **Establish the mathematical connection between GCN and GPCA**: By regarding the graph convolution operation in GCN as the first - order approximation of GPCA, the relationship between the two is revealed. 2. **Evaluate the performance of GPCA in node classification tasks**: Verify whether using only GPCA (coupled with a simple Multi - Layer Perceptron MLP) can achieve or exceed the performance of GCN in semi - supervised node classification tasks. 3. **Extend GPCA to (semi - ) supervised settings**: Introduce "ghost edges", that is, add virtual edges between nodes with the same label, to enhance the expressive power of GPCA. 4. **Design an effective initialization strategy**: Based on the characteristics of GPCA, propose a new initialization method, enabling GCN to converge faster and maintain robust performance in deeper situations. ### Specific Problem Description #### 1. Mathematical Connection The paper first establishes the mathematical connection between the graph convolution operation in GCN and GPCA. Specifically, the graph convolution operation can be regarded as a first - order approximation form of GPCA. The formula is as follows: \[ Z^* \approx \tilde{A}_{sym} X W^* \] where \(\tilde{A}_{sym}\) is the symmetrically normalized adjacency matrix, \(X\) is the feature matrix, and \(W^*\) is the optimal projection matrix obtained from the GPCA solution. #### 2. Performance Evaluation To verify the effectiveness of GPCA, the researchers conducted experiments on multiple benchmark datasets and compared the performance of single - layer unsupervised GPCA (coupled with 1 - 2 layers of MLP) with multi - layer GCN. The results show that in some cases, the performance of GPCA is even better than that of GCN. #### 3. Extension to (Semi - ) Supervised Settings The paper further extends GPCA to (semi - ) supervised settings. By adding "ghost edges" between nodes with the same label, the expressive power of the model is enhanced. This extension enables GPCA to achieve better performance on some datasets. #### 4. Initialization Strategy Based on the above findings, the researchers proposed a new initialization strategy - GPCA NET - initialization. This strategy initializes GCN using the characteristics of GPCA, significantly improving the convergence speed and performance stability of GCN in deep structures. ### Conclusion Through these studies, the paper not only deepens the understanding of GCN but also provides new perspectives and tools for future research. In particular, it shows the potential of graph - regularized techniques in improving the performance of graph neural networks and provides an effective method for the initialization of GNN. Hope this information can help you better understand the core problems of this paper and their solutions. If you have more questions or need further explanation, please feel free to let me know!

Connecting Graph Convolutional Networks and Graph-Regularized PCA

Graph Learning in Low Dimensional Space for Graph Convolutional Networks

Semi-supervised classification by graph p-Laplacian convolutional networks

Graph Construction using Principal Axis Trees for Simple Graph Convolution

Deeper Insights Into Graph Convolutional Networks for Semi-Supervised Learning

Unified GCNs: Towards Connecting GCNs with CNNs

Deep Graph-Convolutional Generative Adversarial Network for Semi-Supervised Learning on Graphs

Explainable, Stable, and Scalable Graph Convolutional Networks for Learning Graph Representation

Graph convolutional networks of reconstructed graph structure with constrained Laplacian rank

From Cluster Assumption to Graph Convolution: Graph-based Semi-Supervised Learning Revisited

HesGCN: Hessian graph convolutional networks for semi-supervised classification

Graph Convolutional Network with elastic topology

A Unified Framework for Convolution-Based Graph Neural Networks

PCGCN: Partition-Centric Processing for Accelerating Graph Convolutional Network

Perturbation-augmented Graph Convolutional Networks: A Graph Contrastive Learning architecture for effective node classification tasks

Permutohedral-GCN: Graph Convolutional Networks with Global Attention

Semi-Supervised Classification of Graph Convolutional Networks with Laplacian Rank Constraints

Dynamic Graph Learning Convolutional Networks for Semi-supervised Classification

Revisiting 2D Convolutional Neural Networks for Graph-based Applications

A deep graph convolutional neural network architecture for graph classification

Exploring Structure-Adaptive Graph Learning for Robust Semi-Supervised Classification