Abstract:Network embedding aims to learn the low-dimensional representations/embeddings of vertices which preserve the structure and inherent properties of the networks. The resultant embeddings are beneficial to downstream tasks such as vertex classification and link prediction. A vast majority of real-world networks are coupled with a rich set of vertex attributes, which could be potentially complementary in learning better embeddings. Existing attributed network embedding models, with shallow or deep architectures, typically seek to match the representations in topology space and attribute space for each individual vertex by assuming that the samples from the two spaces are drawn uniformly. The assumption, however, can hardly be guaranteed in practice. Due to the intrinsic sparsity of sampled vertex sequences and incompleteness in vertex attributes, the discrepancy between the attribute space and the network topology space inevitably exists. Furthermore, the interactions among vertex attributes, a.k.a cross features, have been largely ignored by existing approaches. To address the above issues, in this paper, we propose Nettention, a self-attentive network embedding approach that can efficiently learn vertex embeddings on attributed network. Instead of sample-wise optimization, Nettention aggregates the two types of information through minimizing the difference between the representation distributions in the low-dimensional topology and attribute spaces. The joint inference is encapsulated in a generative adversarial training process, yielding better generalization performance and robustness. The learned distributions consider both locality-preserving and global reconstruction constraints which can be inferred from the learning of the adversarially regularized autoencoders. Additionally, a multi-head self-attention module is developed to explicitly model the attribute interactions. Extensive experiments on benchmark datasets have verified the effectiveness of the proposed Nettention model on a variety of tasks, including vertex classification and link prediction.

Understanding and Improvement of Adversarial Training for Network Embedding from an Optimization Perspective

Identity-Preserving Adversarial Training for Robust Network Embedding

Adversarial Representation Mechanism Learning for Network Embedding

Auditing Network Embedding: an Edge Influence Based Approach

Self-Training Enhanced: Network Embedding and Overlapping Community Detection With Adversarial Learning

Improving the transferability of adversarial examples with path tuning

Adversarial Robustness of Probabilistic Network Embedding for Link Prediction

Adversarial Training: embedding adversarial perturbations into the parameter space of a neural network to build a robust system

Improving the Robustness of Wasserstein Embedding by Adversarial PAC-Bayesian Learning

A View-Adversarial Framework for Multi-View Network Embedding

Adversarial Training Can Provably Improve Robustness: Theoretical Analysis of Feature Learning Process Under Structured Data

Interpreting and Improving Adversarial Robustness of Deep Neural Networks With Neuron Sensitivity

Interpreting Adversarial Examples by Activation Promotion and Suppression

Adversarial Deep Network Embedding for Cross-network Node Classification

Self-Attentive Attributed Network Embedding Through Adversarial Learning

Node Pair Information Preserving Network Embedding Based on Adversarial Networks

GANE: A Generative Adversarial Network Embedding

Optimization Models and Interpretations for Three Types of Adversarial Perturbations against Support Vector Machines

Discerning Edge Influence for Network Embedding

Towards A Unified Understanding and Improving of Adversarial Transferability

L G ] 1 9 Ju n 20 19 Convergence of Adversarial Training in Overparametrized Networks