Abstract:Network embedding aims to learn the low-dimensional representations/embeddings of vertices which preserve the structure and inherent properties of the networks. The resultant embeddings are beneficial to downstream tasks such as vertex classification and link prediction. A vast majority of real-world networks are coupled with a rich set of vertex attributes, which could be potentially complementary in learning better embeddings. Existing attributed network embedding models, with shallow or deep architectures, typically seek to match the representations in topology space and attribute space for each individual vertex by assuming that the samples from the two spaces are drawn uniformly. The assumption, however, can hardly be guaranteed in practice. Due to the intrinsic sparsity of sampled vertex sequences and incompleteness in vertex attributes, the discrepancy between the attribute space and the network topology space inevitably exists. Furthermore, the interactions among vertex attributes, a.k.a cross features, have been largely ignored by existing approaches. To address the above issues, in this paper, we propose Nettention, a self-attentive network embedding approach that can efficiently learn vertex embeddings on attributed network. Instead of sample-wise optimization, Nettention aggregates the two types of information through minimizing the difference between the representation distributions in the low-dimensional topology and attribute spaces. The joint inference is encapsulated in a generative adversarial training process, yielding better generalization performance and robustness. The learned distributions consider both locality-preserving and global reconstruction constraints which can be inferred from the learning of the adversarially regularized autoencoders. Additionally, a multi-head self-attention module is developed to explicitly model the attribute interactions. Extensive experiments on benchmark datasets have verified the effectiveness of the proposed Nettention model on a variety of tasks, including vertex classification and link prediction.

Attributed Network Embedding Based on Self-Attention Mechanism for Recommendation Method

Flexible Attributed Network Embedding.

Self-Attention Mechanism Enhanced User Interests Modeling for Personalized Recommendation Services in Cyber-Physical-Social Systems.

RTANet: Recommendation Target-Aware Network Embedding.

A United Approach to Learning Sparse Attributed Network Embedding

Attributed Social Network Embedding

AdaS&S: a One-Shot Supernet Approach for Automatic Embedding Size Search in Deep Recommender System

A Scalable Attribute-Aware Network Embedding System

Recommendation with Attribute-aware Product Networks: A Representation Learning Model

Mutually Reinforced Network Embedding: an Integrated Approach to Research Paper Recommendation

Self-Attentive Attributed Network Embedding Through Adversarial Learning

Sparse Attributed Network Embedding Via Adaptively Aggregating Neighborhood Information

Anchor Link Prediction Across Attributed Networks Via Network Embedding

Learning Distributed Representations for Recommender Systems with a Network Embedding Approach

Point-of-Interest Recommendation: Exploiting Self-Attentive Autoencoders with Neighbor-Aware Influence

Hyperbolic Attributed Network Embedding with Self-Adaptive Random Walks.

An Enhanced Attentive Implicit Relation Embedding for Social Recommendation

Deep Attributed Network Embedding by Preserving Structure and Attribute Information

Enhancing Attributed Graph Networks with Alignment and Uniformity Constraints for Session-based Recommendation

Long-short Interest Network with Graph-Based Method for Sequential Recommendation

Graph convolutional network and self-attentive for sequential recommendation