Abstract:Despite the success of Heterogeneous Graph Neural Networks (HGNNs) in modeling real-world Heterogeneous Information Networks (HINs), challenges such as expressiveness limitations and over-smoothing have prompted researchers to explore Graph Transformers (GTs) for enhanced HIN representation learning. However, research on GT in HINs remains limited, with two key shortcomings in existing work: (1) A node's neighbors at different distances in HINs convey diverse semantics. Unfortunately, existing methods ignore such differences and uniformly treat neighbors within a given distance in a coarse manner, which results in semantic confusion. (2) Nodes in HINs have various types, each with unique semantics. Nevertheless, existing methods mix nodes of different types during neighbor aggregation, hindering the capture of proper correlations between nodes of diverse types. To bridge these gaps, we design an innovative structure named (k,t)-ring neighborhood, where nodes are initially organized by their distance, forming different non-overlapping k-ring neighborhoods for each distance. Within each k-ring structure, nodes are further categorized into different groups according to their types, thus emphasizing the heterogeneity of both distances and types in HINs naturally. Based on this structure, we propose a novel Hierarchical Heterogeneous Graph Transformer (HHGT) model, which seamlessly integrates a Type-level Transformer for aggregating nodes of different types within each k-ring neighborhood, followed by a Ring-level Transformer for aggregating different k-ring neighborhoods in a hierarchical manner. Extensive experiments are conducted on downstream tasks to verify HHGT's superiority over 14 baselines, with a notable improvement of up to 24.75% in NMI and 29.25% in ARI for node clustering task on the ACM dataset compared to the best baseline.

Heterogeneous Graph Transformer for Graph-to-Sequence Learning

Graph Transformer for Graph-to-Sequence Learning

AMR-To-Text Generation with Graph Transformer.

Heterogeneous Graph Transformer for Meta-structure Learning with Application in Text Classification

Best of Both Worlds: Advantages of Hybrid Graph Sequence Models

Densely Connected Graph Convolutional Networks for Graph-to-Sequence Learning

Graph2Seq: Graph to Sequence Learning with Attention-based Neural Networks.

Knowledge Graph Embedding with Diversity of Structures

HINormer: Representation Learning on Heterogeneous Information Networks with Graph Transformer.

Graph-to-Tree Neural Networks for Learning Structured Input-Output Translation with Applications to Semantic Parsing and Math Word Problem.

HHGT: Hierarchical Heterogeneous Graph Transformer for Heterogeneous Graph Representation Learning

Seq-HGNN: Learning Sequential Node Representation on Heterogeneous Graph

A Graph is Worth $K$ Words: Euclideanizing Graph using Pure Transformer

Unleashing the Power of Transformer for Graphs

Memory-Enhanced Transformer for Representation Learning on Temporal Heterogeneous Graphs

A Pure Transformer Pretraining Framework on Text-attributed Graphs

A Heterogeneous Graph to Abstract Syntax Tree Framework for Text-to-SQL

Better AMR-To-Text Generation with Graph Structure Reconstruction.

Heterogeneous Graph Neural Networks for Extractive Document Summarization

Hierarchical Transformer for Scalable Graph Learning

Transformers as Graph-to-Graph Models