Abstract:We propose a heterogeneous graph mamba network (HGMN) as the first exploration in leveraging the selective state space models (SSSMs) for heterogeneous graph learning. Compared with the literature, our HGMN overcomes two major challenges: (i) capturing long-range dependencies among heterogeneous nodes and (ii) adapting SSSMs to heterogeneous graph data. Our key contribution is a general graph architecture that can solve heterogeneous nodes in real-world scenarios, followed an efficient flow. Methodologically, we introduce a two-level efficient tokenization approach that first captures long-range dependencies within identical node types, and subsequently across all node types. Empirically, we conduct comparisons between our framework and 19 state-of-the-art methods on the heterogeneous benchmarks. The extensive comparisons demonstrate that our framework outperforms other methods in both the accuracy and efficiency dimensions.

What problem does this paper attempt to address?

### Problems the Paper Aims to Solve This paper aims to address two main challenges in Heterogeneous Graphs: 1. **Long-Distance Dependency**: In heterogeneous graphs, there may be long-distance dependencies between nodes. For example, in a sparse network like IMDB, which has 21,000 nodes and only 87,000 edges, it is necessary to utilize information from distant neighbors to enhance node embeddings. This dependency is particularly complex in heterogeneous graphs because different types of nodes and edges increase the processing difficulty. 2. **Graph-to-Sequence Conversion**: Converting unordered graph data into an ordered sequence structure to effectively utilize Selective State-Space Models (SSSMs) for processing. This process needs to consider the characteristics of heterogeneous graphs to ensure that the model can capture the complex structures and dependencies within the graph. To address these challenges, the authors propose the **Heterogeneous Graph Mamba Network (HGMN)**. HGMN introduces a data-dependent state transition mechanism that can capture long-distance dependencies in large-scale heterogeneous graphs while reducing computational costs. Specifically, the design of HGMN includes the following six steps: 1. **Tokenization**: Mapping the graph into a series of tokens, where each token is a subgraph of a target node and its meta-path instances. 2. **Heterogeneity Alignment**: Projecting different types of nodes into the same latent representation space, aggregating each token (subgraph) to update the representation of the target node. 3. **Inner Ordering**: Grouping by node type and sorting within each group based on the number of meta-path instances, reflecting the importance of nodes within their type. 4. **Inner Updating**: Using the Mamba mechanism to scan and select relevant nodes for updating. Due to recursive updates, each token can only obtain information from previous tokens, so more important nodes are placed at the end of the sequence to maximize their visibility and impact. 5. **Outer Ordering**: Sorting all types of nodes by node degree and applying a similar updating process across all node types to ensure global updates. 6. **Outer Updating**: Applying the updating process across all node types to ensure comprehensive updates and capture long-distance heterogeneous dependencies. Through these steps, HGMN can generate final node embeddings for downstream tasks such as node prediction, ensuring that the system can capture long-distance heterogeneous dependencies. Experimental results show that HGMN performs excellently in multiple benchmarks, surpassing existing Transformer-based and sparse attention methods in both accuracy and efficiency.

HeteGraph-Mamba: Heterogeneous Graph Learning via Selective State Space Model

Graph Mamba: Towards Learning on Graphs with State Space Models

Graph-Mamba: Towards Long-Range Graph Sequence Modeling with Selective State Spaces

Heterogeneous Graph Neural Network With Multi-View Representation Learning

STG-Mamba: Spatial-Temporal Graph Learning via Selective State Space Model

What Can We Learn from State Space Models for Machine Learning on Graphs?

HGAMLP: Heterogeneous Graph Attention MLP with De-Redundancy Mechanism

Multi-Hyperbolic Space-based Heterogeneous Graph Attention Network

Hyperbolic Heterogeneous Graph Attention Networks

Heterogeneous Deep Graph Infomax

HiGPT: Heterogeneous Graph Language Model

HGBER: Heterogeneous Graph Neural Network With Bidirectional Encoding Representation

Heterogeneous graph neural networks analysis: a survey of techniques, evaluations and applications

Meta-path-based heterogeneous graph neural networks in academic network

Addressing Heterogeneity and Heterophily in Graphs: A Heterogeneous Heterophilic Spectral Graph Neural Network

Heterogeneous Graph Tree Networks

GraphMSE: Efficient Meta-path Selection in Semantically Aligned Feature Space for Graph Neural Networks

Metapaths guided Neighbors aggregated Network for?Heterogeneous Graph Reasoning

Heterogeneous Learning for Scene Graph Generation

Megnn: Meta-path extracted graph neural network for heterogeneous graph representation learning

DynHEN: A heterogeneous network model for dynamic bipartite graph representation learning