Abstract:In service-oriented architectures, accurately predicting the Quality of Service (QoS) is crucial for maintaining reliability and enhancing user satisfaction. However, significant challenges remain due to existing methods always overlooking high-order latent collaborative relationships between users and services and failing to dynamically adjust feature learning for every specific user-service invocation, which are critical for learning accurate features. Additionally, reliance on RNNs for capturing QoS evolution hampers models' ability to detect long-term trends due to difficulties in managing long-range dependencies. To address these challenges, we propose the \underline{T}arget-Prompt \underline{O}nline \underline{G}raph \underline{C}ollaborative \underline{L}earning (TOGCL) framework for temporal-aware QoS prediction. TOGCL leverages a dynamic user-service invocation graph to model historical interactions, providing a comprehensive representation of user-service relationships. Building on this graph, it develops a target-prompt graph attention network to extract online deep latent features of users and services at each time slice, simultaneously considering implicit collaborative relationships between target users/services and their neighbors, as well as relevant historical QoS values. Additionally, a multi-layer Transformer encoder is employed to uncover temporal feature evolution patterns of users and services, leading to temporal-aware QoS prediction. Extensive experiments conducted on the WS-DREAM dataset demonstrate that our proposed TOGCL framework significantly outperforms state-of-the-art methods across multiple metrics, achieving improvements of up to 38.80\%. These results underscore the effectiveness of the TOGCL framework for precise temporal QoS prediction.

What problem does this paper attempt to address?

The main problem this paper attempts to address is transferring the powerful semantic understanding and entity relationship comprehension capabilities of large language models (LLMs) to graph neural networks (GNNs) to improve the performance of GNNs when handling text-attributed graphs (TAGs). Specifically, the paper proposes a framework called Linguistic Graph Knowledge Distillation (LinguGKD) to overcome the following challenges: 1. **Computational and storage demands**: Although LLMs perform excellently in handling complex tasks, their deployment is limited by high computational and storage demands. This makes it difficult to deploy LLMs in resource-constrained environments. 2. **Balancing semantic understanding and structural processing**: Traditional GNNs are very efficient in handling graph structures but lack in capturing the complex semantic information in text data. While LLMs excel in semantic understanding, they are less efficient in processing large-scale graph data. 3. **Challenges of knowledge transfer**: Due to the different design purposes and architectures of LLMs and GNNs, effectively transferring the knowledge from LLMs to GNNs is an underexplored problem. To address these issues, the LinguGKD framework achieves knowledge transfer through the following methods: - **Instruction tuning**: First, the pre-trained LLM is instruction-tuned to understand and process graph structures and node semantics, generating an effective teacher model (LinguGraph LLM). - **Hierarchical adaptive contrastive distillation**: A hierarchical adaptive contrastive distillation strategy is introduced, which synchronizes the feature spaces of the LLM and GNN through a feature alignment mechanism, ensuring that the deep semantic knowledge and complex graph structure understanding of the teacher LLM can be effectively transferred to the student GNN. - **Experimental validation**: Extensive experimental evaluations validate the effectiveness of the LinguGKD framework across various LLM and GNN models and multiple benchmark datasets, demonstrating its significant advantages in improving GNN classification accuracy, accelerating convergence speed, and reducing model complexity. In summary, this paper aims to enhance the performance and efficiency of graph data processing tasks by combining the semantic understanding of LLMs and the structural processing advantages of GNNs through the LinguGKD framework.

Large Language Model Meets Graph Neural Network in Knowledge Distillation

GACL: Graph Attention Collaborative Learning for Temporal QoS Prediction

Recurrent Neural Network Based Collaborative Filtering for QoS Prediction in IoV

Time-Aware and Sparsity-Tolerant QoS Prediction Based on Collaborative Filtering

Language Models-enhanced Semantic Topology Representation Learning for Temporal Knowledge Graph Extrapolation

Predicting Quality of Services Based on a Two-stream Deep Learning Model with User and Service Graphs

A Novel Deep-Learning-Based QoS Prediction Model for Service Recommendation Utilizing Multi-Stage Multi-Scale Feature Fusion With Individual Evaluations

CogMG: Collaborative Augmentation Between Large Language Model and Knowledge Graph

TPMCF: Temporal QoS Prediction Using Multi-Source Collaborative Features

Graph Neural Prompting with Large Language Models

QoSGNN: Boosting QoS Prediction Performance with Graph Neural Networks

Topology-Aware Neural Model for Highly Accurate QoS Prediction

Robust QoS Prediction based on Reputation Integrated Graph Convolution Network

A Novel Spatiotemporal Coupling Graph Convolutional Network

Temporal Knowledge Graph Forecasting Without Knowledge Using In-Context Learning

Knowledge Graph Large Language Model (KG-LLM) for Link Prediction

Anomaly Resilient Temporal QoS Prediction using Hypergraph Convoluted Transformer Network

Joint Estimation and Prediction of City-wide Delivery Demand: A Large Language Model Empowered Graph-based Learning Approach

TCL: Transformer-based Dynamic Graph Modelling Via Contrastive Learning

Location-aware Collaborative Filtering and Feature Interaction Learning for QoS Prediction

Debate on Graph: a Flexible and Reliable Reasoning Framework for Large Language Models