Abstract:Graph neural networks (GNNs) have emerged as a promising solution to deal with unstructured data, outperforming traditional deep learning architectures. However, most of the current GNN models are designed to work with a single graph, which limits their applicability in many real-world scenarios where multiple graphs may be involved. To address this limitation, we propose a novel graph-based deep learning architecture to handle tasks where two sets of signals exist, each defined on a different graph. First we consider the setting where the input is represented as a signal on top of one graph (input graph) and the output is a graph signal defined over a different graph (output graph). For this setup, we propose a three-block architecture where we first process the input data using a GNN that operates over the input graph, then apply a transformation function that operates in a latent space and maps the signals from the input to the output graph, and finally implement a second GNN that operates over the output graph. Our goal is not to propose a single specific definition for each of the three blocks, but rather to provide a flexible approach to solve tasks involving data defined on two graphs. The second part of the paper addresses a self-supervised setup, where the focus is not on the output space but on the underlying latent space and, inspired by Canonical Correlation Analysis, we seek informative representations of the data that can be leveraged to solve a downstream task. By leveraging information from multiple graphs, the proposed architecture can capture more intricate relationships between different entities in the data. We test this in several experimental setups using synthetic and real world datasets, and observe that the proposed architecture works better than traditional deep learning architectures, showcasing the importance of leveraging the information of the two graphs.

Graph Neural Networks for Natural Language Processing in Human-Robot Interaction

Exploring Humanoid Robot Face Preference Using Brain Functional Connectivity and Graph Neural Network

Graph Neural Networks for Natural Language Processing: A Survey

Interactivity Recognition Graph Neural Network (IR-GNN) Model for Improving Human–Object Interaction Detection

Learning Human-Object Interactions by Graph Parsing Neural Networks

Multivariate, Multi-Frequency and Multimodal: Rethinking Graph Neural Networks for Emotion Recognition in Conversation

Adopting Graph Neural Networks to Analyze Human–Object Interactions for Inferring Activities of Daily Living

Everything is Connected: Graph Neural Networks

Modeling social interaction dynamics using temporal graph networks

Graph Neural Networks for Modelling Traffic Participant Interaction

A Hierarchical Framework for Interactive Behaviour Prediction of Heterogeneous Traffic Participants Based on Graph Neural Network

H2G2-Net: A Hierarchical Heterogeneous Graph Generative Network Framework for Discovery of Multi-Modal Physiological Responses

Design Requirements for Human-Centered Graph Neural Network Explanations

Exploiting the Structure of Two Graphs with Graph Neural Networks

Graph Neural Machine: A New Model for Learning with Tabular Data

Graph Neural Network Backend for Speaker Recognition

Learning Graph Representation of Person-specific Cognitive Processes from Audio-visual Behaviours for Automatic Personality Recognition

Scalable Graph Neural Networks for Heterogeneous Graphs

Graph Neural Network (GNN) in Image and Video Understanding Using Deep Learning for Computer Vision Applications