Abstract:Scene graph is a structured representation of a scene that can clearly express the objects, attributes, and relationships between objects in the scene. As computer vision technology continues to develop, people are no longer satisfied with simply detecting and recognizing objects in images; instead, people look forward to a higher level of understanding and reasoning about visual scenes. For example, given an image, we want to not only detect and recognize objects in the image, but also understand the relationship between objects (visual relationship detection), and generate a text description (image captioning) based on the image content. Alternatively, we might want the machine to tell us what the little girl in the image is doing (Visual Question Answering (VQA)), or even remove the dog from the image and find similar images (image editing and retrieval), etc. These tasks require a higher level of understanding and reasoning for image vision tasks. The scene graph is just such a powerful tool for scene understanding. Therefore, scene graphs have attracted the attention of a large number of researchers, and related research is often cross-modal, complex, and rapidly developing. However, no relatively systematic survey of scene graphs exists at present. To this end, this survey conducts a comprehensive investigation of the current scene graph research. More specifically, we first summarize the general definition of the scene graph, then conducte a comprehensive and systematic discussion on the generation method of the scene graph (SGG) and the SGG with the aid of prior knowledge. We then investigate the main applications of scene graphs and summarize the most commonly used datasets. Finally, we provide some insights into the future development of scene graphs.

Road Scene Graph: A Semantic Graph-Based Scene Representation Dataset for Intelligent Vehicles

Toward Driving Scene Understanding: A Paradigm and Benchmark Dataset for Ego-Centric Traffic Scene Graph Representation

Road Graph Extraction via Transformer and Topological Representation

RoadFormer: Duplex Transformer for RGB-Normal Semantic Road Scene Parsing

Towards Satellite Image Road Graph Extraction: A Global-Scale Dataset and A Novel Method

Segment Anything Model for Road Network Graph Extraction

GraphAD: Interaction Scene Graph for End-to-end Autonomous Driving

Enhanced Scene Understanding and Situation Awareness for Autonomous Vehicles Based on Semantic Segmentation

Road Extraction from Overhead Images with Graph Neural Networks

nuScenes Knowledge Graph -- A comprehensive semantic representation of traffic scenes for trajectory prediction

One-shot Scene Graph Generation

GITSR: Graph Interaction Transformer-based Scene Representation for Multi Vehicle Collaborative Decision-making

Graph-based Topology Reasoning for Driving Scenes

Knowledge Graphs of Driving Scenes to Empower the Emerging Capabilities of Neurosymbolic AI

SemanticFormer: Holistic and Semantic Traffic Scene Representation for Trajectory Prediction using Knowledge Graphs

Unsupervised Traffic Scene Generation with Synthetic 3D Scene Graphs

Self-Supervised Road Layout Parsing with Graph Auto-Encoding

Semantic Relation Model and Dataset for Remote Sensing Scene Understanding

A Scene Understanding Network Based on Driving Scene

Knowledge-Embedded Routing Network for Scene Graph Generation

A Comprehensive Survey of Scene Graphs: Generation and Application