Abstract:In the era of Generative AI, Neurosymbolic AI is emerging as a powerful approach for tasks spanning from perception to cognition. The use of Neurosymbolic AI has been shown to achieve enhanced capabilities, including improved grounding, alignment, explainability, and reliability. However, due to its nascent stage, there is a lack of widely available real-world benchmark datasets tailored to Neurosymbolic AI tasks. To address this gap and support the evaluation of current and future methods, we introduce DSceneKG -- a suite of knowledge graphs of driving scenes built from real-world, high-quality scenes from multiple open autonomous driving datasets. In this article, we detail the construction process of DSceneKG and highlight its application in seven different tasks. DSceneKG is publicly accessible at: <a class="link-external link-https" href="https://github.com/ruwantw/DSceneKG" rel="external noopener nofollow">this https URL</a>

What problem does this paper attempt to address?

The problem that this paper attempts to solve is: currently, the benchmark datasets used to evaluate the capabilities of Neurosymbolic AI cannot accurately reflect the complexity of real - world scenarios, especially in the field of autonomous driving. Specifically, the existing benchmark datasets have significant differences from industrial - level knowledge graphs in terms of structure, modality, the degree of conformity to ontology, the in - degree and out - degree of nodes, cardinality, etc., resulting in limitations on their applicability and relevance in practical applications. To solve this problem, the author introduced DSceneKG - a suite of driving - scene knowledge graphs constructed from multiple publicly available autonomous - driving datasets. DSceneKG aims to provide a more realistic and practical benchmark to support the development and evaluation of Neurosymbolic AI methods. By capturing multi - modal data (such as LiDAR, camera, and GPS sensor data), DSceneKG can represent a wide range of driving scenes, including urban and rural environments, different weather conditions, and various traffic situations. In addition, the applications of DSceneKG cover seven different tasks: 1. **Entity Prediction** 2. **Scene Clustering/Typing** 3. **Scene Similarity Calculation** 4. **Cross - modal Retrieval** 5. **Root - Cause Analysis** 6. **Semantic Search** 7. **Knowledge Completion and Augmentation** These tasks demonstrate the potential of DSceneKG in machine perception, knowledge completion, causal reasoning, and cross - modal retrieval, thereby promoting the application and development of Neurosymbolic AI in industrial and academic fields. ### Formula Representation There is no complex formula derivation involved in the paper, but in order to ensure the correctness and readability of the formula format, if formulas are involved, they will be presented in Markdown format. For example: - **Average in - degree**: \[ \text{Avg. in - degree}=\frac{\sum_{i = 1}^{n}\text{in - degree}(i)}{n} \] - **Average out - degree**: \[ \text{Avg. out - degree}=\frac{\sum_{i = 1}^{n}\text{out - degree}(i)}{n} \] These formulas are used to describe the connection characteristics of nodes in the knowledge graph and are helpful for understanding the structural features of DSceneKG.

Knowledge Graphs of Driving Scenes to Empower the Emerging Capabilities of Neurosymbolic AI

NeuSyRE: Neuro-symbolic visual understanding and reasoning framework based on scene graph enrichment

nuScenes Knowledge Graph -- A comprehensive semantic representation of traffic scenes for trajectory prediction

A system of vision sensor based deep neural networks for complex driving scene analysis in support of crash risk assessment and prevention

A survey of neurosymbolic visual reasoning with scene graphs and common sense knowledge

A Survey on Knowledge Graph-based Methods for Automated Driving

Dynamic-Scene-Graph-Supported Visual Understanding of Autonomous Driving Scenarios

GraphAD: Interaction Scene Graph for End-to-end Autonomous Driving

Toward Driving Scene Understanding: A Paradigm and Benchmark Dataset for Ego-Centric Traffic Scene Graph Representation

doScenes: An Autonomous Driving Dataset with Natural Language Instruction for Human Interaction and Vision-Language Navigation

Knowledge-Embedded Routing Network for Scene Graph Generation

HKTSG: A Hierarchical Knowledge-guided Traffic Scene Graph Representation Learning Framework for Intelligent Vehicles

Scene-Driven Multimodal Knowledge Graph Construction for Embodied AI

DriveLM: Driving with Graph Visual Question Answering

Acquiring Qualitative Explainable Graphs for Automated Driving Scene Interpretation

Road Scene Graph: A Semantic Graph-Based Scene Representation Dataset for Intelligent Vehicles

CC-SGG: Corner Case Scenario Generation using Learned Scene Graphs

Towards Knowledge-driven Autonomous Driving

Graph-based Topology Reasoning for Driving Scenes

Enhancing scene understanding based on deep learning for end-to-end autonomous driving

Exploring knowledge graph-based neural-symbolic system from application perspective