Spatiotemporal Scene-Graph Embedding for Autonomous Vehicle Collision Prediction

P. Khargonekar,Brandon Hsu,S. Yu,M. A. Faruque,Deepan Muthirayan,A. Malawade
DOI: https://doi.org/10.1109/jiot.2022.3141044
IF: 10.6
2021-11-11
IEEE Internet of Things Journal
Abstract:In autonomous vehicles (AVs), early warning systems rely on collision prediction to ensure occupant safety. However, state-of-the-art methods using deep convolutional networks either fail at modeling collisions or are too expensive/slow, making them less suitable for deployment on AV edge hardware. To address these limitations, we propose SG2VEC, a spatiotemporal scene-graph embedding methodology that uses the graph neural network (GNN) and long short-term memory (LSTM) layers to predict future collisions via visual scene perception. We demonstrate that SG2VEC predicts collisions 8.11% more accurately and 39.07% earlier than the state-of-the-art method on synthesized data sets, and 29.47% more accurately on a challenging real-world collision data set. We also show that SG2VEC is better than the state of the art at transferring knowledge from synthetic data sets to real-world driving data sets. Finally, we demonstrate that SG2VEC performs inference $9.3\times $ faster with an 88.0% smaller model, 32.4% less power, and 92.8% less energy than the state-of-the-art method on the industry-standard Nvidia DRIVE PX 2 platform, making it more suitable for implementation on the edge.
Engineering,Computer Science
What problem does this paper attempt to address?