Large Language Models in Fire Engineering: An Examination of Technical Questions Against Domain Knowledge

Haley Hostetter,M. Z. Naser,Xinyan Huang,John Gales
2024-03-05
Abstract:This communication presents preliminary findings from comparing two recent chatbots, OpenAI's ChatGPT and Google's Bard, in the context of fire engineering by evaluating their responses in handling fire safety related queries. A diverse range of fire engineering questions and scenarios were created and examined, including structural fire design, fire prevention strategies, evacuation, building code compliance, and fire suppression systems (some of which resemble those commonly present in the Fire Protection exam (FPE)). The results reveal some key differences in the performance of the chatbots, with ChatGPT demonstrating a relatively superior performance. Then, this communication highlights the potential for chatbot technology to revolutionize fire engineering practices by providing instant access to critical information while outlining areas for further improvement and research. Evidently, and when it matures, this technology will likely be elemental to our engineers' practice and education.
Artificial Intelligence,Machine Learning,Computation and Language
What problem does this paper attempt to address?
This paper explores the potential applications of large language models (LLMs), such as OpenAI's ChatGPT and Google's Bard, in the field of fire engineering. The study evaluates the abilities of these two chatbots in handling queries related to fire safety and finds that ChatGPT performs relatively well. The paper suggests that chatbot technology has the potential to revolutionize fire engineering practices by providing instant access to critical information, while also indicating areas that require further improvement and research. As the technology matures, such technologies may become an integral part of engineers' practice and education. The study also compares different types of chatbot architectures and describes their strengths and weaknesses in understanding and generating human language. The paper tests ChatGPT and Bard through an Ask Me Anything approach, involving questions on topics such as structural fire design, prevention strategies, evacuation, building code compliance, and firefighting systems, to assess their knowledge and application abilities in the field of fire engineering.