Abstract:Large language models (LLMs) have exhibited great potential in fault diagnosis of heating, ventilation, and air conditioning systems. However, the fault diagnosis accuracy of LLMs is still unsatisfactory, due to the lack of effective diagnosis accuracy enhancement methods for LLMs. To fill this gap, this study proposes a LLM finetuning method supervised by data with fault and fault-free labels to enhance the fault diagnosis accuracy of LLMs. This method designs a LLM self-correction strategy to automatically generate a fine-tuning dataset based on the labeled data. The generated fine-tuning dataset is applied to fine-tune a LLM. Moreover, a data augmentation-based approach is put forward to adaptively update the fine-tuning dataset for iteratively developing a high-performance fine-tuned LLM. The proposed method is utilized to fine-tune the GPT-3.5 model using the air handling unit (AHU) fault dataset from the RP-1312 project. The results show that the diagnosis accuracy of the GPT-3.5 model is increased from 29.5 % to 100.0 % after model fine-tuning. Compared with the GPT-4 model, the fine-tuned GPT-3.5 model achieves a 31.1 % higher average diagnosis accuracy. The fine-tuned GPT-3.5 model is also applied to diagnose faults in two AHUs from another open-source dataset to verify the generalization ability of this model. The two AHUs have different system structures and sensor configurations compared to the AHU in the RP-1312 dataset, and this dataset is not utilized to fine-tune the GPT-3.5 model. The average diagnosis accuracy of the GPT-3.5 model is increased from 46.0 % to 99.1 % and from 38.8 % to 98.9 % for the faults in the two AHUs, respectively, after model fine-tuning. Furthermore, the proposed method is verified using two fault datasets from a variable air volume box and a chiller plant system. After fine-tuning the GPT-3.5 model using the two datasets, the average diagnosis accuracy of this model is increased from 33.0 % to 98.3 % for variable air volume box faults and from 36.0 % to 99.1 % for chiller plant system faults. This study provides an effective solution to the development of domain-specific LLMs for this domain.

Empirical Study on Fine-Tuning Pre-Trained Large Language Models for Fault Diagnosis of Complex Systems

Evaluation and Improvement of Fault Detection for Large Language Models

Consultation on Industrial Machine Faults with Large language Models

Fault Diagnosis in Power Grids with Large Language Model

Domain-specific Large Language Models for Fault Diagnosis of Heating, Ventilation, and Air Conditioning Systems by Labeled-Data-supervised Fine-Tuning

Prompt Engineering Approach Study for Supervised Fine-Tuned (SFT) Large Language Models (LLMs) in Spacecraft Fault Diagnosis

Fine-Tuning Medical Language Models for Enhanced Long-Contextual Understanding and Domain Expertise

Improving Clinical Expertise in Large Language Models Using Electronic Medical Records

Large Language Models for Disease Diagnosis: A Scoping Review

FD-LLM: Large Language Model for Fault Diagnosis of Machines

Explainable Fault Diagnosis of Control Systems Using Large Language Models

Diagnosing Robotics Systems Issues with Large Language Models

Adapting Large Language Models to Log Analysis with Interpretable Domain Knowledge

Combining Insights From Multiple Large Language Models Improves Diagnostic Accuracy

LLM-based Framework for Bearing Fault Diagnosis

Software Testing with Large Language Models: Survey, Landscape, and Vision

Integrating LLMs for Explainable Fault Diagnosis in Complex Systems

A novel multiscale hybrid neural network for intelligent fine-grained fault diagnosis

Technical Language Supervision for Intelligent Fault Diagnosis in Process Industry

Large Language Models for Test-Free Fault Localization

D-Bot: Database Diagnosis System using Large Language Models