Why is a Rule-based Shock Early Warning System Not Accurate - a Case Study.

Tianhua Tang,Shan Nan,Ling Lin,Xinhao Jin,Weichao Liao,Xudong Lv
DOI: https://doi.org/10.1109/bibm52615.2021.9669458
2021-01-01
Abstract:Early recognition and treatment of shock are vital for critically ill patients. Timely shock treatment depends on the rapid identification of patients with potential shock, which is challenging in the busy working environment of ICU. Hypotension events are typically clinical signs of shock, which can be detected by monitoring patients’ blood pressure (BP). The monitor issues a sound alarm as soon as BP measured is below a preconfigured threshold. However, nurses have to filter out the valid hypotension events and report to the responsible intensivists. This process is error-prone due to nurses’ and intensivists’ workload and alert fatigue. Rule-based early warning systems have the potential to detect all hypotension events and filter out invalid hypotension events. Consequently, they lead to a lower workload and better performance. Nevertheless, the accuracy of the rule-based approach remains a question, especially considering the complex nature of intensive care. In order to validate the accuracy and understand the limitations of the rule-based approach, we carried out a case study in a Chinese hospital. A ruled-based system, named Shock Alert Assistant (SAA), had been developed and tested. This system screens the real-time BP of all patients every 5 minutes and sends alert messages to intensivists. In a one-month testing phase, SAA produced 619 shock early warnings for 82 patients. The SAA’s performance was compared against 126 nurses’ notes. Clinical experts checked the inconsistency of SAA and nurses’ notes and made a golden standard. Regarding SAA, the sensitivity is 87.6%, whereas the specificity is 10.7%. Among all the false-positive alerts, 52.7% are pure low systolic blood pressure without clinical meaning, which does not lead to shock. By analyzing the results, in addition to systolic blood pressure (SBP), mean arterial pressure (MAP) and medication order related to BP control should also be used to determine hypotension events.
What problem does this paper attempt to address?