Abstract:Background: As generative artificial intelligence (GenAI) tools continue advancing, rigorous evaluations are needed to understand their capabilities relative to experienced clinicians and nurses. The aim of this study was to objectively compare the diagnostic accuracy and response formats of ICU nurses versus various GenAI models, with a qualitative interpretation of the quantitative results. Methods: This formative study utilized four written clinical scenarios representative of real ICU patient cases to simulate diagnostic challenges. The scenarios were developed by expert nurses and underwent validation against current literature. Seventy-four ICU nurses participated in a simulation-based assessment involving four written clinical scenarios. Simultaneously, we asked ChatGPT-4 and Claude-2.0 to provide initial assessments and treatment recommendations for the same scenarios. The responses from ChatGPT-4 and Claude-2.0 were then scored by certified ICU nurses for accuracy, completeness and response. Results: Nurses consistently achieved higher diagnostic accuracy than AI across open-ended scenarios, though certain models matched or exceeded human performance on standardized cases. Reaction times also diverged substantially. Qualitative response format differences emerged such as concision versus verbosity. Variations in GenAI models system performance across cases highlighted generalizability challenges. Conclusions: While GenAI demonstrated valuable skills, experienced nurses outperformed in open-ended domains requiring holistic judgement. Continued development to strengthen generalized decision-making abilities is warranted before autonomous clinical integration. Response format interfaces should consider leveraging distinct strengths. Rigorous mixed methods research involving diverse stakeholders can help iteratively inform safe, beneficial human-GenAI partnerships centred on experience-guided care augmentation. Relevance to clinical practice: This mixed-methods simulation study provides formative insights into optimizing collaborative models of GenAI and nursing knowledge to support patient assessment and decision-making in intensive care. The findings can help guide development of explainable GenAI decision support tailored for critical care environments. Patient or public contribution: Patients or public were not involved in the design and implementation of the study or the analysis and interpretation of the data.

Simulations in human‐AI crisis management systems

The Rise of AI-Driven Simulators: Building a New Crystal Ball

The role of artificial intelligence in nuclear crisis decision making: a complement, not a substitute

AI and crisis leadership: Using the POP-DOC Loop to explore potential implications and opportunities for leaders

Simulating Public Administration Crisis: A Novel Generative Agent-Based Simulation System to Lower Technology Barriers in Social Science Research

Agent-based social simulations for health crises response: utilising the everyday digital health perspective

Strategic Insights from Simulation Gaming of AI Race Dynamics

A Simulation System Towards Solving Societal-Scale Manipulation

Artificial intelligence for wargaming and modeling

Multi-AI Complex Systems in Humanitarian Response

An AI Framework and a Metamodel for Collaborative Situations: Application to Crisis Management Contexts

Augmenting intensive care unit nursing practice with generative AI: A formative study of diagnostic synergies using simulation-based clinical cases

The Role of Artificial Intelligence Technologies in Crisis Response

An Integrative Simulation to Study Team Cognition in Emergency Crisis Management

AI-Assisted Ethics? Considerations of AI Simulation for the Ethical Assessment and Design of Assistive Technologies

Directions for the Development of Social Sciences and Humanities in the Context of Creating Artificial General Intelligence

Artificial Intelligence and Healthcare Simulation: The Shifting Landscape of Medical Education

Using artificial intelligence to detect crisis related to events: Decision making in B2B by artificial intelligence

AI Agents in Emergency Response Applications

Inhuman Rhetoric: Generative AI and Crisis Communication

Crowd simulation for crisis management: The outcomes of the last decade