Abstract:Background Analysing large qualitative datasets can present significant challenges, including the time and resources required for manual analysis and the potential for missing nuanced insights. This paper aims to address these challenges by exploring the application of Big Qualitative (Big Qual) methods and artificial intelligence (AI) methods to efficiently analyse Big Qual data while retaining the depth and complexity of human understanding. The free-text responses from the Royal College of Anaesthetists' 7th National Audit Project (NAP7) baseline survey on peri-operative cardiac arrest experiences serve as a case study to test and validate this approach. Methodology/Principal Findings Quantitative analysis segmented the data and identified keywords using AI methods. In-depth sentiment and thematic analysis combined natural language processing (NLP) and machine learning (ML) with human input - researchers assigned topic/theme labels and sentiments to responses, while discourse analysis explored sub-topics and thematic diversity. Human annotation refined the machine-generated sentiments, leading to an additional "ambiguous" category to capture nuanced, mixed responses. Comparative analysis was used to evaluate the concordance between human and machine-assisted sentiment labelling. While ML reduced analysis time significantly, human input was crucial for refining sentiment categories and capturing nuances. Conclusions/Significance The application of AI-assisted data analysis tools, combined with human expertise, offers a powerful approach to efficiently analyse large-scale qualitative datasets while preserving the nuance and complexity of the data. This study demonstrates the potential of this novel methodology to streamline the analysis process, reduce resource requirements, and generate meaningful insights from Big Qual data. The integration of NLP, ML, and human input allows for a more comprehensive understanding of the themes, sentiments, and experiences captured in free-text responses. This study underscores the importance of continued interdisciplinary collaboration among domain experts, data scientists, and AI specialists to optimise these methods, ensuring their reliability, validity, and ethical application in real-world contexts.

Supporting Serendipity: Opportunities and Challenges for Human-AI Collaboration in Qualitative Analysis

Human-AI Collaboration in Thematic Analysis using ChatGPT: A User Study and Design Recommendations

Can AI Serve as a Substitute for Human Subjects in Software Engineering Research?

CoAIcoder: Examining the Effectiveness of AI-assisted Human-to-Human Collaboration in Qualitative Analysis

Impact of Human-AI Interaction on User Trust and Reliance in AI-Assisted Qualitative Coding

Human-Centric AI: Understanding and Enhancing Collaboration between Humans and Intelligent Systems

AI and Citizen Science for Serendipity

From Human-Human Collaboration to Human-AI Collaboration

“Collaborating” with AI: Taking a System View to Explore the Future of Work

Human-AI Collaboration in a Cooperative Game Setting

Challenges of Human-Aware AI Systems

Agency plus automation: Designing artificial intelligence into interactive systems

Human-AI Collaboration in Data Science: Exploring Data Scientists' Perceptions of Automated AI

Studying the Effects of Collaboration in Interactive Theme Discovery Systems

Human-Centered AI Transformation: Exploring Behavioral Dynamics in Software Engineering

Deconstructing Human-AI Collaboration: Agency, Interaction, and Adaptation

Comparing human vs. machine-assisted analysis to develop a new approach for Big Qualitative Data Analysis

Synergizing Human-AI Agency: A Guide of 23 Heuristics for Service Co-Creation with LLM-Based Agents

Why is AI not a Panacea for Data Workers? An Interview Study on Human-AI Collaboration in Data Storytelling

Human-AI teams—Challenges for a team-centered AI at work