Abstract:Qualitative data collection and analysis approaches, such as those employing interviews and focus groups, provide rich insights into customer attitudes, sentiment, and behavior. However, manually analyzing qualitative data requires extensive time and effort to identify relevant topics and thematic insights. This study proposes a novel approach to address this challenge by leveraging Retrieval Augmented Generation (RAG) based Large Language Models (LLMs) for analyzing interview transcripts. The novelty of this work lies in strategizing the research inquiry as one that is augmented by an LLM that serves as a novice research assistant. This research explores the mental model of LLMs to serve as novice qualitative research assistants for researchers in the talent management space. A RAG-based LLM approach is extended to enable topic modeling of semi-structured interview data, showcasing the versatility of these models beyond their traditional use in information retrieval and search. Our findings demonstrate that the LLM-augmented RAG approach can successfully extract topics of interest, with significant coverage compared to manually generated topics from the same dataset. This establishes the viability of employing LLMs as novice qualitative research assistants. Additionally, the study recommends that researchers leveraging such models lean heavily on quality criteria used in traditional qualitative research to ensure rigor and trustworthiness of their approach. Finally, the paper presents key recommendations for industry practitioners seeking to reconcile the use of LLMs with established qualitative research paradigms, providing a roadmap for the effective integration of these powerful, albeit novice, AI tools in the analysis of qualitative datasets within talent

Apprentices to Research Assistants: Advancing Research with Large Language Models

Large Language Models in Qualitative Research: Can We Do the Data Justice?

LLMs as Research Tools: A Large Scale Survey of Researchers' Usage and Perceptions

Exploring the landscape of large language models: Foundations, techniques, and challenges

An Interdisciplinary Outlook on Large Language Models for Scientific Research

Practical Applications of Large Language Models for Health Care Professionals and Scientists

Challenges and Applications of Large Language Models

A Comprehensive Overview of Large Language Models

An analysis of large language models: their impact and potential applications

Large Language Models Meet NLP: A Survey

Large Language Model for Qualitative Research -- A Systematic Mapping Study

A Review of Current Trends, Techniques, and Challenges in Large Language Models (LLMs)

Enhancing qualitative research in psychology with large language models: a methodological exploration and examples of simulations

Reconciling Methodological Paradigms: Employing Large Language Models as Novice Qualitative Research Assistants in Talent Management Research

A Primer on Large Language Models and their Limitations

Large Language Models Humanize Technology

Large language models and academic writing: Five tiers of engagement

To prompt or not to prompt: Navigating the use of large language models for integrating and modeling heterogeneous data

Survey of different Large Language Model Architectures: Trends, Benchmarks, and Challenges

An Examination of the Use of Large Language Models to Aid Analysis of Textual Data