ASHABot: An LLM-Powered Chatbot to Support the Informational Needs of Community Health Workers

Pragnya Ramjee,Mehak Chhokar,Bhuvan Sachdeva,Mahendra Meena,Hamid Abdullah,Aditya Vashistha,Ruchit Nagar,Mohit Jain
2024-09-17
Abstract:Community health workers (CHWs) provide last-mile healthcare services but face challenges due to limited medical knowledge and training. This paper describes the design, deployment, and evaluation of ASHABot, an LLM-powered, experts-in-the-loop, WhatsApp-based chatbot to address the information needs of CHWs in India. Through interviews with CHWs and their supervisors and log analysis, we examine factors affecting their engagement with ASHABot, and ASHABot's role in addressing CHWs' informational needs. We found that ASHABot provided a private channel for CHWs to ask rudimentary and sensitive questions they hesitated to ask supervisors. CHWs trusted the information they received on ASHABot and treated it as an authoritative resource. CHWs' supervisors expanded their knowledge by contributing answers to questions ASHABot failed to answer, but were concerned about demands on their workload and increased accountability. We emphasize positioning LLMs as supplemental fallible resources within the community healthcare ecosystem, instead of as replacements for supervisor support.
Human-Computer Interaction
What problem does this paper attempt to address?
### Problems the paper attempts to solve The paper aims to solve the problem of insufficient knowledge and training faced by Accredited Social Health Activists (ASHA) in India when providing medical services. Specifically, the paper describes the design, deployment, and evaluation process of ASHABot, a WhatsApp chatbot with expert participation based on a large language model (LLM), which is designed to meet the information needs of ASHA. ### Main problem background 1. **ASHA's working environment**: - ASHA is an important part of the Indian health system and is responsible for providing basic medical services, especially in maternal and child health. - ASHA usually receives 3 - 4 weeks of initial training and is paid through a task - incentive mechanism. - Although ASHA plays an important role in the community, they often have difficulty in providing services effectively due to their limited medical knowledge and insufficient training. 2. **Existing challenges**: - The reference materials available to ASHA are often insufficient to answer complex questions. - ASHA lacks opportunities for further training and skill improvement. - The monthly meetings held in local hospitals have limited time and cannot fully answer ASHA's individual questions. - There is a lack of qualified teachers, funds, space, and materials to effectively train and support ASHA. 3. **Existing technical solutions**: - Previous studies have attempted to use technologies such as voice messages, Interactive Voice Response Systems (IVRS), feature phone applications, and smartphone applications to support ASHA, but these tools have limited effectiveness due to the steep learning curve and ASHA's unfamiliarity with digital technologies. - Recently, chatbots have become a more user - friendly alternative because of their natural - language interfaces, 24x7 availability, multilingual support, and voice interaction capabilities. ### Design and features of ASHABot - **Platform selection**: ASHABot is integrated into WhatsApp, a platform already familiar to ASHA and Auxiliary Nurse Midwives (ANM). - **Knowledge base**: ASHABot is based on a customized knowledge base that contains the content of government guidelines and ASHA manuals. - **Expert participation**: When ASHABot is unable to answer a question, it will forward the question to ANM, collect their answers, generate a consensus answer, and return it to ASHA. - **Language support**: ASHABot supports Hindi and voice input and output, making it more suitable for ASHA and ANM with different literacy levels and technical capabilities. ### Research methods - **Pilot testing**: Multiple iterative pilot tests were carried out to ensure the usability and accuracy of the chatbot's answers. - **Field deployment**: A two - month field - deployment study was conducted in Salumbar district, Rajasthan, India, with the participation of 20 ASHA and 15 ANM. - **Interview and log analysis**: Through interviews and log data analysis, the usage of ASHABot by ASHA and ANM and the influencing factors were evaluated. ### Main findings - **ASHA's user experience**: - ASHABot provides a convenient channel for accessing detailed information at any time, enabling ASHA to study independently and save time. - The chatbot provides a private channel where ASHA can ask basic or sensitive questions without worrying about being questioned by ANM or doctors. - ASHA generally trusts the answers from the chatbot, especially when ASHABot admits that it does not know the answer. - **ANM's role**: - ANM expands their knowledge by providing answers, but expresses concerns about the increased workload and higher responsibilities. ### Conclusions and contributions - **Design and implementation**: A chatbot with expert participation based on LLM was designed, built, and deployed to support the information needs of low - skilled and low - literate ASHA. - **Evaluation and impact**: Through field deployment and interviews, the effectiveness of the chatbot was evaluated, and the factors affecting the use of the chatbot by ASHA and ANM were identified. - **Future directions**: The limitations and opportunities of using emerging AI technologies (such as LLM) to strengthen community health infrastructure were discussed, including personalization, incentive mechanisms, and continuous training.