Achieving health equity through conversational AI: A roadmap for design and implementation of inclusive chatbots in healthcare

Tom Nadarzynski,Nicky Knights,Deborah Husbands,Cynthia A Graham,Carrie D Llewellyn,Tom Buchanan,Ian Montgomery,Damien Ridge,Cynthia A. Graham,Carrie D. Llewellyn
DOI: https://doi.org/10.1371/journal.pdig.0000492
2024-05-03
PLOS Digital Health
Abstract:The rapid evolution of conversational and generative artificial intelligence (AI) has led to the increased deployment of AI tools in healthcare settings. While these conversational AI tools promise efficiency and expanded access to healthcare services, there are growing concerns ethically, practically and in terms of inclusivity. This study aimed to identify activities which reduce bias in conversational AI and make their designs and implementation more equitable. A qualitative research approach was employed to develop an analytical framework based on the content analysis of 17 guidelines about AI use in clinical settings. A stakeholder consultation was subsequently conducted with a total of 33 ethnically diverse community members, AI designers, industry experts and relevant health professionals to further develop a roadmap for equitable design and implementation of conversational AI in healthcare. Framework analysis was conducted on the interview data. A 10-stage roadmap was developed to outline activities relevant to equitable conversational AI design and implementation phases: 1) Conception and planning, 2) Diversity and collaboration, 3) Preliminary research, 4) Co-production, 5) Safety measures, 6) Preliminary testing, 7) Healthcare integration, 8) Service evaluation and auditing, 9) Maintenance, and 10) Termination. We have made specific recommendations to increase conversational AI's equity as part of healthcare services. These emphasise the importance of a collaborative approach and the involvement of patient groups in navigating the rapid evolution of conversational AI technologies. Further research must assess the impact of recommended activities on chatbots' fairness and their ability to reduce health inequalities. Artificial intelligence (AI) systems like conversational chatbots are increasingly being used in healthcare settings. However, there are concerns that these AI tools may perpetuate biases and worsen health inequalities if they are not designed and implemented carefully with diverse populations in mind. Our study aimed to develop a roadmap to help ensure that AI chatbots for healthcare are created and deployed in an equitable and inclusive way. We reviewed existing guidelines on ethical AI use in healthcare and interviewed 33 stakeholders including community members, healthcare professionals, and AI developers. Based on this, we outlined a 10-stage roadmap covering key activities such as involving underrepresented communities in co-production, conducting research on AI biases, implementing safety protocols, evaluating real-world effectiveness, and planning for chatbot maintenance or termination while minimising disruption to care. The roadmap emphasises the importance of close collaboration between chatbot creators, healthcare providers, policymakers, and the communities being served. It provides a checklist of equity considerations at each stage of the chatbot lifecycle. Following this roadmap can help mitigate unfair biases, increase acceptability among diverse patient groups, and harness the potential of conversational AI to improve healthcare access and reduce inequalities.
What problem does this paper attempt to address?