Can artificial intelligence improve medicine's uncomfortable relationship with Maths?

Alexandra Valetopoulou,Simon Williams,Hani J. Marcus
DOI: https://doi.org/10.1038/s41746-024-01168-8
IF: 15.2
2024-06-23
npj Digital Medicine
Abstract:npj Digital Medicine, Published online: 22 June 2024; doi:10.1038/s41746-024-01168-8 Can artificial intelligence improve medicine's uncomfortable relationship with Maths?
health care sciences & services,medical informatics
What problem does this paper attempt to address?
This paper attempts to address the issue of insufficient understanding of statistics in the medical field. Specifically, the authors explore the changes in medical professionals' ability and confidence in calculating Positive Predictive Value (PPV) with and without AI assistance by replicating the studies of Casscells et al. (1978) and Manrai et al. (2014). ### Main Issues: 1. **Insufficient understanding of statistics in the medical field**: - A 1978 study showed that most medical professionals could not correctly calculate the Positive Predictive Value (PPV) and overestimated its value. - 36 years later, Manrai et al.'s study found similar results, indicating that despite advances in medical education, this issue still persists. 2. **Can AI assistance improve medical professionals' statistical abilities**: - The authors explore whether AI assistance tools (such as ChatGPT) can improve medical professionals' ability and confidence in calculating PPV with the help of AI. ### Research Methods: - **Participants**: Including 20 attending physicians, 20 interns/residents, and 20 final-year medical students. - **Experimental Design**: Participants first answered questions without AI assistance, then reviewed AI responses and decided whether to modify their answers. - **Evaluation Metrics**: Recording the accuracy and confidence levels of participants with and without AI assistance. ### Results: - **Without AI assistance**: Only 17% of participants gave the correct answer. - **With AI assistance**: 60% of participants modified their answers, and 83% of them gave the correct answer with AI assistance. - **Confidence Levels**: Participants' confidence levels significantly increased from "slightly confident" to "fairly confident." ### Conclusion: - Despite 45 years of development in medical education, most medical professionals still cannot correctly calculate PPV. - AI assistance significantly improved participants' ability and confidence in calculating PPV. - Further exploration is needed on how to utilize generative AI tools to enhance medical professionals' understanding and application of statistics to promote data-driven clinical decision-making. Through this study, the authors hope to emphasize the need for more focus on the cultivation of probabilistic reasoning skills in modern medical education and to explore the potential role of AI in this process.