Practical Guide to Machine Learning and Artificial Intelligence in Surgical Education Research

Daniel A. Hashimoto,Julian Varas,Todd A. Schwartz

DOI: https://doi.org/10.1001/jamasurg.2023.6687

2024-01-03

JAMA Surgery

Abstract:This Guide to Statistics and Methods gives an overview of artificial intelligence techniques and tools in surgical education research.

surgery

What problem does this paper attempt to address?

The Application of Artificial Intelligence to Medical Education in the Last Decade - a Bibliometric Analysis (Preprint)

Shuang Wang,Yang Liu,Min Li

DOI: https://doi.org/10.2196/preprints.51411

2023-01-01

Abstract:As an important driving force leading a new round of scientific and technological revolution and industrial transformation, Artificial Intelligence (AI) has spawned a large number of new products, new technologies, new formats and new models, and also brought more possibilities for the modernization of medical education. Looking into the future, we should work together to build a high-quality and warm AI medical education ecology. How to make human-machine collaboration smarter and human-machine dialogue more friendly is a long-term topic of "AI + medical education" Education is dynamic and evolving, thinking rationally about the relationship between people and technology. Further promoting the deep integration and innovative development of AI and medical education can better empower the modernization of education. Use CiteSpace and VOSviewer to determine the current and emerging trends in Artificial Intelligence on medical education from 2013 to 2022. Search the literatures related to Artificial Intelligence on medical education in Web of Science core database from 2013 to 2022. Use CiteSpace and VOS viewers to analyze countries, institutions, authors, references and keywords that meet the requirements. We identified 195 articles about Artificial Intelligence on medical education from 2013 to 2022 and found that the annual incidence rate increased over time. The most active country was the United States, the most active institution was the Harvard Med Sch and Univ Toronto. And Bissonnette, Vincent; Blacketer, Charlotte; Del Maestro, Rolando f; Ledows, Nicole; Mirchi, Nykan; Winkler-schwartz, Alexander; Yilamaz, Recai was the leading authors. Besides, “Medical students' attitude towards Artificial Intelligence: a multicentre survey” was the largest number of citation papers. References and keyword analysis showed that “radiology”, “medical physics” ,“ehealth” ,“surgery”,and “specialty” were the focus of these studies, while “big data”, and “management” were the frontiers of research. The bibliometric study shows that the research of Artificial Intelligence on medical education is a promising research field. The current research is mainly focused on a few aspects of medical education, with the progress of technology, it is expected to broaden the direction in the future. At present, the urgent problem to be solved is to strengthen inter-regional cooperation and improve the quality of research. Our findings provide valuable information for researchers to determine a better perspective and develop future research directions.
Practical Guide to Survey Research in Surgical Education

Adnan A. Alseidi,Jason S. Haukoos,Christian de Virgilio

DOI: https://doi.org/10.1001/jamasurg.2023.6678

2024-03-14

JAMA Surgery

Abstract:This Guide to Statistics and Methods describes aspects of methods of survey research in surgical education, important considerations, and pitfalls and limitations.

surgery
The fundamentals of artificial intelligence in medical education research: AMEE Guide No. 156

Martin G. Tolsgaard,Martin V. Pusic,Stefanie S. Sebok-Syer,Brian Gin,Morten Bo Svendsen,Mark D. Syer,Ryan Brydges,Monica M. Cuddy,Christy K. Boscardin

DOI: https://doi.org/10.1080/0142159X.2023.2180340

IF: 4.277

2023-03-04

Medical Teacher

Abstract:The use of Artificial Intelligence (AI) in medical education has the potential to facilitate complicated tasks and improve efficiency. For example, AI could help automate assessment of written responses, or provide feedback on medical image interpretations with excellent reliability. While applications of AI in learning, instruction, and assessment are growing, further exploration is still required. There exist few conceptual or methodological guides for medical educators wishing to evaluate or engage in AI research. In this guide, we aim to: 1) describe practical considerations involved in reading and conducting studies in medical education using AI, 2) define basic terminology and 3) identify which medical education problems and data are ideally-suited for using AI.

education, scientific disciplines,health care sciences & services
Practical Guide to Surgical Simulation Research

Aimee K. Gardner,Amy H. Kaji,Marja Boermeester

DOI: https://doi.org/10.1001/jamasurg.2023.6684

2024-01-03

JAMA Surgery

Abstract:This Guide to Statistics and Methods provides an overview of simulation-based education and its impact on surgeon performance.

surgery
Practical Guide to Experimental and Quasi-Experimental Research in Surgical Education

Roy Phitayakorn,Todd A. Schwartz,Gerard M. Doherty

DOI: https://doi.org/10.1001/jamasurg.2023.6693

2024-01-03

JAMA Surgery

Abstract:This Guide to Statistics and Methods describes the methods and pitfalls of experimental and quasi-experimental study designs in surgical education.

surgery
A Surgeon's Guide to Artificial Intelligence-Driven Predictive Models

Abbas M Hassan,Aashish Rajesh,Malke Asaad,Jonas A Nelson,J Henk Coert,Babak J Mehrara,Charles E Butler

DOI: https://doi.org/10.1177/00031348221103648

Abstract:Artificial intelligence (AI) focuses on processing and interpreting complex information as well as identifying relationships and patterns among complex data. Artificial intelligence- and machine learning (ML)-driven predictions have shown promising potential in influencing real-time decisions and improving surgical outcomes by facilitating screening, diagnosis, risk assessment, preoperative planning, and shared decision-making. Fundamental understanding of the algorithms, as well as their development and interpretation, is essential for the evolution of AI in surgery. In this article, we provide surgeons with a fundamental understanding of AI-driven predictive models through an overview of common ML and deep learning algorithms, model development, performance metrics and interpretation. This would serve as a basis for understanding ML-based research, while fostering new ideas and innovations for furthering the reach of this emerging discipline.
Practical Guide to Ethics in Surgical Education Research

Michael M. Awad,Amy H. Kaji,Timothy M. Pawlik

DOI: https://doi.org/10.1001/jamasurg.2023.6705

2024-01-03

JAMA Surgery

Abstract:This Guide to Statistics and Methods describes ethical considerations when a study population includes learner participants.

surgery
A foundation for evaluating the surgical artificial intelligence literature

Daniel A Hashimoto,Sai Koushik Sambasastry,Vivek Singh,Sruthi Kurada,Maria Altieri,Takuto Yoshida,Amin Madani,Matjaz Jogan,Daniel A. Hashimoto

DOI: https://doi.org/10.1016/j.ejso.2024.108014

IF: 4.037

2024-02-01

European Journal of Surgical Oncology

Abstract:With increasing growth in applications of artificial intelligence (AI) in surgery, it has become essential for surgeons to gain a foundation of knowledge to critically appraise the scientific literature, commercial claims regarding products, and regulatory and legal frameworks that govern the development and use of AI. This guide offers surgeons a framework with which to evaluate manuscripts that incorporate the use of AI. It provides a glossary of common terms, an overview of prerequisite knowledge to maximize understanding of methodology, and recommendations on how to carefully consider each element of a manuscript to assess the quality of the data on which an algorithm was trained, the appropriateness of the methodological approach, the potential for reproducibility of the experiment, and the applicability to surgical practice, including considerations on generalizability and scalability.

oncology,surgery
Practical Guide to Qualitative Research in Surgical Education

Gurjit Sandhu,Amy H. Kaji,Amalia Cochran

DOI: https://doi.org/10.1001/jamasurg.2023.6681

2024-01-03

JAMA Surgery

Abstract:This Guide to Statistics and Methods provides an overview of the selection and application of qualitative research in surgical education.

surgery
Machine Learning and Artificial Intelligence for Surgical Decision Making

Saskya Byerly,Lydia R Maurer,Alejandro Mantero,Leon Naar,Gary An,Haytham M A Kaafarani

DOI: https://doi.org/10.1089/sur.2021.007

Abstract:Background: The use of machine learning (ML) and artificial intelligence (AI) in medical research continues to grow as the amount and availability of clinical data expands. These techniques allow complex interpretation of data and capture non-linear relations not immediately apparent by classic statistical techniques. Methods: This review of the ML/AI literature provides a brief overview for practicing surgeons and clinicians of the current and future roles these methods will have within surgical infection research. Results: A conceptual overview of the techniques is provided along with concrete examples in the surgical infections literature. Further examples of ML/AI techniques in clinical decision support as well as therapy discovery with model-based deep reinforcement learning are illustrated. Conclusions: Artificial intelligence and ML are important and increasingly utilized techniques within the expanding body of surgical infection research. This article provides a minimal baseline literacy in ML/AI to be able to view such projects in an appropriately critical fashion.
Artificial Intelligence in Surgical Learning

Niklas Pakkasjärvi,Tanvi Luthra,Sachit Anand

DOI: https://doi.org/10.3390/surgeries4010010

2023-02-17

Surgeries

Abstract:(1) Background: Artificial Intelligence (AI) is transforming healthcare on all levels. While AI shows immense potential, the clinical implementation is lagging. We present a concise review of AI in surgical learning; (2) Methods: A non-systematic review of AI in surgical learning of the literature in English is provided; (3) Results: AI shows utility for all components of surgical competence within surgical learning. AI presents with great potential within robotic surgery specifically; (4) Conclusions: Technology will evolve in ways currently unimaginable, presenting us with novel applications of AI and derivatives thereof. Surgeons must be open to new modes of learning to be able to implement all evidence-based applications of AI in the future. Systematic analyses of AI in surgical learning are needed.
Practical Guide to Common Flaws With Surgical Education Research

Dimitrios Stefanidis,Laura Torbeck,Amy H. Kaji

DOI: https://doi.org/10.1001/jamasurg.2023.6675

2024-03-14

JAMA Surgery

Abstract:This Guide to Statistics and Methods provides an overview of common flaws with surgical education research, including how to recognize and avoid them.

surgery
Surgical data science and artificial intelligence for surgical education

Thomas M. Ward,Pietro Mascagni,Amin Madani,Nicolas Padoy,Silvana Perretta,Daniel A. Hashimoto

DOI: https://doi.org/10.1002/jso.26496

2021-07-10

Journal of Surgical Oncology

Abstract:<p>Surgical data science (SDS) aims to improve the quality of interventional healthcare and its value through the capture, organization, analysis, and modeling of procedural data. As data capture has increased and artificial intelligence (AI) has advanced, SDS can help to unlock augmented and automated coaching, feedback, assessment, and decision support in surgery. We review major concepts in SDS and AI as applied to surgical education and surgical oncology.</p>

oncology,surgery
A practical guide to the implementation of artificial intelligence in orthopaedic research-Part 2: A technical introduction

Bálint Zsidai,Janina Kaarre,Eric Narup,Eric Hamrin Senorski,Ayoosh Pareek,Alberto Grassi,Christophe Ley,Umile Giuseppe Longo,Elmar Herbst,Michael T Hirschmann,Sebastian Kopf,Romain Seil,Thomas Tischer,Kristian Samuelsson,Robert Feldt,ESSKA Artificial Intelligence Working Group

DOI: https://doi.org/10.1002/jeo2.12025

2024-05-07

Abstract:Recent advances in artificial intelligence (AI) present a broad range of possibilities in medical research. However, orthopaedic researchers aiming to participate in research projects implementing AI-based techniques require a sound understanding of the technical fundamentals of this rapidly developing field. Initial sections of this technical primer provide an overview of the general and the more detailed taxonomy of AI methods. Researchers are presented with the technical basics of the most frequently performed machine learning (ML) tasks, such as classification, regression, clustering and dimensionality reduction. Additionally, the spectrum of supervision in ML including the domains of supervised, unsupervised, semisupervised and self-supervised learning will be explored. Recent advances in neural networks (NNs) and deep learning (DL) architectures have rendered them essential tools for the analysis of complex medical data, which warrants a rudimentary technical introduction to orthopaedic researchers. Furthermore, the capability of natural language processing (NLP) to interpret patterns in human language is discussed and may offer several potential applications in medical text classification, patient sentiment analysis and clinical decision support. The technical discussion concludes with the transformative potential of generative AI and large language models (LLMs) on AI research. Consequently, this second article of the series aims to equip orthopaedic researchers with the fundamental technical knowledge required to engage in interdisciplinary collaboration in AI-driven orthopaedic research. Level of evidence: Level IV.
Machine learning in medicine: a practical introduction to techniques for data pre-processing, hyperparameter tuning, and model comparison

André Pfob,Sheng-Chieh Lu,Chris Sidey-Gibbons

DOI: https://doi.org/10.1186/s12874-022-01758-8

2022-11-02

BMC Medical Research Methodology

Abstract:There is growing enthusiasm for the application of machine learning (ML) and artificial intelligence (AI) techniques to clinical research and practice. However, instructions on how to develop robust high-quality ML and AI in medicine are scarce. In this paper, we provide a practical example of techniques that facilitate the development of high-quality ML systems including data pre-processing, hyperparameter tuning, and model comparison using open-source software and data.

health care sciences & services
Artificial Intelligence in Surgery: Neural Networks and Deep Learning

Deepak Alapatt,Pietro Mascagni,Vinkle Srivastav,Nicolas Padoy

DOI: https://doi.org/10.48550/arXiv.2009.13411

2020-09-28

Abstract:Deep neural networks power most recent successes of artificial intelligence, spanning from self-driving cars to computer aided diagnosis in radiology and pathology. The high-stake data intensive process of surgery could highly benefit from such computational methods. However, surgeons and computer scientists should partner to develop and assess deep learning applications of value to patients and healthcare systems. This chapter and the accompanying hands-on material were designed for surgeons willing to understand the intuitions behind neural networks, become familiar with deep learning concepts and tasks, grasp what implementing a deep learning model in surgery means, and finally appreciate the specific challenges and limitations of deep neural networks in surgery. For the associated hands-on material, please see <a class="link-external link-https" href="https://github.com/CAMMA-public/ai4surgery" rel="external noopener nofollow">this https URL</a>.

Neural and Evolutionary Computing
Advancing Surgical Education: The Use of Artificial Intelligence in Surgical Training

David T. Guerrero,Malke Asaad,Aashish Rajesh,Abbas Hassan,Charles E. Butler

DOI: https://doi.org/10.1177/00031348221101503

2022-05-15

The American Surgeon

Abstract:The technology of artificial intelligence (AI) has made significant in-roads into the field of medicine over the last decade. With surgery being a discipline where repetition is the key to mastery, the scope of AI presents enormous potential for resident education through the analysis of technique and delivery of structured feedback for performance improvement. In an era marred by a raging pandemic that has decreased exposure and opportunity, AI offers an attractive solution towards improving operating room efficiency, safe patient care in the hands of supervised residents and can ultimately culminate in reduced health care costs. Through this article, we elucidate the current adoption of the artificial intelligence technology and its prospects for advancing surgical education.

surgery
Artificial Intelligence in Surgery: Promises and Perils

Daniel A Hashimoto,Guy Rosman,Daniela Rus,Ozanan R Meireles,Meireles, Ozanan R.,Rus, Daniela,Rosman, Guy,Hashimoto, Daniel A.

DOI: https://doi.org/10.1097/SLA.0000000000002693

IF: 13.787

2019-10-03

Annals of Surgery

Abstract:Objective: The aim of this review was to summarize major topics in artificial intelligence (AI), including their applications and limitations in surgery. This paper reviews the key capabilities of AI to help surgeons understand and critically evaluate new AI applications and to contribute to new developments. Summary Background Data: AI is composed of various subfields that each provide potential solutions to clinical problems. Each of the core subfields of AI reviewed in this piece has also been used in other industries such as the autonomous car, social networks, and deep learning computers. Methods: A review of AI papers across computer science, statistics, and medical sources was conducted to identify key concepts and techniques within AI that are driving innovation across industries, including surgery. Limitations and challenges of working with AI were also reviewed. Results: Four main subfields of AI were defined: (1) machine learning, (2) artificial neural networks, (3) natural language processing, and (4) computer vision. Their current and future applications to surgical practice were introduced, including big data analytics and clinical decision support systems. The implications of AI for surgeons and the role of surgeons in advancing the technology to optimize clinical effectiveness were discussed. Conclusions: Surgeons are well positioned to help integrate AI into modern practice. Surgeons should partner with data scientists to capture data across phases of care and to provide clinical context, for AI has the potential to revolutionize the way surgery is taught and practiced with the promise of a future optimized for the highest quality patient care.

surgery
O52: Empowering Medical Students and Early Career Surgeons in Critical Appraisal of AI Research: Design and Evaluation of a Pilot Course

Victoria Ngai,Arif Hanafi Bin Jalal,John Gerrard Hanrahan,Adrito Das,Danyal Z Khan,Elizabeth Cotton,Shazia Sharela,Martyna Stasiak,Hani J Marcus,Anand S Pandit

DOI: https://doi.org/10.1093/bjs/znae046.053

2024-03-01

Abstract:Abstract Introduction As artificial intelligence (AI) plays an increasing role in surgical practice, there is a need for surgeons to understand and evaluate literature surrounding the use of AI. There is currently a lack of training in this area for medical students and surgical trainees. We designed and assessed the impact of a short educational course. Methods A two-day online course was held, consisting of lectures and small-group discussions surrounding the fundamentals and appraisal of AI literature with a focus on neurosurgery. Participants included individuals working in neurosurgery (trainees, consultants and researchers) and medical students interested in surgery. A Qualtrics survey was distributed before and after the course. Participants’ knowledge was assessed before and after using 15 multiple-choice questions. Individuals allocated to the course’s waiting list completed a control survey. Results 62 participants (33 participants, 29 waitlist controls) attended the course and completed the pre-course survey. After the course, participants significantly improved in their knowledge of AI (mean difference = 3.86, 95% CI = 2.97-4.75, p-value < 0.0001) and confidence in critically appraising AI literature (p-value = 0.002). Differences in knowledge (mean difference = 3.15, 95% CI = 1.82-4.47, p-value < 0.0001) and confidence (p-value < 0.0001) were also found when compared to the control group. Significant differences were maintained when medical students or researchers were excluded from analysis. Conclusion Short courses that are tailored to the specialty and experience of surgical trainees and medical students can improve their understanding of AI, without the need for in-depth technical knowledge or programming skills.
A practical guide to the implementation of AI in orthopaedic research, Part 6: How to evaluate the performance of AI research?

Felix C. Oettl,Ayoosh Pareek,Philipp W. Winkler,Bálint Zsidai,James A. Pruneski,Eric Hamrin Senorski,Sebastian Kopf,Christophe Ley,Elmar Herbst,Jacob F. Oeding,Alberto Grassi,Michael T. Hirschmann,Volker Musahl,Kristian Samuelsson,Thomas Tischer,Robert Feldt,ESSKA Artificial Intelligence Working Group

DOI: https://doi.org/10.1002/jeo2.12039

2024-06-01

Journal of Experimental Orthopaedics

Abstract:Artificial intelligence's (AI) accelerating progress demands rigorous evaluation standards to ensure safe, effective integration into healthcare's high‐stakes decisions. As AI increasingly enables prediction, analysis and judgement capabilities relevant to medicine, proper evaluation and interpretation are indispensable. Erroneous AI could endanger patients; thus, developing, validating and deploying medical AI demands adhering to strict, transparent standards centred on safety, ethics and responsible oversight. Core considerations include assessing performance on diverse real‐world data, collaborating with domain experts, confirming model reliability and limitations, and advancing interpretability. Thoughtful selection of evaluation metrics suited to the clinical context along with testing on diverse data sets representing different populations improves generalisability. Partnering software engineers, data scientists and medical practitioners ground assessment in real needs. Journals must uphold reporting standards matching AI's societal impacts. With rigorous, holistic evaluation frameworks, AI can progress towards expanding healthcare access and quality. Level of Evidence Level V.

Practical Guide to Machine Learning and Artificial Intelligence in Surgical Education Research

The Application of Artificial Intelligence to Medical Education in the Last Decade - a Bibliometric Analysis (Preprint)

Practical Guide to Survey Research in Surgical Education

The fundamentals of artificial intelligence in medical education research: AMEE Guide No. 156

Practical Guide to Surgical Simulation Research

Practical Guide to Experimental and Quasi-Experimental Research in Surgical Education

A Surgeon's Guide to Artificial Intelligence-Driven Predictive Models

Practical Guide to Ethics in Surgical Education Research

A foundation for evaluating the surgical artificial intelligence literature

Practical Guide to Qualitative Research in Surgical Education

Machine Learning and Artificial Intelligence for Surgical Decision Making

Artificial Intelligence in Surgical Learning

Practical Guide to Common Flaws With Surgical Education Research

Surgical data science and artificial intelligence for surgical education

A practical guide to the implementation of artificial intelligence in orthopaedic research-Part 2: A technical introduction

Machine learning in medicine: a practical introduction to techniques for data pre-processing, hyperparameter tuning, and model comparison

Artificial Intelligence in Surgery: Neural Networks and Deep Learning

Advancing Surgical Education: The Use of Artificial Intelligence in Surgical Training

Artificial Intelligence in Surgery: Promises and Perils

O52: Empowering Medical Students and Early Career Surgeons in Critical Appraisal of AI Research: Design and Evaluation of a Pilot Course

A practical guide to the implementation of AI in orthopaedic research, Part 6: How to evaluate the performance of AI research?