Artificial Intelligence in Cancer Clinical Research: II. Development and Validation of Clinical Prediction Models

Gary H. Lyman,Nicole M. Kuderer

DOI: https://doi.org/10.1080/07357907.2024.2354991

2024-05-23

Cancer Investigation

Abstract:In the first segment of this series on Artificial Intelligence in Cancer Clinical Research, we approached the subject first by developing a better understanding of human intelligence ( Citation 1 ). The goals of AI often revolve around creating algorithms and machines capable of outperforming or amplifying the capabilities of humans to perform complex tasks more rapidly and beyond our own capabilities both in terms of speed and complexity. While unprecedented capabilities of AI in addressing logical algorithmic or computable tasks have been observed, it has long been clear that there are activities of the human mind that are particularly challenging to replicate or even define. The other primary goal of AI is thus to emulate or, at least, better understand the inner workings and unique capabilities of the human brain. Clearly, many of these capabilities appear to reside outside of logical and computational aspects of the human brain and enable unique aspects of human intelligence such as insight, intuition, discovery, imagination, discrimination, and recognition arguably representing a deeper understanding of the meaning of our world and our actions.

oncology

What problem does this paper attempt to address?

The paper mainly discusses the problems and challenges faced in the development and validation of clinical prediction models for cancer research using artificial intelligence (AI). The authors point out that although AI performs well in certain aspects, the opacity of its underlying workings, especially in neural network applications, can lead to an inability to fully understand and interpret the results, which is a significant issue in the establishment and validation of prediction models.

The article emphasizes the important role of prediction models in understanding disease progression, risk stratification, treatment selection, etc. However, AI models have limitations in terms of accuracy, reliability, and effectiveness, which may result in incorrect predictions and require clinical supervision. Studies have shown that many AI-driven prediction models have a high risk of bias, including insufficient sample size, improper internal validation methods, inadequate handling of missing data, and inadequate overfitting evaluation.

Furthermore, the paper mentions that prediction models developed using machine learning (ML) have not shown significant performance advantages compared to traditional logistic regression-based models. There are also shortcomings in terms of reporting standards and quality control, such as low compliance with TRIPOD statements, insufficient external validation, and the phenomenon of overinterpretation and exaggeration of research results.

In summary, the problem that this paper attempts to address is how to establish and validate reliable AI clinical prediction models in cancer research while identifying and overcoming biases, limitations, and methodological challenges associated with model development. to ensure the accurate, reliable, and effective application of these models in clinical practice. The authors call for researchers to return to basics, focusing on data quality, model validation, and transparent reporting to improve the credibility of AI models in medicine, particularly in the field of oncology.