Real-world Validation of the Chemotherapy-Induced Nausea and Vomiting Predictive Model and Its Optimization for Identifying High-Risk Chinese Patients
Linlin Zhang,Lili Zeng,Yue Sun,Jing Wang,Cong Wang,Chang Liu,Ming Ding,Manman Quan,Zhanyu Pan,Diansheng Zhong
DOI: https://doi.org/10.1097/cm9.0000000000002265
2023-01-01
Abstract:To the Editor: Chemotherapy-induced nausea and vomiting (CINV) is the most common and feared side effect reported by patients receiving chemotherapy. Despite significant progress, the incidence of CINV remains high.[1] Prevention is the key for controlling CINV. Evidence-based antiemetic guidelines recommend that strategies to prevent CINV should consider both the emetic risk of chemotherapeutic agents and patients’ individual risk factors.[2] To date, several personal factors predicting a high risk of CINV have been identified, including expectancy of CINV, female sex, younger age, history of morning sickness, and anxiety.[3] Efforts have been made to develop different CINV risk predictive models incorporating all of the aforementioned factors to guide individualized antiemetic prevention. Among these models, the most commonly recommended model is the large-scale investigation published in 2017 in Annals of Oncology based on a non-Asian population.[4] However, the applicability of this model to Chinese patients has not been investigated. Thus, we conducted a prospective study to validate the existing CINV predictive model and optimize it for identifying high-risk Chinese patients. This study was approved by the Tianjin Medical University General Hospital Ethics Committee (No. IRB2021-WZ-078), and patients provided informed consent for participation in the study. Patients were recruited from the Department of Medical Oncology of Tianjin Medical University General Hospital and Department of Integrated Traditional and Western Medicine of Tianjin Medical University Cancer Institute and Hospital in China. The included patients were divided into two groups. Group 1, patients from Tianjin Medical University General Hospital, was used to validate and optimize the existing predictive model. Group 2, patients from Tianjin Medical University Cancer Institute and Hospital, was used for the optimized model re-validation. The inclusion criteria of patients and data collection procedure in this study are presented in Appendix A, https://links.lww.com/CM9/B212. First, the association of CINV predictive factors with the occurrence of CINV was analyzed via multivariate generalized estimating equation (GEE) regression analysis in Chinese patients. Odds ratios and their 95% confidence intervals (CIs) were determined, and the results were significant at P <0.05. To validate the reliability and accuracy of the existing CINV predictive model in Chinese patients, each predictor should display its proper potential correlation with the occurrence of CINV compared with the existing predictive model. Second, to optimize the predictive model suitable for Chinese patients, we removed the risk factors demonstrating opposite predictive value as compared with the existing predictive model, so that a more reliable set of variables can be submitted for optimized model development. Then, the set of remaining predictors was re-analyzed in both univariate and multivariate GEE regression to develop the optimized model. Furthermore, the goodness-of-fit of the optimized predictive model was measured using the Hosmer-Lemeshow test. A calibration plot was drawn to evaluate the agreement between the predicted probabilities and observed outcomes of CINV. Additionally, the risk scoring system of the optimized model was re-built. The procedure for re-building the risk scoring system is presented in Appendix B, https://links.lww.com/CM9/B212. The predictive accuracy of the optimized model with the scoring system was measured using the specificity, sensitivity, positive predictive value (PPV), negative predictive value (NPV) and area under the receiver-operating characteristic (ROC) curve (AUC). The Youden index was calculated to select the best total score cutoff for identifying patients at high risk of CINV. Third, the optimized predictive model was confirmed by re-validation using the validation patient group. The predictive accuracy and goodness-of-fit were measured again in the re-validation set. In all, this study included 1356 cycles of chemotherapy across 568 Chinese patients. Demographic and clinical characteristics of patients and treatment-related variables were listed in Supplementary Tables 1 and 2, https://links.lww.com/CM9/B212. In our clinical validation in Chinese patients, we identified five factors displaying the same predictive value as compared with the existing CINV predictive model, and two of them (age <60 years and CINV in the prior cycle) had statistical significance (P = 0.032 and P <0.001) [Supplementary Table 3, https://links.lww.com/CM9/B212]. However, four factors displayed the opposite predictive value as compared with the existing CINV predictive model, including platinum- or anthracycline-based chemotherapy, cycle 2 and cycle ≥3 of chemotherapy, and <7 h of sleep the night before chemotherapy [Supplementary Table 3, https://links.lww.com/CM9/B212]. Considering that the predictive values of those factors were opposite between the existing model and our validation set, we speculated that the predictive reliability of the existing model might be impaired by these predictors in our patients. Thus, optimization of the predictive model should be performed. Establishment of the optimized model was performed by removing the factors displaying opposite predictive value first. Then, we finalized the optimized model by integrating all the remaining factors together. The remaining predictive factors were as follows: (1) age <60 years, (2) anticipatory CINV, (3) history of morning sickness, (4) use of antiemetics at home, and (5) CINV in the prior cycle. GEE regression revealed that all the remaining factors were positive predictors of CINV [Table 1]. Calibration plot for the probability of CINV revealed good agreement between the prediction and actual observation [Supplementary Figure 1A, https://links.lww.com/CM9/B212]. Table 1 - Optimized model and scoring system of CINV predictive model. Optimized model development set Optimized model validation set Predictive factors Risk score Odds ratio 95% CI P values Odds ratio 95% CI P values Age <60 years 1 1.228 0.888–1.698 0.215 1.496 0.917–2.441 0.212 Anticipatory nausea and vomiting 2 1.544 1.163–2.048 0.003 2.252 1.432–3.540 0.002 History of morning sickness 1 1.389 0.917–2.105 0.121 1.206 0.918–2.139 0.524 Use of non-prescribed antiemetics at home 1 1.359 0.895–2.063 0.150 1.849 1.197–2.858 0.032 Nausea or vomiting in the prior cycle 2 2.133 1.533–2.968 <0.001 2.421 1.324–4.429 0.010 CI: Confidence interval; CINV: Chemotherapy-induced nausea and vomiting. The risk scoring system was developed according to the regression coefficients of each predictor generated from multivariable GEE analysis. The total risk score was a summary of the transformed coefficient value of each predictor, and the product was a total score ranging from 0 to 7 where higher total scores indicated an increased risk of CINV [Table 1]. All patients in the sample were assigned a total risk score based on the algorithm. Next, ROC curve analysis was performed, and the AUC was acceptable at 0.629 (95% CI: 0.592–0.665). Based on the results of ROC curve analysis, a total score of 2 displayed the highest Youden index, demonstrating that patient with a total score ≥2 was associated with a high risk of CINV in their following cycle of chemotherapy. The sensitivity and specificity of this predictive model for identifying patients at high risk of CINV were 53.2% and 76.0%, respectively. The PPV and NPV of this model were 69.0% and 62.1%, respectively. Additionally, considering several predictive factors were of no statistical significance in the optimized model [Table 1], we performed another ROC curve analysis including only the factors significantly associated with CINV (anticipatory CINV and CINV in the prior cycle). However, we found that the AUC was only 0.610 (95% CI: 0.573–0.647), which was lower than that of the optimized model including all of the positive value predictors. Re-validation of the optimized model was performed using 201 patients who received a total of 452 cycles of chemotherapy. Multivariate GEE regression demonstrated that all factors were positive predictors of CINV. ROC curve analysis was performed, and the AUC was acceptable at 0.685 (95% CI: 0.631–0.738). The calibration plot for the probability of CINV also demonstrated a good agreement between the prediction by the optimized model and actual observation [Supplementary Figure 1B, https://links.lww.com/CM9/B212]. In general, this study performed real-world validation and optimization of a highly-recommended CINV predictive model in Chinese patients. Real-world validation is important because it can more accurately reflect the daily practice of clinicians and the CINV burden on patients. The validation results revealed that some predictive factors in the existing model did not display their proper potential correlations with the occurrence of CINV in Chinese patients. The uncertainty of the predictive value of these factors among different populations and in clinical practice was detected (reasons for the uncertainty are presented in Appendix C, https://links.lww.com/CM9/B212). To optimize the model to suit our clinical practice and patients, we removed these factors from the existing predictive model. However, reducing the number of predictors increased the weight of each remaining factor. For example, in the optimized model, if a patient develops CINV in the first cycle, he/she will at least get a total score of 2 before the second cycle of chemotherapy, which indicates he/she will be at a high risk of CINV at this cycle of chemotherapy so that adjustment of antiemetic prevention should be performed according to the change of only one predictor. Furthermore, it was important to realize that the optimized predictive model excluded the anti-cancer agents themselves. Thus, physicians should consider both the emetogenic potential of anti-cancer drugs and the patient's personal predictive risk factors to select a suitable antiemetic preventive treatment in clinical practice. Furthermore, it is notable that several predictors in this optimized model were not significantly associated with CINV. However, these risk factors demonstrated the same predictive value as the risk factors in the existing CINV predictive model, and their roles in CINV prediction have been substantially investigated and confirmed by previous studies.[3] In addition, the inclusion of these predictors was suggested by a superior prediction accuracy revealed by the AUC (0.629 vs. 0.610). Therefore, we ultimately included these predictors in the optimized model. The lack of statistically significant associations may have resulted from the limited sample size as well as the influence of other unknown variables on the occurrence of CINV; therefore, precise optimization of the CINV predictive model is still required in the future. To conclude, our study validated an existing CINV predictive model and optimized it for identifying Chinese patients at high risk of CINV. This optimized model represents an easy-to-use method for the personalized management of CINV. Additionally, it is a dynamic prediction tool for effective CINV control throughout all chemotherapy cycles. Despite several limitations, it will help professionals estimate individual risk of CINV development and make suitable decisions for CINV prophylaxis in clinical practice for Chinese patients. Funding This study was supported by a grant from the Tianjin Municipal Bureau of Public Health, China (No. 2013KZ121). Conflicts of interest None.