Abstract:Background: The efficiency of regimens for selection by clinicians in compliance with clinical guidelines is enhanced by digital clinical decision support. We are building a clinical decision support system (CSCO AI) for breast cancer patients to improve the efficiency of clinical decision-making. We aimed to assess cancer treatment regimens given by CSCO AI and different levels of clinicians.Methods: 400 breast cancer patients were screened from the CSCO database, including 100 cases of a new diagnosis, post-operation, before the first-line, and after first-line therapy respectively. These 400 cases were randomly divided into two volumes according to hormone receptor and HER2 status. Clinicians with similar levels were randomly assigned one of the volumes (200 cases). CSCO AI was asked to assess all cases. Two reviewers with more than 10 years of clinical experience were asked to evaluate the regimens from clinicians and CSCO AI independently. Regimens were masked before evaluation. After evaluation, a third reviewer with more than 30 years of clinical experience gave the final scores if there were different points between the prior two reviewers. The primary outcome was the proportion of high-level conformity. The second outcome was a concordance rate, which was defined as the percentage of the regimens with the same level of conformity (points) or the uniform decisions in radiotherapy and target therapy between clinicians and CSCO AI.Findings: A total of 14 clinicians from 4 hospitals with distinct professions and levels participated in this trial, 6 surgeons and 8 physicians. The overall concordance between clinicians and CSCO AI was 73.9% (3621/4900). It was 78.8%(2757/3500) in the early-stage, higher than that in the metastatic stage (61.7% [864/1400], p<0.001). The concordance was 90.7% (635/700) and 89.4% (626/700) in adjuvant radiotherapy and adjuvant target therapy respectively, while the proportion was only 56.4% (395/700) in second-line therapy. High-level conformity in CSCO AI was 95.8% (479/500, 95%CI:94.0%-97.6%), significantly higher than that in clinicians (90.8%, 3178/3500, 95%CI:89.8%-91.8%). Considering professions, the high-level conformity of surgeons was 85.9%, lower than that of CSCO AI (OR=0.25,95%CI: 0.16-0.41). The most noteworthy difference in high-level conformity between surgeons and CSCO AI was in first-line therapy (OR=0.06, 95%CI:0.01-0.41). When clinicians were divided according to their levels, the high-level conformity of clinicians was significantly lower than that of CSCO AI (p<0.05). However, there was no statistical significance between CSCO AI and level 2 or 3 clinicians. Scores from CSCO AI were significantly higher than that from clinicians in each treatment stage, especially in first-line therapy (Δ=0.24,p<0.001). Physicians got 2.75±0.61 scores in second-line therapy, significantly higher than that CSCO AI did (2.60±0.77, p=0.006).Interpretation: There are differences in decision-making when compared with the CSCO AI system and clinicians according to diverse professions and levels. Clinical decision support for breast cancer was superior for most process outcomes except for second-line therapy. Funding: This work was supported by the Capital Clinical Characteristic Application Research(Grant numberZ181100001718215) and the innovative projects of PLA generalhospital (Grant number CX19011)Declaration of Interest: All authors declare no competing interestsEthical Approval: This study was approved by the Henan tumour hospital ethic committee(2018019)

Concordance between treatment recommendations provided by IBM Watson for Oncology and a multidisciplinary tumor board for breast cancer in China

Artificial Intelligence in Decision-Making for Colorectal Cancer Treatment Strategy: an Observational Study of Implementing Watson for Oncology in a 250-Case Cohort

Concordance Evaluation of an Artificial Intelligence Technology with a Multidisciplinary Tumor Board in Gastric Cancer.

Watson for Oncology and breast cancer treatment recommendations: agreement with an expert multidisciplinary tumor board

Concordance of Treatment Recommendations for Metastatic Non-Small-Cell Lung Cancer Between Watson for Oncology System and Medical Team

Real world study for the concordance between IBM Watson for Oncology and clinical practice in advanced non‐small cell lung cancer patients at a lung cancer center in China

Concordance Between Watson for Oncology and a Multidisciplinary Clinical Decision-Making Team for Gastric Cancer and the Prognostic Implications: Retrospective Study

Artificial intelligence in urologic oncology: the actual clinical practice results of IBM Watson for Oncology in South Korea

Quality control system of Watson for oncology: Artificial intelligence for supporting clinical decisions in oncology.

A meta-analysis of Watson for Oncology in clinical application

Artificial intelligence and lung cancer treatment decision: agreement with recommendation of multidisciplinary tumor board

Can AI Technology Augment Tumor Board Treatment Decisions for Stage II Colon Cancer Care?

Deployment and Integration of a Cognitive Technology in China: Experiences and Lessons Learned.

Adequacy and Effectiveness of Watson For Oncology in the Treatment of Thyroid Carcinoma

Consistency of CSCO AI with Multidisciplinary Clinical Decision-Making Teams in Breast Cancer: A Retrospective Study

Precision Oncology and AI: the ACMA System for Personalized Treatment Matching Using Real-World Data.

Employing an Oncology Decision-Support System to Quantify Treatment Variation.

Impact of Decision-Support System and Guideline Treatment Concordance on Response Rate in Advanced Lung Cancer.

A Comparison Between CSCO AI and Clinicians in Breast Cancer: A Double-Blind, Controlled Phase III Trial

The Fidelity of Artificial Intelligence to Multidisciplinary Tumor Board Recommendations for Patients with Gastric Cancer: A Retrospective Study

A comparison between a clinical decision support system and clinicians with guidelines in breast cancer.