Jingsi Dong
Yingcai Geng1
Bingjie Li
Yonggang Zhang- 1Department of Lung Cancer Center, West China Hospital, Sichuan University, Chengdu, China
- 2Department of Otorhinolaryngology, Head and Neck Surgery, West China Hospital, Sichuan University, Chengdu, China
- 3Department of Periodical Press and National Clinical Research Center for Geriatrics, West China Hospital, Sichuan University, Chengdu, China
- 4Chinese Evidence-Based Medicine Center, West China Hospital, Sichuan University, Chengdu, China
Objective: Clinical trials are the most effective way to judge the merits of diagnosis and treatment strategies. The in-depth mining of clinical trial data enables us to grasp the application trend of artificial intelligence (AI) for cancer diagnosis. The aim of this study was to analyze the characteristics of registered trials on AI for cancer diagnosis.
Methods: Clinical trials on AI for cancer diagnosis registered on the ClinicalTrials.gov database were searched and downloaded. Statistical analysis was performed by using SPSS 20.0 software.
Results: A total of 97 registered trials were included. Of them, only 27 (27.8%) were interventional trials and 70 (72.1%) were observational trials. Fifteen (15.4%) trials had been completed. Fifty trials were in recruitment, and another 18 remained unrecruited. The number of cases included in the clinical trials tended to be large, 31 (32.0%) trials including samples ranging from 100 to 499 cases and 17 (17.5%) trials including samples ranging from 500 to 999 cases. Of the 27 interventional trials, only two trials reported trials' phase. Most (85.2%) interventional trials were for diagnosis, and a few (3.7%) were for the purpose of both the diagnosis and therapy of cancers. For the observational clinical trials, 46 (65.7%) were cohort studies, and 11 (15.7%) were case-only studies. Among the observational trials, 46 (65.7%) were prospective studies and 13 (18.6%) were retrospective studies. Among 97 trials, 37 (38.1%) involved colorectal cancer, 11 (11.3%) involved breast cancer, 43 (44.3%) were for imaging diagnosis, 33 (34.0%) were for endoscopic diagnosis, and 11 (11.3%) were for pathological diagnosis. For the interventional trials, 11 trials were parallel assignment (40.7%), and 14 were single group assignment (51.9%). Among the 27 interventional trials, 18 (66.7%) trials were performed without masking, 6 (22.2%) trials were performed with single masking, only 1 (3.7%) was performed with double masking, and 2 (7.4%) was performed with triple masking.
Conclusion: It appears that most registered trials on AI for cancer diagnosis are observational design, and more trials are needed in this field.
Introduction
With the development of new computing methods combined with the availability of training data, the application of powerful mathematical algorithms in the field of artificial intelligence (AI) has been promoted. It is hard to define AI precisely. It has been suggested that a machine is intelligent if its working behavior is indistinguishable from that of a human being (1). In modern concepts, AI refers to the ability of machines to communicate, reason, and operate independently at work in a manner similar to that of humans. AI programs have been developed and applied in many medical areas, including diagnosis, treatment, drug development, and patient cares; in addition, there are increased researches regarding AI in various specialties, especially in cancer diagnosis (2–5).
Because cancer is still the leading cause of death worldwide (6), accurate diagnosis of cancers is essentially important. In terms of the imaging diagnosis of tumors (7) [i.e., pathological diagnosis (8) and endoscopic diagnosis (9)], the performance of AI is as good as that of human experts. In the era of big data, medical activities are accelerating to produce a vast amount of health-disease data (10). With the help of AI, doctors can provide medical services to patients more efficiently and accurately (11, 12). At present, the most critical problem for AI application in imaging is that there is no gold standard of AI for cancer diagnosis (12); thus, many researchers performed trials to assess AI for cancer diagnosis, because trials are the most effective way to judge the merits of diagnosis and treatment strategies (13). Most trials were registered in ClinicalTrials.gov, which is a public clinical trial registry platform jointly launched by the US Food and Drug Administration (FDA) and the US National Library of Medicine (NLM). Studying characteristics of registered trials will help to know the development of trials in specific filed. Up to now, there is no such study on AI for cancer diagnosis; thus, we performed the current study. The aim of this study was to analyze and summarize the characteristics of AI for cancer diagnosis. The in-depth mining of clinical trial data from ClinicalTrials.gov enables us to grasp the application trend of AI in cancer diagnosis earlier.
Materials and Methods
Inclusion Criteria and Exclusion Criteria
The inclusion criteria were registered trials on AI for cancer diagnosis; in each trial, cancer can be a single cancer or all kinds of cancer. The exclusion criteria were AI in purely therapeutic applications and incomplete registration information.
Data Search
According to our previous studies (14, 15), we search the ClinicalTrials.gov on February 20, 2020, for trials on AI for cancer diagnosis. In case of missing any trials, we used the following words: artificial intelligence, deep learning, machine learning, etc. All searched results were downloaded. The data were updated on June 18, 2020.
Trial Screening and Data Extraction
Two authors independently screened the trials according to the inclusion and exclusion criteria. In case of any disagreement, discussion was performed. And then, two authors independently extracted the data of the included trials. The following information was extracted: registered number, study type, start date, status of the trial, study result, study sample, participant age, primary sponsor, location, primary purpose, phases of the trial, allocation, intervention model, masking and intervention, and types of cancer.
Data Analysis
The methodology of the study is similar to our previous study (14). This is a cross-sectional study, so a descriptive analysis was used to analyze the characteristics of registered trials. The outcomes included year, status of trials, study results, age, enrolment, sponsor, location, funding source, characteristics of study designs, type of cancer, and application method. Statistical analysis was performed by using SPSS 20.0 software. P-value <0.05 was considered statistically significant.
Results
The Characteristics of the Included Trials
On June 18, 2020, 884 results were searched from the ClinicalTrials.gov website. After a careful review of the clinical trial information, 787 results were excluded. Finally, a total of 97 trials were included in this study.
The characteristics of the included trials are shown in Table 1. Of the 97 trials, only 27 (27.8%) were interventional trials and 70 (72.1%) were observational trials. Fifteen (15.4%) trials were completed, the largest proportion of trials (50 trials) were in recruitment, and another 18 trials remained unrecruited. None of the trials had available results. Eighty-seven (89.7%) trials included subjects over the age of 18, and 10 trials (10.3%) included patients of all ages. The number of cases included in the trials tended to be large, with 31 (32.0%) trials including samples ranging from 100 to 499 cases and 17 (17.5%) trials including samples ranging from 500 to 999 cases. Universities were listed as the primary sponsor for 57 (58.8%) trials, hospitals for 30 (30.9%) trials, and industries for 10 (10.3%) trials. Of all the trials, 48 (49.5%) trials were performed in Asia, 15 in Europe, 33 in North America, and 1 in Australia.
Table 1. Characteristics of included trials.
Characteristics of the Study Design
The characteristics of interventional trials are displayed in Table 2. Of the 27 interventional trials, only two trials reported phase (1 in phase 1 and 1 in phase 3), and other trials did not report phases. Among all of the interventional trials, most (85.2%) of the interventional trials were for diagnosis, a few (3.7%) were for the purpose of both the diagnosis and therapy of cancers, 2 (7.4%) trials were for the primary purpose of screening, and 1 (3.7%) trial was for the primary purpose of device feasibility. Among all of the interventional trials, there were 10 randomized trials (37%) and 4 (14.8%) non-randomized trials, and 13 (48.1%) trials did not mention the allocation value. There were 11 parallel assignment (40.7%) and 14 single group assignment (51.9%). Among the 27 interventional trials, 18 (66.7%) trials were performed without masking, six (22.2%) trials were performed with single masking, only one (3.7%) was performed with double masking, and two (7.4%) was performed with triple masking.
Table 2. Study design elements of interventional trials.
Table 3 shows the characteristics of the observational trials (n = 70). For the observational trials, 46 (65.7%) were cohort studies, 11 (15.7%) were case-only studies, 1 (1.4%) was defined population, and 12 (17.1%) were other. Among the observational trials, 46 (65.7%) were prospective studies and 13 (18.6%) were retrospective studies.
Table 3. Study design elements of observational trials.
Overview of Clinical Trials for Diagnosis
Table 4 shows the overview of trials for diagnosis. All 97 trials were designed for a variety of cancers, 37 trials (38.1%) were for colorectal cancer, and 11 (11.3%) were for breast cancer. Of the 97 trials, 43 (44.3%) were for imaging diagnosis, 33 (34%) for endoscopic diagnosis, and 11 (11.3%) for pathological diagnosis. To verify whether colonoscopy would be much more effective with the assistance of an automatic quality control system (AQCS), a prospective interventional trial was performed (NCT03622281). The enrolled patients were randomly assigned into the AQCS group and the control group who received colonoscopy without AQCS. An increased adenoma detection rate was seen in the AQCS group, which indicated that AQCS could practically improve the accuracy of colonoscopy (16).
Table 4. Overview of clinical trials in diagnosis.
Another trial with published results aimed to confirm whether a designed chatbot was not inferior to physicians regarding the satisfaction of breast cancer patients with the information provided (NCT03556813). Two groups of randomly assigned patients asked 12 predefined questions that were previously answered by a chatbot or a medical committee and received the response from either a chatbot or a physician. The chatbot group had higher success rates (69 vs. 64%) than the physician group, which showed noninferiority (P < 0.001) (17).
Discussion
This study provided an assessment of the clinical trials registered on ClinicalTrials.gov about AI diagnosis in cancer. Most trials were observational, and only a few were interventional. Most trials were registered after 2017, indicating that the application of AI for cancer diagnosis was a new technology. More than half of the trials were in recruitment, and only one trial published available results. Most trials tended to be large sample size studies, and only a few studies had an expected sample size of <100. Most trials were sponsored by universities and hospitals. Notably, the vast majority (n = 48) of trials were conducted in Asia, with only 15 trials initiated in Europe and 33 trials initiated in North America. Most interventional trials used randomization, but most did not use blinding methods. Most observational trials were prospective design.
Ten trials were designed to diagnose lung cancer. In terms of the imaging diagnosis of lung cancer, it was reported that machine learning could predict the histological type of lung cancer through the imaging characteristics of PET/CT (18). In 2017, Song et al. (19) reported that CT imaging features could be used to predict the pathological type of micropapillary adenocarcinoma, but whether it could be recognized by AI had not been reported (20). In 2016–2017, Luo et al. (21) and Yu et al. (22) reported that the automatic analysis method of AI could perform pathological image analysis to predict the prognosis of patients with lung cancer. The CT diagnosis and pathological diagnosis of lung cancer are important prerequisites for the treatment of lung cancer (23–25). It was expected that AI could provide more functions for accurate diagnosis in the future.
In reviewing all included trials, the highest proportion trials were for colorectal cancer (37, 38.1%). This suggests that application of AI in diagnosis of colorectal cancer is a hot topic. Because the overall rate of missed polyps is as high as 22%, the associated colorectal cancer after colonoscopy is of concern (26). Computer-aided diagnosis (CAD) offers a promising solution for reducing the rate of missed diagnoses with colonoscopy (27). AI technology must address a number of important issues before it can be incorporated into routine clinical practice. The key stages for the implementation of CAD technology in routine colonoscopy have been detailed elsewhere, particularly by Mori et al. (28), who described the following steps: product development and feasibility studies, clinical trials, regulatory approval, and insurance reimbursement (29).
Eleven (11.3%) trials were for the pathological diagnosis of cancer. In 2019, Chen et al. (30) reported that thanks to ARM technology, AI can be integrated into the microscopic workflow to improve the efficiency and consistency of the microscopic inspection of biological specimens. The technology would be used to diagnose cancers. Among all the studies of AI in the field of cancer pathological diagnosis, the implementation in breast cancer was earlier and more mature (8), and AI diagnosis had an excellent application in the diagnosis of primary breast cancer and metastatic breast cancer (31). In terms of cancer imaging diagnosis, AI could also improve the specificity of diagnosis by integrating patient information and image analysis (32). In this study, we found that the diagnostic modes of lung cancer were mainly imaging diagnosis and pathological diagnosis. The diagnostic mode of colorectal cancer was mainly endoscopic diagnosis. The diagnostic modes of breast cancer were mainly imaging diagnosis and pathological diagnosis. In addition, both published studies used the randomization method, suggesting that the comparison of AI and traditional methods in tumor diagnosis was more operable in trials.
With the aid of AI, the detection rate of polyps and adenomas under endoscopy will be greatly improved (33). This apparent advantage, however, remains to be demonstrated in multicenter studies. The deficiency of AI in cancer diagnosis is also obvious. Chatbots designed to aid diagnosis could communicate with breast cancer patients like doctors, showing the potential to help doctors. But chatbot's questions are too routine to fully help doctors' diagnoses and decisions. The main purpose of this study is to understand the current situation of the application of AI in the field of medicine, which has a good hint to the scholars in related fields. Therefore, our study did not focus on the specific results of each trial.
The study had several limitations. First, not all studies were registered on ClinicalTrials.gov, which limits the application of the results in the future. However, ClinicalTrials.gov contained more than 80% of the trials on the World Health Organization's International Clinical Trials Registry Platform (34). Therefore, even if our study cannot cover all trials, it still reflected the mainstream of such trials. Second, the study analyzed data from clinical trials in which AI was used to diagnose cancer, but due to the short time span of the clinical trials, most clinical trials have not published results, so the analysis of results was insufficient. Third, this study included the application of AI in the diagnosis of all cancer types, but as only 97 trials were included, specific cancer types were not targeted for detailed analysis.
In conclusion, the current study presents the characteristics of registered trials on AI for cancer diagnosis. It suggested that more trials are needed to provide more evidence.
Data Availability Statement
All datasets presented in this study are included in the article.
Author Contributions
YZ designed the study. JD and YG provided the source for the study and edited the manuscript. JD, LT, and DLi searched, extracted, assessed, and analyzed the data and drafted the manuscript. JD, BL, and DLu analyzed the data. All authors contributed to the article and approved the submitted version.
Funding
This work was partly supported by the Health Commission of Sichuan Province (18PJ432).
Conflict of Interest
The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
References
1. Turing AM. Computing machinery and intelligence. Mind LIX. (1950) 433–60. doi: 10.1093/mind/LIX.236.433
2. Adamson AS, Welch HG. Machine learning and the cancer-diagnosis problem - no gold standard. N Engl J Med. (2019) 381:2285–7. doi: 10.1056/NEJMp1907407
3. Bi WL, Hosny A, Schabath MB, Giger ML, Birkbak NJ, Mehrtash A, et al. Artificial intelligence in cancer imaging: clinical challenges and applications. CA Cancer J Clin. (2019) 69:127–57. doi: 10.3322/caac.21552
4. Mori Y, Berzin TM, Kudo SE. Artificial intelligence for early gastric cancer: early promise and the path ahead. Gastrointest Endosc. (2019) 89:816–7. doi: 10.1016/j.gie.2018.12.019
5. Strom P, Kartasalo K, Olsson H, Solorzano L, Delahunt B, Berney DM, et al. Artificial intelligence for diagnosis and grading of prostate cancer in biopsies: a population-based, diagnostic study. Lancet Oncol. (2020) 21:222–32. doi: 10.1016/S1470-2045(19)30738-7
6. Torre LA, Bray F, Siegel RL, Ferlay J, Lortet-Tieulent J, Jemal A. Global cancer statistics, 2012. CA Cancer J Clin. (2015) 65:87–108. doi: 10.3322/caac.21262
7. Mak RH, Endres MG, Paik JH, Sergeev RA, Aerts H, Williams CL, et al. Use of crowd innovation to develop an artificial intelligence-based solution for radiation therapy targeting. JAMA Oncol. (2019) 5:654–61. doi: 10.1001/jamaoncol.2019.0159
8. Ehteshami Bejnordi B, Veta M, Johannes Van Diest P, Van Ginneken B, Karssemeijer N, Litjens G, et al. Diagnostic assessment of deep learning algorithms for detection of lymph node metastases in women with breast cancer. JAMA. (2017) 318:2199–210. doi: 10.1001/jama.2017.14585
9. Rees CJ, Koo S. Artificial intelligence - upping the game in gastrointestinal endoscopy? Nat Rev Gastroenterol Hepatol. (2019) 16:584–5. doi: 10.1038/s41575-019-0178-y
10. Zeng T, Huang T, Lu C. Editorial: machine learning advanced dynamic omics data analysis for precision medicine. Front Genet. (2019) 10:1343. doi: 10.3389/fgene.2019.01343
11. Obermeyer Z, Lee TH. Lost in thought - the limits of the human mind and the future of medicine. N Engl J Med. (2017) 377:1209–11. doi: 10.1056/NEJMp1705348
12. Rajkomar A, Dean J, Kohane I. Machine learning in medicine. N Engl J Med. (2019) 380:1347–58. doi: 10.1056/NEJMra1814259
13. Feizabadi M, Fahimnia F, Mosavi Jarrahi A, Naghshineh N, Tofighi S. Iranian clinical trials: an analysis of registered trials in International Clinical Trial Registry Platform. (ICTRP). J Evid Based Med. (2017) 10:91–6. doi: 10.1111/jebm.12248
14. Chen L, Su Y, Quan L, Zhang Y, Du L. Clinical trials focusing on drug control and prevention of ventilator-associated pneumonia: a comprehensive analysis of trials registered on ClinicalTrials.gov. Front Pharmacol. (2018) 9:1574. doi: 10.3389/fphar.2018.01574
15. Chen L., Wang M., Yang Y., Shen J., Zhang Y. (2020). Registered interventional clinical trials for old populations with infectious diseases on clinicaltrials.gov: a cross-sectional study. Front. Pharmacol. 11:942. doi: 10.3389/fphar.2020.00942
16. Su JR, Li Z, Shao XJ, Ji CR, Ji R, Zhou RC, et al. Impact of a real-time automatic quality control system on colorectal polyp and adenoma detection: a prospective randomized controlled study (with videos). Gastrointest Endosc. (2020) 91:415–24 e414. doi: 10.1016/j.gie.2019.08.026
17. Bibault JE, Chaix B, Guillemasse A, Cousin S, Escande A, Perrin M, et al. A chatbot versus physicians to provide information for patients with breast cancer: blind, randomized controlled noninferiority trial. J Med Internet Res. (2019) 21:e15787. doi: 10.2196/15787
18. Hyun SH, Ahn MS, Koh YW, Lee SJ. A machine-learning approach using PET-based radiomics to predict the histological subtypes of lung cancer. Clin Nucl Med. (2019) 44:956–60. doi: 10.1097/RLU.0000000000002810
19. Song SH, Park H, Lee G, Lee HY, Sohn I, Kim HS, et al. Imaging phenotyping using radiomics to predict micropapillary pattern within lung adenocarcinoma. J Thorac Oncol. (2017) 12:624–32. doi: 10.1016/j.jtho.2016.11.2230
20. Jacobs C, Van Ginneken B. Google's lung cancer AI: a promising tool that needs further validation. Nat Rev Clin Oncol. (2019) 16:532–3. doi: 10.1038/s41571-019-0248-7
21. Luo X, Zang X, Yang L, Huang J, Liang F, Rodriguez-Canales J, et al. Comprehensive computational pathological image analysis predicts lung cancer prognosis. J Thorac Oncol. (2017) 12:501–9. doi: 10.1016/j.jtho.2016.10.017
22. Yu KH, Zhang C, Berry GJ, Altman RB, Re C, Rubin DL, et al. Predicting non-small cell lung cancer prognosis by fully automated microscopic pathology image features. Nat Commun. (2016) 7:12474. doi: 10.1038/ncomms12474
23. Gu J, Lu C, Guo J, Chen L, Chu Y, Ji Y, et al. Prognostic significance of the IASLC/ATS/ERS classification in Chinese patients-A single institution retrospective study of 292 lung adenocarcinoma. J Surg Oncol. (2013) 107:474–80. doi: 10.1002/jso.23259
24. Hung JJ, Yeh YC, Jeng WJ, Wu KJ, Huang BS, Wu YC, et al. Predictive value of the international association for the study of lung cancer/American Thoracic Society/European Respiratory Society classification of lung adenocarcinoma in tumor recurrence and patient survival. J Clin Oncol. (2014) 32:2357–64. doi: 10.1200/JCO.2013.50.1049
25. Tsao M-S, Marguet S, Teuff GL, Lantuejoul S, Shepherd FA, Seymour L, et al. Subtype classification of lung adenocarcinoma predicts benefit from adjuvant chemotherapy in patients undergoing complete resection. J Clin Oncol. (2015) 33:3439–46. doi: 10.1200/JCO.2014.58.8335
26. Van Rijn JC, Reitsma JB, Stoker J, Bossuyt PM, Van Deventer SJ, Dekker E. Polyp miss rate determined by tandem colonoscopy: a systematic review. Am J Gastroenterol. (2006) 101:343–50. doi: 10.1111/j.1572-0241.2006.00390.x
27. Ahmad OF, Soares AS, Mazomenos E, Brandao P, Vega R, Seward E, et al. Artificial intelligence and computer-aided diagnosis in colonoscopy: current evidence and future directions. Lancet Gastroenterol Hepatol. (2019) 4:71–80. doi: 10.1016/S2468-1253(18)30282-6
28. Mori Y, Kudo S-E, Berzin TM, Misawa M, Takeda K. Computer-aided diagnosis for colonoscopy. Endoscopy. (2017) 49:813–9. doi: 10.1055/s-0043-109430
29. Byrne MF, Shahidi N, Rex DK. Will computer-aided detection and diagnosis revolutionize colonoscopy? Gastroenterology. (2017) 153:1460–4.e1461. doi: 10.1053/j.gastro.2017.10.026
30. Chen P-HC, Gadepalli K, Macdonald R, Liu Y, Kadowaki S, Nagpal K, et al. An augmented reality microscope with real-time artificial intelligence integration for cancer diagnosis. Nat Med. (2019) 25:1453–7. doi: 10.1038/s41591-019-0539-7
31. Liu Y, Kohlberger T, Norouzi M, Dahl GE, Smith JL, Mohtashamian A, et al. Artificial intelligence-based breast cancer nodal metastasis detection: insights into the black box for pathologists. Arch Pathol Lab Med. (2019) 143:859–68. doi: 10.5858/arpa.2018-0147-OA
32. Dalmis MU, Gubern-Merida A, Vreemann S, Bult P, Karssemeijer N, Mann R, et al. Artificial intelligence-based classification of breast lesions imaged with a multiparametric breast MRI protocol with ultrafast DCE-MRI, T2, and DWI. Invest Radiol. (2019) 54:325–32. doi: 10.1097/RLI.0000000000000544
33. El Hajjar A., and Rey J. F. (2020). Artificial intelligence in gastrointestinal endoscopy: general overview. Chin Med J (Engl). 133, 326–34. doi: 10.1097/CM9.0000000000000623
Keywords: clinical trial, diagnosis, artificial intelligence, ClinicalTrials.gov, cancer
Citation: Dong J, Geng Y, Lu D, Li B, Tian L, Lin D and Zhang Y (2020) Clinical Trials for Artificial Intelligence in Cancer Diagnosis: A Cross-Sectional Study of Registered Trials in ClinicalTrials.gov. Front. Oncol. 10:1629. doi: 10.3389/fonc.2020.01629
Received: 19 March 2020; Accepted: 27 July 2020;
Published: 15 September 2020.
Edited by:
Chunxiao Guo, University of Minnesota, United StatesReviewed by:
Song Xu, Tianjin Medical University General Hospital, ChinaYifeng Zhou, Soochow University, China
Xiaoxing Li, Sun Yat-sen University, China
Louis Joseph Vaickus, Dartmouth College, United States
David Valle-Cruz, Universidad Autónoma del Estado de México, Mexico
Copyright © 2020 Dong, Geng, Lu, Li, Tian, Lin and Zhang. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
*Correspondence: Yonggang Zhang, jebm_zhang@yahoo.com