Attitudes Toward Artificial Intelligence Within Dermatopathology: An International Online Survey

Background: Artificial intelligence (AI) has recently surfaced as a research topic in dermatology and dermatopathology. In a recent survey, dermatologists were overall positive toward a development with an increased use of AI, but little is known about the corresponding attitudes among pathologists working with dermatopathology. The objective of this investigation was to make an inventory of these attitudes.

Participants and Methods: An anonymous and voluntary online survey was prepared and distributed to pathologists who regularly analyzed dermatopathology slides/images. The survey consisted of 39 question divided in five sections; (1) AI as a topic in pathology; (2) previous exposure to AI as a topic in general; (3) applications for AI in dermatopathology; (4) feelings and attitudes toward AI and (5) self-reported tech-savviness and demographics. The survey opened on March 13, 2020 and closed on May 5, 2020.

Results: Overall, 718 responders (64.1% females) representing 91 countries were analyzed. While 81.5% of responders were aware of AI as an emerging topic in pathology, only 18.8% had either good or excellent knowledge about AI. In terms of diagnosis classification, 42.6% saw strong or very strong potential for automated suggestion of skin tumor diagnoses. The corresponding figure for inflammatory skin diseases was 23.0% (P_adj < 0.0001). For specific applications, the highest potential was considered for automated detection of mitosis (79.2%), automated suggestion of tumor margins (62.1%) and immunostaining evaluation (62.7%). The potential for automated suggestion of immunostaining (37.6%) and genetic panels (48.3%) were lower. Age did not impact the overall attitudes toward AI. Only 6.0% of the responders agreed or strongly agreed that the human pathologist will be replaced by AI in the foreseeable future. For the entire group, 72.3% agreed or strongly agreed that AI will improve dermatopathology and 84.1% thought that AI should be a part of medical training.

Conclusions: Pathologists are generally optimistic about the impact and potential benefit of AI in dermatopathology. The highest potential is expected for narrow specified tasks rather than a global automated suggestion of diagnoses. There is a strong need for education about AI and its use within dermatopathology.

Introduction

Even though artificial intelligence (AI) has been discussed for decades, the unprecedented computer processor development, storage availability, and universal internet access including cloud services have contributed to the recent accelerated research interest in AI in several domains of medicine including pathology (1). AI and machine learning (ML) have recently pervaded every aspect of medical image analysis at an extraordinary pace (2).

While historically restricted to traditional microscopes and glass slides, pathology has recently transitioned to become a more digitally oriented specialty. Within the past decade, the use of whole slide imaging (WSI) has been validated for surgical pathology and dermatopathology (3, 4). Although WSI has not uniformly been well-received among dermatopathologists (5), the diagnostic modality has proven non-inferior to traditional microscopes for most diagnostic classes, except for a subset of melanocytic lesions, in a recent intra-observer validation investigation (6). With the rise of whole slide scanner technology (7), and free-hand smartphone photography (8), large numbers of tissue slides are being captured and archived digitally. For dermatopathology specifically, closed online forums now represent an important piece of continuing medical education and attract thousands of users from multiple countries across the world in all age groups (9). In many centers, all glass slides are routinely scanned and are made available digitally. While the primary diagnosis might still be made by pathologists using traditional microscopes, ancillary diagnoses by ML-based software might only be a few computer maneuvers away, fundamentally changing pathology as we know it.

The immense accumulation of online dermatopathology images represents a substantial resource not only for education, but also for ML research purposes. Recently, Schaumberg et al. presented the first pan-tissue and pan-disease ML-method for predicting disease as well as immunostaining (10). The training set consisted of thousands of images obtained through social media and PubMed articles and was prospectively tested in public on social media. Although in its early stages, the potential seen for ML in dermatopathology have been demonstrated both pertaining to specific tasks as well as diagnosis (11–13). The use of ML within dermatopathology has recently been listed as one of the important dermatological applications of AI (14). Nonetheless, little is known about dermatopathologists' attitudes toward an increased use of AI within their field. In a recent study, including 1,271 dermatologists, investigating the attitudes toward AI within dermatology, the majority of responders were positive toward a development with an increased use of AI. Moreover, only a minority expressed fear toward an increased use. Interestingly, a majority of responders (58.1%) saw either strong or very strong potential for automated detection of skin diseases based on dermatopathology images (15).

Dermatopathology and dermatology depend on one another and collaboration between our disciplines is essential for clinical-pathological correlation. In order to prepare for a future with an increased use of AI, it is our belief that a first inventory of dermatopathologists' views on the topic is important. Ultimately, it is our hope that dermatopathologists together with dermatologists should be the ones who guide and lead the development in our fields. Thus, the following study was performed as an exploratory survey with the aim to specifically address feelings and attitudes toward AI among dermatopathologists.

Materials and Methods

An anonymized and voluntary English online survey was prepared and distributed through SurveyMonkey® (www.surveymonkey.com, San Mateo, CA, USA). The survey started with a brief introduction followed by 39 questions (Q1–Q39). The first baseline question (Q1) tested for eligibility (i.e., if the responders worked as a pathologist who regularly analyzed dermatopathology slides/images). If a responder answered “no” to this first question, the survey ended. The rest of the survey questions were divided into five sections: (1) AI as a topic in pathology (Q2-Q3); (2) previous exposure to AI as a topic in general (Q4-Q7); (3) applications for AI in dermatopathology (Q8-Q14); (4) feelings and attitudes toward AI (Q15-Q25) and (5) self-reported tech-savviness and demographics (Q26-Q39). Most of the response options were prepared as Likert-type scales (16). The complete survey as displayed to the responder is available in the Supplementary Material. Answers to all questions were required for the survey to be considered complete.

The survey opened on March 13, 2020 and closed on May 5, 2020 (53 survey days). The survey link was advertised on a closed Facebook® (Menlo Park, CA, USA) group called McKee Derm, which is frequently visited by international pathologists, dermatopathologists and dermatologists and has ~15,000 members (17). The group opened in May 2017 and its main purpose is to share and discuss dermatopathology cases. The survey link was also shared publicly by one of the authors (JMG) on their public Facebook page and Twitter account. Since all responses were obtained voluntarily and anonymously from professional colleagues, the study was exempt from approval by an ethics review board.

Statistical Analysis

All data were analyzed using R version 3.5.3 (https://www.r-project.org/). Fisher's exact test and Wilcoxon's rank sum test were used for two-sample tests. Wilcoxon's signed rank test was used for paired tests. For questions with dichotomous answers, logistic regression was used with “no”/“yes” as the dependent variable and sex and age group as independent variables. For questions with non-dichotomous answers, linear regression models were used to correlate answers to sex and age group using a score for the answers, for example: “Strongly disagree” = 1, “Disagree” = 2, “Neither agree nor disagree” = 3, “Agree” = 4 and “Strongly agree” = 5. The age groups (20-24, 25-34, 35-44, 45-54, 55-64, 65-74, and ≥75 years) were used as numeric values in the regression models, i.e., numbers ranging from 1 to 7. P-values were adjusted (P_adj) for multiple comparisons using the Holm method (18). All tests were two-sided and P_adj < 0.05 was considered as statistically significant.

Results

Of 1,263 surveys received, 848 responders worked as pathologists who regularly analyzed dermatopathology slides/images in a clinical setting. Among these, 718 completed the survey resulting in an 84.7% response rate among the eligible audience and these responses were further analyzed. The median time to complete the survey was 6.1 min (interquartile range, 4.7–8.5 min). All aggregated result summaries for Q1-Q38 are available in Supplementary Material. An overview of the age and sex distribution of the responders as well as other demographic data are presented in Figure 1. Practicing specialists and residents in pathology constituted 80.5% of the responders. The median age (range) was 38 years (22–79 years), 64.1% (n = 460) were females and 39.0% (n = 280) had access to WSI at work. The great majority of the surveyed dermatopathologists mainly used traditional microscopes for routine diagnostic dermatopathology while only 4% (29 of 718) of the responders mainly used digitally scanned slides. Overall, 91 different countries were represented. The three countries where most responders worked were India (n = 96, 13.4%), USA (n = 80, 11.1%) and United Kingdom (n = 35, 4.9%).

FIGURE 1

Figure 1. Responder characteristics. (A) Age distribution for each sex. (B) Proportion in each type of practice setting. (C) Distribution of the number of years working within pathology. (D) Current position. (E) Access to whole slide imaging at work.

While 81.5% of responders were aware of AI as an emerging topic in pathology, only 18.8% had either good or excellent knowledge about AI within pathology. Younger responders (<40 years) were equally aware of the recent development compared to older responders (≥40 years) with 322 of 400 (80.5%) vs. 263 of 318 (82.7%) claiming awareness (P = 0.50, P_adj = 1). A higher percentage of responders mainly working in a university hospital (255 of 293, 87.0%) were aware of the AI development within pathology compared to those working in other settings (330 of 425, 77.6%) (P = 0.0017, P_adj = 0.073). Males generally had a higher self-reported level of knowledge about AI than females (26.0 vs. 14.8%, P = 0.003, P_adj = 0.12).

Among all responders, 22.3% had used AI as a diagnostic aid in real life within pathology and 11.0% had used AI within dermatopathology in particular. Use of AI was not associated with age nor sex (Table 1). Physicians with access to WSI and those with previous use of AI within pathology and dermatopathology were more knowledgeable about the technique (Supplementary Table 1).

TABLE 1

Table 1. Distribution of answers to questions regarding background knowledge about AI.

Responders with ≤ 10 years of experience within pathology (339 of 423, 80.1%) were equally aware of the AI development as responders with >10 years of experience (237 of 285, 83.2%) (P = 0.33, P_adj = 1). In the tech-savvy group, 99 of 376 responders (26.3%) reported either good or excellent knowledge about AI within pathology compared to 36 of 342 non-tech-savvy responders (10.5%) (P < 0.0001, P_adj < 0.0001).

When responders were asked about previous exposure to AI as a topic in general (Q4-Q7), 71.0, 74.9, 53.5, and 65.5% had heard about AI from the media, social media, lectures and friends, respectively. Social media and friends were more common sources for younger age groups. Other media was a more common source among men compared to females (Table 2).

TABLE 2

Table 2. Distribution of answers to questions regarding sources about AI applications.

The aggregated results from Q8-Q14 about seven applications for AI within dermatopathology are summarized in Figure 2. In terms of classification of diagnoses, 42.6% (n = 306) saw strong or very strong potential for automated suggestion of skin tumor diagnoses. The corresponding figure for inflammatory skin diseases was 23.0% (n = 165) (P < 0.0001, P_adj < 0.0001). For specific applications, the highest potential was considered for automated detection of mitosis (79.2%), automated suggestion of tumor margins (62.1%) and immunostaining evaluation (62.7%). The potential for automated suggestion of immunostaining (37.6%) and genetic panels (48.3%) were lower.

FIGURE 2

Figure 2. Potential seen for AI within dermatopathology. Regardless of whether you have thought about this before, which potential do you personally see for AI for dermatopathology images regarding each of the following. (A) Potential seen for automated suggestion of diagnoses of cutaneous tumors and inflammatory skin diseases. (B) Potential seen for specific tasks including; automated detection of mitoses; automated suggestion of tumor margins; automated evaluation of immunostaining results; automated suggestion of which immunostaining panels to order; automated suggestion of which complementary genetic panels to order.

Results for feelings and attitudes toward AI (Q15-Q25) are presented in Table 3. Age did not impact the overall attitudes toward AI. Males displayed more excitement and less fear about the use of AI within dermatopathology as well as within medicine in general. The level of fear toward a development with an increased use of AI within dermatopathology did not differ between dermatopathologists working mainly at a university hospital compared to those who mainly worked in another setting (16.7 vs. 15.8%, P = 0.76, P_adj = 1). Among 280 responders with known access to WSI, 14.6% (n = 41) expressed fear toward a development with an increase use of AI within dermatopathology, which did not differ significantly to expressed fears among 74 of 421 responders without such access (17.6%) (P = 0.35, P_adj = 1). Furthermore, tech-savviness did not influence expressed levels of fear (data not shown). The attitudes toward AI among those with and without access to WSI did not differ, whereas those with previous use of the technique in dermatopathology had generally more positive attitudes (Supplementary Table 2).

TABLE 3

Table 3. Distribution of answers to questions regarding attitudes and feelings about AI.

Only 6.0% (43 of 718) of the responders agreed or strongly agreed that the human pathologist will be replaced by AI in the foreseeable future. Among tech-savvy responders, the corresponding figure was 7.7% (29 of 376, P = 0.086, P_adj = 1). For the entire group, 72.3% agreed or strongly agreed that AI will improve dermatopathology and 84.1% thought that AI should be a part of medical training.

Discussion

Our results indicate a generally positive attitude toward AI within dermatopathology among surveyed pathologists. The majority of responders have not yet used AI tools as a diagnostic aid within dermatopathology, underlining that such tools are still under development and a broad implementation to a large extent still is pending.

Age did not influence the results and, while males generally expressed more excitement and less fear compared to female responders, the differences were small in absolute terms. Nonetheless, some of the differences between the genders, might be due to systematic gaps in confidence levels between men and women (19, 20). The majority of the responders were not afraid of being replaced by AI and felt that AI will improve dermatopathology making it more exciting. Only 4% of the responders mainly worked with digital pathology which could explain the low usage of AI tools. Interestingly, only 25% of the responders had read a medical publication regarding AI within dermatopathology which may reflect that studies in this field are still rare.

Narrow tasks such as automated detection of mitoses and tumor margins were seen to have higher potential compared to automated diagnosis of both cutaneous tumors and inflammatory skin disorders. This is not surprising since several studies have shown AI's potential in detecting mitoses while evidence in diagnostic use is still rare (21, 22). In regards to AI-driven automated diagnostics, its usefulness for tumors was considered more likely than for dermatoses. In fact, a few studies have shown the potential of AI in differentiating skin tumors in WSI (12, 23). AI has even outperformed pathologists in the classification of melanoma (24). Melanocytic lesions in specific have proven particularly difficult for dermatopathologists, where significant levels of inter- and intra-observer variability have been reported (25). ML-based solutions likely hold strong promise in this domain.

The responders reported a high potential for automated suggestion of tumor margins. Such benefits were observed in a recent study in which fluorescence lifetime imaging and ML were used to visualize tumor margins in excised breast specimens showing high specificity and sensitivity (26). In a previous survey about general pathologists' attitudes toward AI, 75% of the pathologists showed interest or excitement about AI as a diagnostic tool and the vast majority felt it could increase diagnostic efficiency (27). This is in line with the findings in our survey.

As the fields of dermatology and dermatopathology are getting ready for a tremendous transition, we believe inventories of attitudes toward AI are important since it will clarify the potential in specific domains and hopefully unify the community to try to work together. Future ML tools will probably make use of several sources of information simultaneously and parallel systems might provide an even more accurate output. Recently, online dermatopathology forums have collected large amounts of digital cases. This online educational resource could potentially be used to train convolutional neural networks. Ideally, these tools should be available through a web-browser and should be freely available to all users. We believe that pathologists must engage in this development in order to take a leading position in AI. The positive attitudes presented here could certainly help promote this development. Nonetheless, this survey also illustrates that the majority (61.0%) of responders don't yet have access to WSI. A broad implementation of WSI is most likely a prerequisite for an increased use of AI in dermatopathology.

When interpreting our results, it is imperative to remember that the majority of responders received the survey invitation via their social media interest in dermatopathology. The possibility of a selection bias exists and physicians with positive attitudes may have been more likely to have completed the survey. Moreover, AI is a new technique and different responders might have varying ideas of what it includes and represents, particularly since the processes between input and output by the algorithms are not usually clearly defined, and the methods and variable combinations may differ from our own decision-making.

Setting up an online link rather than solely inviting pathologists from a predetermined mailing list also makes it difficult to obtain an exact survey response rate. We acknowledge that attitudes toward new technologies including AI can be fluid and have a tendency to change over time. Therefore, it will also be interesting to reassess attitudes toward AI in follow-up investigations. Moreover, we recognize the inherent difficulties in analyzing and merging the results of attitudes. Undoubtedly, the same question and answer options can represent different meanings to different responders who also have different baseline knowledge about the technology. Finally, ethical and legal aspects, which may vary between countries, were not addressed in our survey.

Conclusion

In summary, our results demonstrate an overall optimistic attitude toward AI in dermatopathology among surveyed physicians. Nonetheless, our data also highlight the need for education about AI for pathologists. Surveyed pathologists predict greater potential for automated suggestion of skin tumor diagnoses than for inflammatory skin diseases. The greatest potential of AI in dermatopathology was predicted for automated detection of mitoses and tumor margins as well as for immunostaining evaluation.

Data Availability Statement

The anonymized datasets generated in this survey are available on request to the corresponding author.

Ethics Statement

Ethical review and approval was not required for the study on human participants in accordance with the local legislation and institutional requirements. Written informed consent for participation was not required for this study in accordance with the national legislation and the institutional requirements.

Author Contributions

SP and JP contributed to study concept and design, supervised the project, and wrote the manuscript. SP carried out the implementation, data collection, analysis, and statistics with contributions from MG. MG provided biostatistical advice. PM, JG, JS, and NN contributed to design and distribution of the questionnaire. All authors critically reviewed and edited the questionnaire and the manuscript.

Conflict of Interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Supplementary Material

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fmed.2020.591952/full#supplementary-material

Supplementary Material 1. Complete survey text.

Supplementary Material 2. Aggregated survey responses.

Supplementary Table 1. Distribution of answers to questions regarding background knowledge about AI. Proportions of answers to questions on the specific rows stratified according to access to WSI, previous use of AI in pathology and previous use of AI in dermatopathology. P-values are computed with Fisher's exact test.

Supplementary Table 2. Distribution of answers to questions regarding attitudes and feelings about AI. Proportions of answers to questions on the specific rows stratified according to access to WSI, previous use of AI in pathology and previous use of AI in dermatopathology. P-values are computed with Fisher's exact test.

References

1. Tizhoosh HR, Pantanowitz L. Artificial intelligence and digital pathology: challenges and opportunities. J Pathol Inform. (2018) 9:38. doi: 10.4103/jpi.jpi_53_18

PubMed Abstract | CrossRef Full Text | Google Scholar

2. Litjens G, Kooi T, Bejnordi BE, Setio AAA, Ciompi F, Ghafoorian M, et al. A survey on deep learning in medical image analysis. Med Image Anal. (2017) 42:60–88. doi: 10.1016/j.media.2017.07.005

PubMed Abstract | CrossRef Full Text | Google Scholar

3. Bauer TW, Schoenfield L, Slaw RJ, Yerian L, Sun Z, Henricks WH. Validation of whole slide imaging for primary diagnosis in surgical pathology. Arch Pathol Lab Med. (2013) 137:518–24. doi: 10.5858/arpa.2011-0678-OA

PubMed Abstract | CrossRef Full Text | Google Scholar

4. Fertig RM, Sangueza O, Gaudi S, Gamret AC, Cervantes J, Jukic DM. Whole Slide Imaging. Am J Dermatopathol. (2018) 40:938–9. doi: 10.1097/DAD.0000000000001008

PubMed Abstract | CrossRef Full Text | Google Scholar

5. Onega T, Reisch LM, Frederick PD, Geller BM, Nelson HD, Lott JP, et al. Use of digital whole slide imaging in dermatopathology. J Digit Imaging. (2016) 29:243–53. doi: 10.1007/s10278-015-9836-y

CrossRef Full Text | Google Scholar

6. Kent MN, Olsen TG, Feeser TA, Tesno KC, Moad JC, Conroy MP, et al. Diagnostic accuracy of virtual pathology vs traditional microscopy in a large dermatopathology study. JAMA Dermatol. (2017) 153:1285–91. doi: 10.1001/jamadermatol.2017.3284

PubMed Abstract | CrossRef Full Text | Google Scholar

7. Madabhushi A, Lee G. Image analysis and machine learning in digital pathology: challenges and opportunities. Med Image Anal. (2016) 33:170–5. doi: 10.1016/j.media.2016.06.037

PubMed Abstract | CrossRef Full Text | Google Scholar

8. Morrison AS, Gardner JM. Smart phone microscopic photography: a novel tool for physicians and trainees. Arch Pathol Lab Med. (2014) 138:1002. doi: 10.5858/arpa.2013-0425-ED

PubMed Abstract | CrossRef Full Text | Google Scholar

9. Gardner JM, McKee PH. Social media use for pathologists of all ages. Arch Pathol Lab Med. (2019) 143:282–6. doi: 10.5858/arpa.2018-0431-ED

PubMed Abstract | CrossRef Full Text | Google Scholar

10. Schaumberg AJ, Juarez-Nicanor WC, Choudhury SJ, Pastrián LG, Pritt BS, Prieto Pozuelo M, et al. Interpretable multimodal deep learning for real-time pan-tissue pan-disease pathology search on social media. Modern Pathol. (2020). doi: 10.1038/s41379-020-0540-1. [Epub ahead of print].

PubMed Abstract | CrossRef Full Text | Google Scholar

11. Lang UE, Torres R, Cheung C, Vladar EK, McCalmont TH, Kim J, et al. Ciliation index is a useful diagnostic tool in challenging spitzoid melanocytic neoplasms. J Invest Dermatol. (2020) 140:1401–9.e2. doi: 10.1016/j.jid.2019.11.028

PubMed Abstract | CrossRef Full Text | Google Scholar

12. Olsen TG, Jackson BH, Feeser TA, Kent MN, Moad JC, Krishnamurthy S, et al. Diagnostic performance of deep learning algorithms applied to three common diagnoses in dermatopathology. J Pathol Inform. (2018) 9:32. doi: 10.4103/jpi.jpi_31_18

PubMed Abstract | CrossRef Full Text | Google Scholar

13. Fouad S, Randell D, Galton A, Mehanna H, Landini G. Unsupervised morphological segmentation of tissue compartments in histopathological images. PLoS ONE. (2017) 12:e0188717. doi: 10.1371/journal.pone.0188717

PubMed Abstract | CrossRef Full Text | Google Scholar

14. Gomolin A, Netchiporouk E, Gniadecki R, Litvinov IV. Artificial intelligence applications in dermatology: where do we stand? Front Med. (2020) 7:100. doi: 10.3389/fmed.2020.00100

PubMed Abstract | CrossRef Full Text | Google Scholar

15. Polesie S, Gillstedt M, Kittler H, Lallas A, Tschandl P, Zalaudek I, et al. Attitudes towards artificial intelligence within dermatology: an international online survey. Br J Dermatol. (2020) 183:159–61. doi: 10.1111/bjd.18875

PubMed Abstract | CrossRef Full Text | Google Scholar

16. Likert R. A technique for the measurement of attitudes. Arch Psychol. (1932)

Google Scholar

17. McKee Derm [Last accessed 10 September 2020]. Available online at: https://www.facebook.com/groups/mckeederm/

Google Scholar

18. Holm S. A simple sequentially rejective multiple test procedure. Scand J Stat. (1979) 65–70.

Google Scholar

19. Vajapey SP, Weber KL, Samora JB. Confidence gap between men and women in medicine: a systematic review. Curr Orthopaed Pract. (2020) 31:494–502. doi: 10.1097/BCO.0000000000000906

CrossRef Full Text | Google Scholar

20. Nomura K, Yano E, Fukui T. Gender differences in clinical confidence: a nationwide survey of resident physicians in Japan. Acad Med. (2010) 85:647–53. doi: 10.1097/ACM.0b013e3181d2a796

PubMed Abstract | CrossRef Full Text | Google Scholar

21. Andres C, Andres-Belloni B, Hein R, Biedermann T, Schäpe A, Brieu N, et al. iDermatoPath - a novel software tool for mitosis detection in H&E-stained tissue sections of malignant melanoma. J Eur Acad Dermatol Venereol. (2017) 31:1137–47. doi: 10.1111/jdv.14126

PubMed Abstract | CrossRef Full Text | Google Scholar

22. Nielsen PS, Spaun E, Riber-Hansen R, Steiniche T. Automated quantification of MART1-verified Ki-67 indices: useful diagnostic aid in melanocytic lesions. Hum Pathol. (2014) 45:1153–61. doi: 10.1016/j.humpath.2014.01.009

PubMed Abstract | CrossRef Full Text | Google Scholar

23. Campanella G, Hanna MG, Geneslaw L, Miraflor A, Werneck Krauss Silva V, Busam KJ, et al. Clinical-grade computational pathology using weakly supervised deep learning on whole slide images. Nat Med. (2019) 25:1301–9. doi: 10.1038/s41591-019-0508-1

PubMed Abstract | CrossRef Full Text | Google Scholar

24. Hekler A, Utikal JS, Enk AH, Solass W, Schmitt M, Klode J, et al. Deep learning outperformed 11 pathologists in the classification of histopathological melanoma images. Eur J Cancer. (2019) 118:91–6. doi: 10.1016/j.ejca.2019.06.012

PubMed Abstract | CrossRef Full Text | Google Scholar

25. Elmore JG, Barnhill RL, Elder DE, Longton GM, Pepe MS, Reisch LM, et al. Pathologists' diagnosis of invasive melanoma and melanocytic proliferations: observer accuracy and reproducibility study. BMJ. (2017) 357:j2813. doi: 10.1136/bmj.j2813

PubMed Abstract | CrossRef Full Text | Google Scholar

26. Unger J, Hebisch C, Phipps JE, Lagarto JL, Kim H, Darrow MA, et al. Real-time diagnosis and visualization of tumor margins in excised breast specimens using fluorescence lifetime imaging and machine learning. Biomed Opt Express. (2020) 11:1216–30. doi: 10.1364/BOE.381358

PubMed Abstract | CrossRef Full Text | Google Scholar

27. Sarwar S, Dent A, Faust K, Richer M, Djuric U, Van Ommeren R, et al. Physician perspectives on integration of artificial intelligence into diagnostic pathology. NPJ Digital Med. (2019) 2:28. doi: 10.1038/s41746-019-0106-0

PubMed Abstract | CrossRef Full Text | Google Scholar