Patient perspectives of artificial intelligence as a medical device in a skin cancer pathway

The use of artificial intelligence as a medical device (AIaMD) in healthcare systems is increasing rapidly. In dermatology, this has been accelerated in response to increasing skin cancer referral rates, workforce shortages and backlog generated by the COVID-19 pandemic. Evidence regarding patient perspectives of AIaMD is currently lacking in the literature. Patient acceptability is fundamental if this novel technology is to be effectively integrated into care pathways and patients must be confident that it is implemented safely, legally, and ethically. A prospective, single-center, single-arm, masked, non-inferiority, adaptive, group sequential design trial, recruited patients referred to a teledermatology cancer pathway. AIaMD assessment of dermoscopic images were compared with clinical or histological diagnosis, to assess performance (NCT04123678). Participants completed an online questionnaire to evaluate their views regarding use of AIaMD in the skin cancer pathway. Two hundred and sixty eight responses were received between February 2020 and August 2021. The majority of respondents were female (57.5%), ranged in age between 18 and 93 years old, Fitzpatrick type I-II skin (81.3%) and all 6 skin types were represented. Overall, there was a positive sentiment regarding potential use of AIaMD in skin cancer pathways. The majority of respondents felt confident in computers being used to help doctors diagnose and formulate management plans (median = 70; interquartile range (IQR) = 50–95) and as a support tool for general practitioners when assessing skin lesions (median = 85; IQR = 65–100). Respondents were comfortable having their photographs taken with a mobile phone device (median = 95; IQR = 70–100), which is similar to other studies assessing patient acceptability of teledermatology services. To the best of our knowledge, this is the first comprehensive study evaluating patient perspectives of AIaMD in skin cancer pathways in the UK. Patient involvement is essential for the development and implementation of new technologies. Continued end-user feedback will allow refinement of services to ensure patient acceptability. This study demonstrates patient acceptability of the use of AIaMD in both primary and secondary care settings.

Introduction

The use of artificial intelligence (AI) is currently being explored across the field of medicine and the deployment of AI as a medical device (AIaMD) in healthcare systems is rapidly expanding. In dermatology, this has been accelerated in response to increasing skin cancer referral rates, workforce shortages and backlog generated by the COVID-19 pandemic (1). AI is making significant contributions in dermatology especially with automated skin lesion analysis, triage of cutaneous lesions, skin cancer detection, and dermatological image recognition, offering benefits to enhance various aspects of patient care (2). However while AI holds great potential, it is not without its challenges with potential concerns regarding patient data privacy and confidentiality, while ensuring these technologies are validated, reliable and accurate (3).

The deep ensemble for recognition of malignancy (DERM) device, designed by Skin Analytics, is an AIaMD that analyzes images of skin lesions to support the identification and appropriate management of skin cancers, premalignant lesions, and benign conditions. DERM was the first UKCA Class IIa certified AIaMD dermatology device on the UK market (4).

Evidence regarding patient perspectives of AIaMD being used by doctors to help make decisions about their care, is currently lacking in the literature. Patient acceptability is fundamental if this novel technology is to be effectively integrated into care pathways and patients must be confident that it is implemented safely, legally, and ethically.

The aim of this study was to explore patients’ perspectives on the use of AI as part of their skin cancer management pathway.

Methods

A prospective, single-center, single-arm, masked, non-inferiority, adaptive, group sequential design trial, designed to demonstrate the potential of DERM to reduce unnecessary referrals, was conducted at Chelsea and Westminster Hospital in London, UK (ClinicalTrials.gov: NCT04123678). Patients over the age of 18, who were referred to a teledermatology skin cancer clinic with at least one skin lesion that could be photographed, were eligible for the study. Patients provided written informed consent for the study, and there was no financial compensation. Ethical approval for the study was granted by the West Midlands, Edgbaston Research Ethics Committee.

Patients attended an appointment with a clinical photographer based within the hospital. In addition to images of the lesions captured for standard of care assessment, macroscopic and dermoscopic images of each skin lesion were taken by a healthcare assistant using an iPhone X smartphone with Dermlite DL1 basic dermoscopic lens attachment. Captured images were uploaded for analysis by DERM, which was certified as a Class I AIaMD at the time. The DERM analysis result was not shared with the patient or the dermatologist, and the patient’s care continued in accordance with routine standard of care. Information on lesion history, risk factors for skin cancer, number of appointments needed to diagnose, and the final diagnosis were collected (5).

After their assessment participants were sent a link by email to an online questionnaire (Supplementary material) which was designed to evaluate their views regarding potential use of AIaMD in the skin cancer pathway. The questionnaire was hosted on an electronic Case Report Form that could be linked with the study record. Reminders were sent to patients who had not completed the survey after at least 1 week. The questionnaire included 4 questions on healthcare appointments prior to their teledermatology appointment, and 14 questions that evaluated patient acceptance of: (i) clinic and photography appointments (ii) AI as a service tool, which were worded both positively and negatively to minimize bias. A visual analog scale (VAS) was used to assess respondents’ satisfaction. The VAS ranged from 0 to 100 with a score of >50 taken to indicate an agreement with a given statement. The impact of patient factors (age, sex, and fitzpatrick skin type) and management outcome on the patient’s response were evaluated using a Kruskal-Wallis (KW) test, with statistical significance set at p < 0.05. Statistical analysis was conducted using the R language version 4.1.3 and environment for statistical computing.

Results

Seven hundred patients were recruited between February 2020 and August 2021, including 12 patients who consented twice. Two hundred and sixty eight questionnaire responses were received (38.2% response rate), including two patients who completed the questionnaire twice. Respondents ranged in age between 18 and 93 years old. Most respondents were female (n = 154, 57.5%) and had Fitzpatrick type I-II skin (n = 218, 81.3%); however all 6 skin types were represented (Table 1).

TABLE 1

Table 1. Baseline characteristics of study population.

Most patients (n = 207, 77.5%) attended the teledermatology clinic within 14 days of their GP appointment and reported that this time was “about right” for them (Table 2). Most patients (n = 191, 71.3%) reported never, or only once or twice, visiting a doctor about the same skin lesions in the past 5 years, with the median number of prior healthcare appointments, to assess the skin lesion/s included in the study, being 1 (IQR = 1–1, max 6).

TABLE 2

Table 2. Appointments made by respondents prior to attending teledermatology clinic.

Overall, there was a positive sentiment regarding potential use of AIaMD in skin cancer pathways (Table 3). The majority of respondents felt confident in ‘computers’ being used to help doctors diagnose and formulate management plans (median = 70; interquartile range (IQR) = 50–95) and as a support tool for general practitioners when assessing skin lesions (median = 85; IQR = 65–100). The majority would rather have had their skin assessed by a computer than wait weeks to see an in-person dermatologist (median = 70; IQR = 50–97.5).

TABLE 3

Table 3. Summary table of results from patient satisfaction of AIaMD in skin cancer pathways.

Responses for most questions (9 out of 14) were comparable across the sub-groups assessed, with no significant variation in the median scores. Differences in responses were most frequently associated with the outcome of the teledermatology assessment, reaching statistical significance for four questions. Women were found to be less comfortable having photographs of their lesions taken, compared to men, while no statistically significant differences in responses were associated with respondent’s age (Table 4).

TABLE 4

Table 4. Survey questions with statistically significant variation in median scores across sub-groups of respondents.

Discussion

AI has demonstrated potential to enhance skin cancer detection and improve efficiency in urgent cancer pathways (5), through the development of several machine learning algorithms to distinguish malignant from benign skin lesions (6). While ongoing technologies are being developed, it is paramount that patient perspectives of AI are explored in parallel, to ensure acceptability of this new technology, and to help inform successful large scale deployment into clinical pathways. Structured feedback from patients who are involved in clinical research and early deployments of AIaMD is one way in which this data can be collected.

To the best of our knowledge, this is the first comprehensive study evaluating patient perspectives of AIaMD in skin cancer pathways in the UK. Our cohort involved a large group of patients that reflect the local population who are referred on a cancer pathway, with all six Fitzpatrick skin types being represented.

Overall our study revealed a positive sentiment regarding potential use of AIaMD in skin cancer pathways. This complements a qualitative study conducted in Germany reporting 75% would recommend AI tools for skin cancer screening to family and friends, with 94% of patients expressing acceptance of the symbiosis between clinicians and AI systems (7).

The majority of our respondents felt confident in computers being used to help doctors diagnose and formulate management plans and as a support tool for general practitioners when assessing skin lesions. Importantly, our survey highlighted acceptability of AIaMD alongside clinicians as a decision-making support tool, however further assessment for stand-alone autonomous applications is required. Respondents were comfortable having their photographs taken, which is similar to other studies assessing patient acceptability of teledermatology services (8), though the differences in responses between sexes may be relevant for the wider deployment of teledermatology. Differences in responses across the Fitzptrick skin types may be influenced by the comparatively small number of responders with Fitzpatrick skin types IV–VI, and the significance of these results should be interpreted with caution.

The differences in responses associated with the outcome of the teledermatology review is interesting as those patients who were referred for a biopsy or urgent referral were consistently more willing to accept the AIaMD as part of their assessment than those who were discharged or referred for a routine appointment. This suggests patients are more amenable to new technologies being used to inform their care when they feel their condition is being more actively managed.

A common limitation of patient surveys is low participation rates, and the resultant self-selection bias with feedback missing from those patients who are unwilling or unable to participate, or those who simply forget to complete the questionnaire. The response rate for this survey was almost 40%, which is similar to response rates to online surveys elsewhere (9, 10). However, it remains possible that the results presented here are not wholly representative of the views of all patients recruited into the clinical study, and indeed the wider population of patients attending teledermatology clinics.

Further work is required to evaluate the psychological status of patients whose care involves an assessment by an AIaMD, compared to those who just attend face to face consultations. Patient feedback will continue to be important as products like DERM develop, and the clinical patient pathways in which they are deployed evolve. Further, larger studies are needed to capture patient feedback from more diverse populations, including different socio-economic groups and a wider variety of ethnicities and skin colors also focusing on acceptability of autonomous AIaMD in clinical pathways.

Conclusion

To the best of our knowledge, this is the first comprehensive study evaluating patient perspectives of AIaMD in skin cancer pathways in the UK. Patient involvement is essential for the successful development and implementation of new technologies. Continued end-user feedback will allow refinement of services to ensure patient acceptability. This study demonstrates patient acceptability of AIaMD in both primary and secondary care settings.

Data availability statement

The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.

Ethics statement

The studies involving humans were approved by the West Midlands, Edgbaston Research Ethics Committee. The studies were conducted in accordance with the local legislation and institutional requirements. The patients provided their written informed consent to participate in this study.

Author contributions

AK: Writing – original draft, Writing – review & editing. KH: Writing – review & editing. DK: Data curation, Investigation, Methodology, Resources, Validation, Writing – review & editing. PK: Formal analysis, Writing – review & editing. HM: Formal analysis, Writing – review & editing, Conceptualization, Data curation, Investigation, Methodology, Project administration, Resources, Software, Validation, Visualization. LT: Conceptualization, Methodology, Project administration, Supervision, Visualization, Writing – review & editing.

Funding

The study was funded by Skin Analytics Ltd., as part of a grant from Innovate UK.

Acknowledgments

The authors would like to thank all patients who consented to the study and completed the patient survey; Chelsea and Westminster Hospital staff involved in patient recruitment, data collection, and diagnosis confirmation; and Skin Analytics staff involved in the development of the AIaMD and the operations of the study.

Conflict of interest

LT is a clinical advisor to Skin Analytics Ltd., has received Skin Analytics shares or share options; has received research funding support from Skin Analytics (salaries and equipment) and AIaMD deployment programme; has received reimbursement of conference fees, travel and accommodation costs from Skin Analytics to present research results; LT has received financial remuneration for separate programme of work as a consultant by Skin Analytics; has received grant funding from NHSX and CW+; has received paid honoraria to lecture for Almirall; was supported to attend a conference by Abbvie and Janssen; and holds multiple unpaid leadership roles. HM is an employee of Skin Analytics Ltd., and has received Skin Analytics shares or share options. DK is an employee of Skin Analytics Ltd., and has received Skin Analytics shares or share options. PK was previously a contractor with Skin Analytics Ltd. The remaining authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Skin Analytics, London, UK sponsored and funded this study, as part of an Innovate UK BioMedical Catalyst project, and was involved with the study design, data collection, statistical analysis and interpretation of the data.

Publisher’s note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Supplementary material

The Supplementary material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fmed.2023.1259595/full#supplementary-material

References

1. Mahmood, F, Bendayan, S, Ghazawi, FM, and Litvinov, IV. Editorial: the emerging role of artificial intelligence in dermatology. Front Med (Lausanne). (2021) 8:751649. doi: 10.3389/fmed.2021.751649

CrossRef Full Text | Google Scholar

2. He, J, Baxter, SL, Xu, J, Xu, J, Zhou, X, and Zhang, K. The practical implementation of artificial intelligence technologies in medicine. Nat Med. (2019) 25:30–6. doi: 10.1038/s41591-018-0307-0

CrossRef Full Text | Google Scholar

3. Liopyris, K, Gregoriou, S, Dias, J, and Stratigos, AJ. Artificial intelligence in dermatology: challenges and perspectives. Dermatol Ther (Heidelb). (2022) 12:2637–51. doi: 10.1007/s13555-022-00833-8

CrossRef Full Text | Google Scholar

4. Esteva, A, Kuprel, B, Novoa, RA, Ko, J, Swetter, SM, Blau, HM, et al. Dermatologist-level classification of skin cancer with deep neural networks. Nature. (2017) 542:115–8. doi: 10.1038/nature21056

CrossRef Full Text | Google Scholar

5. Thomas, L, Hyde, C, Mullarkey, D, Greenhalgh, J, Kalsi, D, Ko, JM, et al. Real-world post-deployment performance of a novel machine learning-based digital health technology for skin lesion assessment and suggestions for post-market surveillance. Front Med.(2023) 10:1264846.

Google Scholar

6. Marsden, H, Kemos, P, Venzi, M, Noy, M, Maheswaran, S, Francis, N, et al. Accuracy of an artificial intelligence as a medical device as part of a UK-based skin cancer teledermatology service. Front. Med. (2023).

Google Scholar

7. Phillips, M, Marsden, H, Jaffe, W, Matin, RN, Wali, GN, Greenhalgh, J, et al. Assessment of accuracy of an artificial intelligence algorithm to detect melanoma in images of skin lesions. JAMA Netw Open. (2019) 2:e1913436. doi: 10.1001/jamanetworkopen.2019.13436

CrossRef Full Text | Google Scholar

8. Nelson, CA, Pérez-Chada, LM, Creadore, A, Li, SJ, Lo, K, Manjaly, P, et al. Patient perspectives on the use of artificial intelligence for skin Cancer screening: a qualitative study. JAMA Dermatol. (2020) 156:501–12. doi: 10.1001/jamadermatol.2019.5014

CrossRef Full Text | Google Scholar

9. Nicholson, P, Macedo, C, Fuller, C, and Thomas, L. Patient satisfaction with a new skin cancer teledermatology service. Clin Exp Dermatol. (2020) 45:691–8. doi: 10.1111/ced.14191

CrossRef Full Text | Google Scholar

10. Meyer, VM, Benjamens, S, Moumni, ME, Lange, JFM, and Pol, RA. Global overview of response rates in patient and health care professional surveys in surgery: a systematic review. Ann Surg. (2022) 275:e75–81. doi: 10.1097/SLA.0000000000004078

CrossRef Full Text | Google Scholar