Skip to main content

ORIGINAL RESEARCH article

Front. Digit. Health
Sec. Health Informatics
Volume 6 - 2024 | doi: 10.3389/fdgth.2024.1456911

Comparative Study of Claude 3.5-Sonnet and Human Physicians in Generating Discharge Summaries for Patients with Renal Insufficiency: Assessment of Efficiency, Accuracy, and Quality

Provisionally accepted
Li Xiaoyang Li Xiaoyang 1*Haijiao Jin Haijiao Jin 2Jinglu Guo Jinglu Guo 3Qisheng Lin Qisheng Lin 2Shaun Wu Shaun Wu 4Weiguo Hu Weiguo Hu 1
  • 1 Ruijin Hospital, School of Medicine, Shanghai Jiao Tong University, Shanghai, China
  • 2 Renji Hospital, School of Medicine, Shanghai Jiao Tong University, Shanghai, China
  • 3 Ningbo Hangzhou Bay Hospital, Ningbo, Zhejiang Province, China
  • 4 WORK Medical Technology Group LTD, Hangzhou, China

The final, formatted version of the article will be published soon.

    Background: The rapid development of artificial intelligence (AI) has shown great potential in medical document generation. This study aims to evaluate the performance of Claude 3.5-Sonnet, an advanced AI model, in generating discharge summaries for patients with renal insufficiency, compared to human physicians. Methods: A prospective, comparative study was conducted involving 100 patients (50 with acute kidney injury and 50 with chronic kidney disease) from the nephrology department of Ningbo Hangzhou Bay Hospital between January and June 2024. Discharge summaries were independently generated by Claude 3.5-Sonnet and human physicians. The main evaluation indicators included accuracy, generation time, and overall quality.Results: Claude 3.5-Sonnet demonstrated comparable accuracy to human physicians in generating discharge summaries for both AKI (90 vs. 92 points, p > 0.05) and CKD patients (88 vs. 90 points, p > 0.05). The AI model significantly outperformed human physicians in terms of efficiency, requiring only about 30 seconds to generate a summary compared to over 15 minutes for physicians (p < 0.001). The overall quality scores showed no significant difference between AI-generated and physician-written summaries for both AKI (26 vs. 27 points, p > 0.05) and CKD patients (25 vs. 26 points, p > 0.05).Conclusion: Claude 3.5-Sonnet demonstrates high efficiency and reliability in generating discharge summaries for patients with renal insufficiency, with accuracy and quality comparable to those of human physicians. These findings suggest that AI has significant potential to improve the efficiency of medical documentation, though further research is needed to optimize its integration into clinical practice and address ethical and privacy concerns.

    Keywords: artificial intelligence, Discharge summaries, Renal Insufficiency, Claude 3.5-Sonnet, Medical documentation

    Received: 29 Jun 2024; Accepted: 20 Nov 2024.

    Copyright: © 2024 Xiaoyang, Jin, Guo, Lin, Wu and Hu. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

    * Correspondence: Li Xiaoyang, Ruijin Hospital, School of Medicine, Shanghai Jiao Tong University, Shanghai, China

    Disclaimer: All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article or claim that may be made by its manufacturer is not guaranteed or endorsed by the publisher.