• Isr Med Assoc J · Feb 2024

    Performance of ChatGPT in Israeli Hebrew Internal Medicine National Residency Exam.

    • David J Ozeri, Adiel Cohen, Noa Bacharach, Offir Ukashi, and Amit Oppenheim.
    • Division of Rheumatology, Sheba Medical Center, Tel Hashomer, Israel.
    • Isr Med Assoc J. 2024 Feb 1; 26 (2): 868886-88.

    BackgroundCompleting internal medicine specialty training in Israel involves passing the Israel National Internal Medicine Exam (Shlav Aleph), a challenging multiple-choice test. multiple-choice test. Chat generative pre-trained transformer (ChatGPT) 3.5, a language model, is increasingly used for exam preparation.ObjectivesTo assess the ability of ChatGPT 3.5 to pass the Israel National Internal Medicine Exam in Hebrew.MethodsUsing the 2023 Shlav Aleph exam questions, ChatGPT received prompts in Hebrew. Textual questions were analyzed after the appeal, comparing its answers to the official key.ResultsChatGPT 3.5 correctly answered 36.6% of the 133 analyzed questions, with consistent performance across topics, except for challenges in nephrology and biostatistics.ConclusionsWhile ChatGPT 3.5 has excelled in English medical exams, its performance in the Hebrew Shlav Aleph was suboptimal. Factors include limited training data in Hebrew, translation complexities, and unique language structures. Further investigation is essential for its effective adaptation to Hebrew medical exam preparation.

      Pubmed     Copy Citation     Plaintext  

      Add institutional full text...

    Notes

     
    Knowledge, pearl, summary or comment to share?
    300 characters remaining
    help        
    You can also include formatting, links, images and footnotes in your notes
    • Simple formatting can be added to notes, such as *italics*, _underline_ or **bold**.
    • Superscript can be denoted by <sup>text</sup> and subscript <sub>text</sub>.
    • Numbered or bulleted lists can be created using either numbered lines 1. 2. 3., hyphens - or asterisks *.
    • Links can be included with: [my link to pubmed](http://pubmed.com)
    • Images can be included with: ![alt text](https://bestmedicaljournal.com/study_graph.jpg "Image Title Text")
    • For footnotes use [^1](This is a footnote.) inline.
    • Or use an inline reference [^1] to refer to a longer footnote elseweher in the document [^1]: This is a long footnote..

    hide…