Performance of large language models in neonatal resuscitation assessments versus healthcare providers: an exploratory study.

Background: Artificial intelligence and large language models (LLMs) have developed rapidly in recent years and involved in medical education, in addition to clinical care. However, it remains unknown how the performance of LLMs in neonatal resuscitation compares to that of healthcare professionals (HCPs). In this exploratory study, we aimed to investigate and compare the performance of LLMs with those of HCPs on 3 sources of examination questions in the neonatal resuscitation training in Canada and China. Methods: In this bi-center study, we evaluated the overall accuracy, accuracy across question types, and reliability of LLMs' (ChatGPT-5 and DeepSeek-R1) responses to neonatal resuscitation questions from workshop in China, NRP® textbook (8 Results: Both LLMs performed comparably to HCPs in Chinese examinations, and showed similar accuracy in NRP® textbook questions with higher scores on multiple-choice than on short answer questions. ChatGPT achieved higher accuracy than DeepSeek and HCPs in the Kahoot quizzes. ChatGPT also had higher accuracy on scenario-based than on non-scenario-based questions in the workshop examination. High reliability of LLMs' responses was found (Fleiss' Kappa>0.89). Conclusion: Both ChatGPT and DeepSeek achieved accuracy comparable to that of HCPs and showed high consistency on selected neonatal resuscitation written examinations. The findings warrant further research to explore their potential integration in neonatal resuscitation training.