Accuracy and stability of DeepSeek-R1 and GPT-4o on pediatric questions.

Background: Large language models are increasingly applied in medical education and clinical decision support. However, comparative evaluations of their performance on pediatric content-spanning multiple subspecialties and question complexities-remain limited. This study sought to assess the performance of two advanced large language models, DeepSeek-R1 and GPT-4o, using a comprehensive set of Pediatrician Licensing Exam practice questions. Methods: We administered 280 expert-validated pediatric questions covering 11 subspecialties and three levels of clinical complexity (A1: direct, A2: simple cases, A3: complex cases) to DeepSeek-R1 and GPT-4o. Each model was tested twice, with a 4-week interval. Performance metrics, including per-run accuracy, consistent accuracy (correct in both runs), aggregate accuracy (correct in either run), and inter-run consistency, were compared using chi-square tests and Bonferroni correction. Results: DeepSeek-R1 outperformed GPT-4o in both runs (1 Conclusion: DeepSeek-R1 demonstrated higher accuracy than GPT-4o on pediatric questions. The observed variability across different runs warrants attention, particularly in contexts that require highly consistent and reproducible performance.