ArticleNature medicine2025
Benchmark evaluation of DeepSeek large language models in clinical decision-making.
Article in Nature medicine, 2025. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 136 papers.
What it found
Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.
The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.
The trial behind it
Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.
Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.
Who cites it
136 citing papers in PubMed.
- Large language models enhance diagnostic reasoning of medical students in rheumatology: a randomized controlled trial.BMC medical education · 2026Trial
- Evolution of guideline-based male infertility responses by large language models between 2025 and 2026: a comparative analysis of ChatGPT, Perplexity, and DeepSeek.Basic and clinical andrology · 2026Article
- ChatGPT Versus DeepSeek in Resectability Categorization of Chronic Rhinosinusitis: A Comparison Based on EPOS 2020.World journal of otorhinolaryngology - head and neck surgery · 2026Article
- Comparison of Two AI Chatbots for Diagnosis and Providing Treatment Suggestions in Retinopathy of Prematurity: Retrospective Study.Journal of medical Internet research · 2026Article
- Article
- Performance of large language models in the Chinese National Medical Licensing Examinations and Beyond: Scoping review.PLOS digital health · 2026Article
- Comparison of DeepSeek AI and Human Reviewers in Structured Evaluation of Neurosurgical Manuscripts: A Pilot Study.Journal of neurological surgery reports · 2026Article
- Detecting Misspelled Drug Names Using Transformer-Based Language Models: Model Development and External Validation.JMIR medical informatics · 2026Article
- Development of a Framework for Evaluating Large Language Model Safety and Reliability: a Proof-of-Concept Evaluation.Journal of medical systems · 2026Article
- Large Language Model Performance on Multistep Clinical Cases: Comparative Study Across Question and Case Levels.JMIR medical education · 2026Article
- Leveraging time-series electronic health records with large language models for chronic kidney disease diagnosis in primary care.BMJ health & care informatics · 2026Article
- A large language model-driven multidisciplinary AI agent system predicts delirium in emergency critically ill patients.Cell reports. Medicine · 2026Article
- LungGPT: A unified multimodal system for interpretable diagnosis and clinical decision support of respiratory diseases.Cell reports. Medicine · 2026Article
- Unmasking bias in the evidence ecosystem: a panoramic analysis of 311,751 meta-analyses using an artificial intelligence agent-based approach.Journal of global health · 2026Article
- A Secure, Scalable Large Language Model-Based System (CIDER) for High-Throughput Clinical Data Extraction From Medical Reports: Retrospective Validation Study.Journal of medical Internet research · 2026Article
- Novelty to necessity: large language models in clinical practice-a call to action for physicians.Internal and emergency medicine · 2026Review
- ASO Author Reflections: Benchmarking Large Language Models for Bladder Cancer: Comparative Knowledge Retrieval and Clinical Reasoning of DeepSeek and ChatGPT.Annals of surgical oncology · 2026Article
- Modern generative large language models in epilepsy care: a scoping review of current applications, challenges, and future directions.Acta epileptologica · 2026Review
- Psychometric evaluation of the Spanish-language version of the health information technology usability evaluation scale (Health-ITUES).International journal of medical informatics · 2026Article
- Error Detection and Correction in Chinese Radiology Reports Using Large Language Models: Real-World Clinical Validation Study.Journal of medical Internet research · 2026Article
76 more citing papers are in PubMed but not listed here.
Corrections and comments
PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.
Authors and funding
7 authors.
Funding
Abstract
Large language models (LLMs) are increasingly transforming medical applications. However, proprietary models such as GPT-4o face significant barriers to clinical adoption because they cannot be deployed on site within healthcare institutions, making them noncompliant with stringent privacy regulations. Recent advancements in open-source LLMs such as DeepSeek models offer a promising alternative because they allow efficient fine-tuning on local data in hospitals with advanced information technology infrastructure. Here, to demonstrate the clinical utility of DeepSeek-V3 and DeepSeek-R1, we benchmarked their performance on clinical decision support tasks against proprietary LLMs, including GPT-4o and Gemini-2.0 Flash Thinking Experimental. Using 125 patient cases with sufficient statistical power, covering a broad range of frequent and rare diseases, we found that DeepSeek models perform equally well and in some cases better than proprietary LLMs. Our study demonstrates that open-source LLMs can provide a scalable pathway for secure model training enabling real-world medical applications in accordance with data privacy and healthcare regulations.
Indexed as
Identifiers
What Socratic holds
Registered trials
Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the Socratic graph.