ArticleEuropean radiology2025
ChatGPT's diagnostic performance based on textual vs. visual information compared to radiologists' diagnostic performance in musculoskeletal radiology.
Article in European radiology, 2025. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 41 papers, 1 of them a synthesis that pooled it.
What it found
Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.
The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.
The trial behind it
Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.
Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.
Who cites it
41 citing papers in PubMed, 1 synthesis or guideline pooled it.
- Comparing Diagnostic Accuracy of Clinical Professionals and Large Language Models: Systematic Review and Meta-Analysis.JMIR medical informatics · 2025Pooled it
- Performance of multimodal large language models versus clinicians for radiographic knee osteoarthritis grading: A multiobserver study.Skeletal radiology · 2026Article
- Assessing and mitigating demographic bias in large language models for diagnostic radiology.Japanese journal of radiology · 2026Article
- Agreement in Staging and Treatment Recommendations Among Clinicians, Text-Only Clinicians, DeepSeek-V3, and ChatGPT-4o for Nasopharyngeal Carcinoma Patients.Cancer medicine · 2026Article
- Prospective quantitative analysis of hyperparameter and input optimization in GPT-5: comparative contribution to radiologist performance in abdominal radiology.Diagnostic and interventional radiology (Ankara, Turkey) · 2026Article
- Evaluating the Performance of ChatGPT-4V in Detecting Inflammatory Magnetic Resonance Imaging Findings of Sacroiliitis: Potentials, Challenges, and Limitations.Journal of imaging informatics in medicine · 2026Article
- Evaluating expert decision-pattern alignment in endovascular planning for intracranial aneurysms using a multimodal large language model.Neuroradiology · 2026Article
- Aesthetic Component of the Index of Orthodontic Treatment Need (IOTN): Can large language models match the performance of orthodontists?Korean journal of orthodontics · 2026Article
- Consensus-Level and Cluster-Adjusted Evaluation of a Large Language Model for Diagnostic Extraction from Musculoskeletal Radiology Reports.Diagnostics (Basel, Switzerland) · 2026Article
- Diagnostic Performance and Confidence Calibration of Large Language Models for Bone Tumor Radiographs.Diagnostics (Basel, Switzerland) · 2026Article
- Artificial superintelligence alignment in healthcare.Japanese journal of radiology · 2026Review
- Evaluating the Accuracy and Diagnostic Reasoning of Multimodal Large Language Models in Interpreting Neuroradiology Cases FromKorean journal of radiology · 2026Article
- Bot vs. doc-who is better at reading proximal humerus fracture x-rays?JSES international · 2026Article
- Independent and collaborative performance of large language models and healthcare professionals in diagnosis and triage.NPJ digital medicine · 2026Article
- Integrating large language models (LLMs) into neuro-oncologic radiology: a retrospective feasibility assessment of GPT-4o for brain tumor diagnosis.Neuroradiology · 2026Article
- Human-large language model collaboration in clinical medicine: a systematic review and meta-analysis.NPJ digital medicine · 2026Article
- Limitations in Chest X-Ray Interpretation by Vision-Capable Large Language Models, Gemini 1.0, Gemini 1.5 Pro, GPT-4 Turbo, and GPT-4o.Diagnostics (Basel, Switzerland) · 2026Article
- Evaluation of the Diagnostic Performance of ChatGPT in Radiographic Staging of Sacroiliitis According to the Modified New York Criteria.Archives of rheumatology · 2026Article
- Artificial intelligence in radiology: diagnostic sensitivity of ChatGPT for detecting hemorrhages in cranial computed tomography scans.Diagnostic and interventional radiology (Ankara, Turkey) · 2026Article
- Observational
Corrections and comments
PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.
Authors and funding
10 authors.
Funding
No grant is acknowledged in the PubMed record.
Abstract
objectivesTo compare the diagnostic accuracy of Generative Pre-trained Transformer (GPT)-4-based ChatGPT, GPT-4 with vision (GPT-4V) based ChatGPT, and radiologists in musculoskeletal radiology. MATERIALS AND
methodsWe included 106 "Test Yourself" cases from Skeletal Radiology between January 2014 and September 2023. We input the medical history and imaging findings into GPT-4-based ChatGPT and the medical history and images into GPT-4V-based ChatGPT, then both generated a diagnosis for each case. Two radiologists (a radiology resident and a board-certified radiologist) independently provided diagnoses for all cases. The diagnostic accuracy rates were determined based on the published ground truth. Chi-square tests were performed to compare the diagnostic accuracy of GPT-4-based ChatGPT, GPT-4V-based ChatGPT, and radiologists.
resultsGPT-4-based ChatGPT significantly outperformed GPT-4V-based ChatGPT (p < 0.001) with accuracy rates of 43% (46/106) and 8% (9/106), respectively. The radiology resident and the board-certified radiologist achieved accuracy rates of 41% (43/106) and 53% (56/106). The diagnostic accuracy of GPT-4-based ChatGPT was comparable to that of the radiology resident, but was lower than that of the board-certified radiologist although the differences were not significant (p = 0.78 and 0.22, respectively). The diagnostic accuracy of GPT-4V-based ChatGPT was significantly lower than those of both radiologists (p < 0.001 and < 0.001, respectively).
conclusionGPT-4-based ChatGPT demonstrated significantly higher diagnostic accuracy than GPT-4V-based ChatGPT. While GPT-4-based ChatGPT's diagnostic performance was comparable to radiology residents, it did not reach the performance level of board-certified radiologists in musculoskeletal radiology. CLINICAL RELEVANCE STATEMENT: GPT-4-based ChatGPT outperformed GPT-4V-based ChatGPT and was comparable to radiology residents, but it did not reach the level of board-certified radiologists in musculoskeletal radiology. Radiologists should comprehend ChatGPT's current performance as a diagnostic tool for optimal utilization. KEY POINTS: This study compared the diagnostic performance of GPT-4-based ChatGPT, GPT-4V-based ChatGPT, and radiologists in musculoskeletal radiology. GPT-4-based ChatGPT was comparable to radiology residents, but did not reach the level of board-certified radiologists. When utilizing ChatGPT, it is crucial to input appropriate descriptions of imaging findings rather than the images.
Indexed as
Identifiers
What Socratic holds
Registered trials
Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the Socratic graph.