Comparative evaluation of large language model-based AI platforms for radiographic assessment in rheumatoid arthritis.

Rheumatoid arthritis (RA) is a chronic autoimmune disease characterized by progressive joint damage. Early diagnosis of this disease is crucial to prevent disability. The traditional interpretation of X-ray images in RA still depends heavily on radiologists, whose availability is limited in many regions across the world. Modern advances in large language models (LLMs) offer potential support for radiological assessments. The aim of this study is to compare the diagnostic capabilities of artificial intelligence (AI) platforms — GPT-5 Thinking, Gemini, and DeepSeek — in identifying radiological signs characteristic of RA on hand X-rays, using the consensus of radiological experts as the reference standard. A comparative diagnostic study was conducted using 20 anonymized radiographs of both hands of patients with clinical signs of RA. Eight specific radiological signs were evaluated: articular space narrowing, deformity, central erosions, marginal erosions, osteopenia, osteophytes, subluxations, and ankylosis. Six certified radiologists independently evaluated the images, and a reference standard was developed. The results of AI model diagnostics were compared with those of experts across 143 observations that could be evaluated. The diagnostic effectiveness was assessed using sensitivity, specificity, accuracy, and the Kappa coefficient of agreement. Gemini demonstrated 66% sensitivity, 63% specificity, and 65% accuracy (k = 0.292, p < 0.001). GPT-5 Thinking was characterized by lower sensitivity (56%) and higher specificity (78%) and 66% accuracy (k = 0.350, p < 0.001). DeepSeek showed balanced performance with sensitivity, specificity, and accuracy of 65% each (k = 0.295, p < 0.001). All of the platforms demonstrated fair, statistically significant compliance with expert assessments. The examined LLMs demonstrate limited but statistically significant capabilities for detecting radiological signs of RA, with varying sensitivities and specificities. These models cannot replace expert radiological assessment, but they can serve as auxiliary tools for pre-screening and obtaining opinions for educational purposes, especially in conditions of limited access to specialist radiologists.