Validation of multiple deep learning models for colorectal tumor differentiation with endoscopic ultrasound images: a dual-center study.

Background: Colorectal cancer (CRC) is one of the most common malignancies worldwide. Differentiating adenomas and cancers in colorectal lesions is essential for reducing morbidity and mortality associated with CRC. Endoscopic ultrasound (EUS) is crucial in the diagnosis of CRC, and artificial intelligence (AI) offers a promising approach for identifying colorectal lesions without the need for histopathological confirmation. The objective of this study was to validate the efficacy of EUS combined with AI for the diagnosis of colorectal adenoma and cancer and to compare it with that of conventional endoscopic diagnosis. Methods: This retrospective study included 554 patients (167 with CRC, 136 with adenomas, and 251 controls) from two independent centers. The dataset was randomly divided into training and test sets in a 2:1 ratio (360 for the training dataset; 194 for the testing dataset). A model was developed using a "feature extractor + multilayer perceptron (MLP) classifier" framework, incorporating Residual Network 50 (ResNet50), EfficientNet-B0, Visual Geometry Group 11_BN (VGG_11_BN), and Vision Transformer (ViT) as feature extractors. Four AI systems were trained and validated, and the model with the highest F1 scores was subsequently compared to four endoscopists using the test dataset, and interobserver agreement measured by Fleiss' kappa. Results: The accuracies for three-category classification (CRC, adenoma and controls) were 70.62% for ResNet50, 68.56% for EfficientNet-B0, 63.4% for ViT, and 70.10% for VGG_11_BN. ResNet50 achieved the highest F1 scores (70.37%) and diagnostic accuracy and was selected for comparison with endoscopists. For CRC diagnosis, ResNet50 outperformed endoscopists with an accuracy of 80.93%, sensitivity of 72.88%, and specificity of 84.44%, which were significantly higher than those of all endoscopists (P<0.05). For adenoma diagnosis, ResNet50 had a sensitivity of 47.92%, which was significantly higher than that of nonexpert endoscopists (P<0.05). The interobserver agreement was fair among AI systems (Fleiss' κ =0.674) and among experts (Fleiss' κ =0.557) and was slight among nonexperts (Fleiss' κ =0.284). Conclusions: EUS-AI has high diagnostic accuracy for CRC and adenoma as compared to non-expert endoscopists. ResNet50 is a promising tool for enhancing diagnostic accuracy in clinical practice using EUS.