Development and Validation of a Multi-Task Artificial Intelligence-Assisted System for Small Bowel Capsule Endoscopy.

Objective: To develop a multi-task artificial intelligence-assisted system for small bowel capsule endoscopy (SBCE) based on various Transformer neural network architectures. The system integrates lesion recognition, cumulative time statistics, and progress bar marking functions to enhance the efficiency and accuracy of endoscopic image interpretation while effectively reducing missed diagnoses. Methods: A dataset comprising 12 annotated categories of images captured by three different brands of capsule endoscopy devices was collected. Transfer learning and fine-tuning were conducted on five pre-trained Transformer models. Performance metrics, including accuracy, sensitivity, specificity, and recognition speed, were evaluated to select the best-performing model. The optimal model was converted from PyTorch to Open Neural Network Exchange (ONNX) format. Using OpenCV and MMCV tools, a multi-task SBCE-assisted reading system was developed. Results: A total of 34,799 images were included in the study. The best-performing model, FocalNet, achieved a weighted average sensitivity of 85.69%, specificity of 98.58%, accuracy of 85.69%, and an AUC of 0.98 across all categories. Its diagnostic accuracy outperformed junior physicians ( Conclusion: The multi-task SBCE-assisted reading system developed using Transformer networks demonstrated rapid and accurate classification of various small bowel lesions. It holds significant potential in enhancing diagnostic efficiency and image review speed for endoscopists. However, the AI system has not yet been validated in prospective clinical trials, and further real-world studies are needed to confirm its clinical applicability.