Evaluation of an interpretable deep-learning model for the automated plan review of intensity-modulated radiation therapy.

Background: Intensity-modulated radiation therapy (IMRT) is a treatment modality for delivering a higher radiation dose to the target using multiple radiation beams from various gantry angles. Since an IMRT plan is complex and presents a high risk, quality assurance is crucial. In routine clinical practice, a radiotherapy treatment plan review is manually conducted by experienced planners to ensure the high quality and reliability of an IMRT plan. To assist this procedure, several machine-learning methods have been developed to automatically identify outliers or anomalous plans from a large amount of treatment plans. Purpose: Due to the highly complex and non-linear nature of most machine-learning methods, their results are less explainable, and therefore, cannot be fully trusted by clinical users. To alleviate this issue, an unsupervised deep-learning model with an interpretable tool was introduced and evaluated on our clinical database. Methods: Six hundred IMRT treatment plans were collected from our institute and the relevant features were extracted. A standard autoencoder (AE) was used to build an anomaly detection model from the normal plans and identify anomalies with exceptionally high reconstruction errors. To help explain the model output, a feature perturbation interpretation (FPI) method was introduced to rank the impacts of the features on the detected anomalies. To validate the effectiveness of the FPI method, the feature ranks provided by the local depth-based isolation forest feature importance (Local-DIFFI) and SHapley Additive exPlanations (SHAP) methods were compared. In addition, the effectiveness of the FPI method was evaluated on the other four classic detection models, local discrete factor (LOF), density-based spatial clustering of applications with noise (HDBSCAN), one class of support vector machines (OC-SVM), and principal component analysis (PCA). Results: The AE achieved the best detection performance among all the methods. The area under the curve (AUC) value of the AE was 0.98, and the average accuracy, precision, and F1 score were 0.91, 0.61, and 0.74, respectively. The top five features with the largest impacts on the detected anomalies obtained by FPI, local-DIFFI, and SHAP are the same. Conclusion: The AE is an effective deep-learning model in identifying anomalous plans compared to classic detection models. The FPI method is universal and a reliable tool in determining the feature impact on the anomalies. Their combination provided a highly interpretable deep learning model for automatic plan review in radiotherapy.