Breast cancer diagnosis from histopathological images and molecular signatures by fusing features with an explainable AI-based residual tabular network model.

Early Breast Cancer (BC) Diagnosis has the potential to cut BC death rates in the long term drastically. Identifying early-stage cancer cells is the most crucial step in determining the best prognosis. Despite recent advances in the use of AI-based methods, such as machine learning and deep learning (DL), to detect breast cancer, current models are generally limited to simple binary classification of data, rely on a single source of data, and lack transparency, thereby limiting their clinical applicability. To overcome these limitations, we proposed an Explainable Artificial Intelligence (AI)-based Residual Tabular Network (ResTab Net) model based on integrating histopathological images and molecular protein expression data patterns to conduct multimodal BC diagnosis. The proposed model utilizes Adaptive Tissue-Aware Gaussian Filtering (ATGF) to enhance the image, Entropy Enhanced Graph-Watershed Segmentation (EGWS) to clearly define the tumor's location, and Self-Adaptive Starfish Optimization (SASFO) to select the features. A hybrid framework of residual convolutional blocks and dense layers can facilitate successful multiclass classification. To ensure tangible transparency and clinical trust, the model captures SHapley Additive exPlanations (SHAP) and Local Interpretable Model-Agnostic Explanations (LIME) approaches illustrates the impact of molecular protein levels, including image features on classification results. The proposed Explainable AI-ResTab Net model is implemented using Python. The performance evolution of the proposed model achieves an accuracy of 98.56%, a precision of 98.10%, a recall of 98.00%, an F1-score of 98.03%, and an Area Under the Curve (AUC) of 99.60%.