AI Trustworthiness in the Perioperative Period for Patients with Serious Illness: Scoping Narrative Review.

Background: Despite the promising potential of AI in the perioperative context, the rapid pace of development and diverse implementation warrant a thorough review to consolidate existing knowledge, identify gaps, and assess the use of trustworthiness principles in AI integration into the perioperative period for patients with serious illness. Objective: The purpose of this study was to address deficiencies in the perioperative AI literature by elucidating the extent to which discussions of equity, ethics, and safety are incorporated, thereby establishing a foundation for the development of robust ethical guidelines for the safe and effective integration of AI in health care. Methods: We searched PubMed, Embase, CENTRAL (Cochrane Central Register of Controlled Trials), and Scopus for studies published from 2010 to July 2024. We included studies that reported patient functional outcomes, occurred in the perioperative period (30 d before and up to 90 d after surgery), incorporated AI integration, and included patients with serious illness (defined as malignancy, advanced organ failure, frailty, dementia or neurodegenerative disease, or stroke). To ensure reliability and minimize bias, 2 independent reviewers screened all studies at the title or abstract and full-text stages; conflicts were resolved through team consensus. The abstraction form was developed iteratively and was tested through pilot abstractions. Any discrepancies identified during data extraction were resolved through discussion and consensus among the reviewers. The ROBINS-I (Risk of Bias Tool in Nonrandomized Studies of Interventions) tool was used to assess quality. Abstraction and risk assessment were conducted through a blinded, independent dual-review process. A narrative review was compiled from the identified studies. Results: Of the 10,980 papers identified through the database searches, this review yielded 81 papers that met the inclusion criteria. Analysis of AI implementation strategies revealed foundational efforts toward equitable access, with 6 studies providing open-access tools and several more designing models with simple inputs suitable for low-resource settings (17 studies). Seven studies mentioned their commitment to transparency (eg, publishing code) to enhance safety and trust. However, significant ethical deficiencies persist, particularly regarding input data, as only 2 studies explicitly addressed racial or ethnic disparities, and concerns about lack of sample diversity (16 studies) and the omission of socially relevant features (5 studies) were frequently noted as limitations. Conclusions: Machine learning for predictive analytics and other types of AI tools for surgical outcomes offer significant potential but require adherence to trustworthiness and safety principles to be clinically viable. Future research should prioritize adherence to guidelines for equity, ethics, and safety, conduct prospective studies, incorporate more external validation of AI models, and facilitate transparent monitoring and reporting of model performance to build clinician and patient trust and encourage broader health care system adoption.