AI Models for Suicide Risk Prediction in Adult Patients Receiving Mental Health Care Using Real-World Data: Retrospective Population-Based Study.

Background: Suicidal behavior is a major public health problem worldwide. The exact etiology remains unclear, representing a complex problem involving multiple factors. Evidence indicates that around 50% to 80% of people who die by suicide have had contact with the health care system in the year prior to their death. Objective: We present the IDICIUS project, whose objective is to develop a clinical decision support system that functions as an early warning system and applies AI to prevent suicide risk using anonymized electronic health record data. Methods: This study shows the first 2 stages of the IDICIUS project, where real-world data from 4 public sources were integrated, curated, standardized, and anonymized in phase 1, and analysis and modeling of suicide risk were performed in phase 2. This retrospective population-based study included 41,557 adult patients receiving mental health care at a large hospital. We evaluated the performance of machine learning classifiers with increasing complexity: logistic regression, elastic net, decision trees and random forests, bagging and boosting ensemble methods (GradientBoosting, XGBoost [extreme gradient boosting], CatBoost, RandomForest, and AdaBoost [adaptive boosting]), support vector machines, and deep neural networks. To address class imbalance, we tested several balancing techniques, with random undersampling providing the best results. Results: Phase 1 yielded a useful database of 32,661 patients receiving mental health care and 112 features. Of those, 2764 patients exhibited suicidal behavior (target), while 29,897 did not (control). The undersampling reduced the class distribution to 4422 vs 2211 (control vs target). The most prevalent features in the target class (N=2764) were psychiatric conditions, anxiety episodes (n=2269, 82.1%) and depressive episodes (n=1682, 60.9%), along with demographic and behavioral factors: female sex (n=1672, 60.5%), alcohol consumption (n=770, 27.9%), and conduct disorder (n=709, 25.7%). The largest patient subgroups were females with combined depression and anxiety (n=300, 10.9%), females with anxiety disorders only (n=182, 6.6%), males with anxiety disorders only (n=120, 4.3%), and males with combined depression and anxiety (n=106, 3.8%). In phase 2, ensemble methods, especially GradientBoosting and XGBoost, achieved the best performance, with receiver operating characteristic-area under the curve scores around 0.95. While these models showed moderate precision in identifying true positives (0.51-0.58), they demonstrated high sensitivity in detecting at-risk patients, with recall scores of 0.85 and 0.84, respectively. Both models achieved a good balance between precision and recall ( Conclusions: We integrated multiple electronic health record data sources and applied AI to develop, model, and optimize ensemble algorithms for suicide prevention, maximizing effectiveness, efficiency, and generalizability. These AI-generated algorithms, based on readily available and well-structured data, may advance the identification of at-risk patients and function as early warning systems, thereby increasing opportunities for timely preventive interventions.