ArticleBMC medical research methodology2023
Performance metrics for models designed to predict treatment effect.
Article in BMC medical research methodology, 2023. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 11 papers, 1 of them a synthesis that pooled it.
What it found
Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.
The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.
The trial behind it
Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.
Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.
Who cites it
11 citing papers in PubMed, 1 synthesis or guideline pooled it.
- Personalized decision-making for aneurysm treatment of aneurysmal subarachnoid hemorrhage: development and validation of a clinical prediction tool.BMC neurology · 2024Pooled it
- Machine learning methods for estimating personalized treatment effects-insights on validity from two large trials.American journal of epidemiology · 2026Article
- Individualised treatment effects of corticosteroids in IgA nephropathy.EBioMedicine · 2026Article
- Validation of the postoperative prognostication tool PREDICT version 2.2 and 3.0 using data from the National cancer center hospital in Japan.Breast cancer (Tokyo, Japan) · 2026Article
- Effect of discontinuing antipsychotic medications on the risk of hospitalization in long-term care: a machine learning-based analysis.BMC medicine · 2025Article
- Review
- Measuring the Performance of Survival Models to Personalize Treatment Choices.Statistics in medicine · 2025Article
- Modeling Normal Tissue Complication Probability of Radiation-Induced Alopecia Following Intensity-Modulated Radiation Therapy in Glioblastoma Patients.Advanced biomedical research · 2025Article
- Pitfalls of single-study external validation illustrated with a model predicting functional outcome after aneurysmal subarachnoid hemorrhage.BMC medical research methodology · 2024Article
- Methodological concerns about "concordance-statistic for benefit" as a measure of discrimination in predicting treatment benefit.Diagnostic and prognostic research · 2023Article
- Measuring the performance of prediction models to personalize treatment choice.Statistics in medicine · 2023Article
Corrections and comments
PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.
Authors and funding
6 authors.
Funding
No grant is acknowledged in the PubMed record.
Abstract
backgroundMeasuring the performance of models that predict individualized treatment effect is challenging because the outcomes of two alternative treatments are inherently unobservable in one patient. The C-for-benefit was proposed to measure discriminative ability. However, measures of calibration and overall performance are still lacking. We aimed to propose metrics of calibration and overall performance for models predicting treatment effect in randomized clinical trials (RCTs).
methodsSimilar to the previously proposed C-for-benefit, we defined observed pairwise treatment effect as the difference between outcomes in pairs of matched patients with different treatment assignment. We match each untreated patient with the nearest treated patient based on the Mahalanobis distance between patient characteristics. Then, we define the E
resultsAs desired, performance metric values of "perturbed models" were consistently worse than those of the "optimal model" (E
conclusionThe proposed metrics are useful to assess the calibration and overall performance of models predicting treatment effect in RCTs.
Indexed as
Identifiers
What Socratic holds
Registered trials
Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the Socratic graph.