Evidence map›Paper›PMID 34075194›Full record

ArticleNPJ digital medicine2021

Automated coronary calcium scoring using deep learning with multicenter external validation.

David Eng, Christopher Chute, Nishith Khandwala, Pranav Rajpurkar, Jin Long, Sam Shleifer, Mohamed H Khalaf, Alexander T Sandhu, Fatima Rodriguez, David J Maron and 14 more

Registry-linked trialOpen access · goldAbstract read
In one paragraph

Article in NPJ digital medicine, 2021. The graph could read no effect estimate from its abstract, so it casts no vote on the map. It is linked to trial NCT05977413 (NOTIFY-OUTCOMES), which is not on this map. Cited by 99 papers, 2 of them syntheses that pooled it.

0numbers the graph read from it
0cells of the map it votes in
99citing papers in PubMed, 2 pooled it
14.0field-weighted citation impact, top 1% of its field
1 · What the graph read from it

What it found

Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.

The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.

2 · The registry

The trial behind it

Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.

NCT05977413 nawithdrawnnot on this mapstarted 2026, after this paper: background citation

NOTIFY-OUTCOMES (New Observations Taking Information From Yesterday)

TypeinterventionalSponsorStanford UniversityRan2026 to 2032Enrolled0ConditionsAtherosclerotic Cardiovascular DiseaseArmsCAC Notification, Clinician Guideline Reminder
3 · Its place in the literature

Who cites it

99 citing papers in PubMed, 2 syntheses or guidelines pooled it, 165 citations in OpenAlex.

  1. Guideline
  2. Pooled it
  3. Trial
  4. Fully automated deep learning powered calcium scoring in patients undergoing myocardial perfusion imaging.Journal of nuclear cardiology : official publication of the American Society of Nuclear Cardiology · 2023
    Trial
  5. Review
  6. Article
  7. Article
  8. Article
  9. Article
  10. Article
  11. Article
  12. Article
  13. Article
  14. Review
  15. Review
  16. Review
  17. Article
  18. Article
  19. Review
  20. Article

39 more citing papers are in PubMed but not listed here.

4 · The record

Corrections and comments

PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.

5 · Who and what money

Authors and funding

24 authors at 8 institutions in 2 countries.

David Eng *Department of Computer Science, Stanford University School of Medicine, Stanford, CA, USA.
Christopher Chute *Department of Computer Science, Stanford University School of Medicine, Stanford, CA, USA.
Nishith KhandwalaBunkerhill, Palo Alto, CA, USA.
Pranav RajpurkarDepartment of Computer Science, Stanford University School of Medicine, Stanford, CA, USA.ORCID http://orcid.org/0000-0002-8030-3727
Jin LongDepartment of Pediatrics, Stanford University School of Medicine, Stanford, CA, USA.
Sam ShleiferDepartment of Computer Science, Stanford University School of Medicine, Stanford, CA, USA.
Mohamed H KhalafDepartment of Radiology, Stanford University School of Medicine, Stanford, CA, USA.ORCID http://orcid.org/0000-0002-3323-2698
Alexander T SandhuDivision of Cardiovascular Medicine and Stanford Prevention Research Center, Department of Medicine, Stanford University School of Medicine, Palo Alto, CA, USA.
Fatima RodriguezDivision of Cardiovascular Medicine and Stanford Prevention Research Center, Department of Medicine, Stanford University School of Medicine, Palo Alto, CA, USA.ORCID http://orcid.org/0000-0002-5226-0723
David J MaronDivision of Cardiovascular Medicine and Stanford Prevention Research Center, Department of Medicine, Stanford University School of Medicine, Palo Alto, CA, USA.
Saeed SeyyediDepartment of Radiology, Stanford University School of Medicine, Stanford, CA, USA.
Daniele MarinDepartment of Radiology, Duke University Medical Center, Durham, NC, USA.
Ilana GolubLundquist Institute at Harbor-UCLA Medical Center, Torrance, CA, USA.
Matthew BudoffLundquist Institute at Harbor-UCLA Medical Center, Torrance, CA, USA.
Felipe KitamuraDiagnósticos da América SA (Dasa), Alphaville Barueri, SP, Brazil.ORCID http://orcid.org/0000-0002-9992-5630
Marcelo Straus TakahashiDiagnósticos da América SA (Dasa), Alphaville Barueri, SP, Brazil.ORCID http://orcid.org/0000-0001-9489-2844
Ross W FiliceDepartment of Radiology, MedStar Georgetown University Hospital, Washington, DC, USA.
Rajesh ShahRadiology Service, VA Palo Alto Health Care System, Palo Alto, CA, USA.ORCID http://orcid.org/0000-0003-4916-1377
John MonganDepartment of Radiology and Biomedical Imaging and Center for Intelligent Imaging, University of California, San Francisco, School of Medicine, San Francisco, CA, USA.ORCID http://orcid.org/0000-0003-2765-7451
Kimberly KallianosDepartment of Radiology and Biomedical Imaging and Center for Intelligent Imaging, University of California, San Francisco, School of Medicine, San Francisco, CA, USA.
Curtis P LanglotzDepartment of Radiology, Stanford University School of Medicine, Stanford, CA, USA.ORCID http://orcid.org/0000-0002-8972-8051
Matthew P LungrenDepartment of Radiology, Stanford University School of Medicine, Stanford, CA, USA.
Andrew Y NgDepartment of Computer Science, Stanford University School of Medicine, Stanford, CA, USA.
Bhavik N PatelDepartment of Radiology, Mayo Clinic, Scottsdale, AZ, USA. patel.bhavik@mayo.edu.ORCID http://orcid.org/0000-0001-5157-9903
Stanford University · USCentrus Diagnósticos por Imagem · BRUCLA Medical Center · USUniversity of California, San Francisco · USDuke Medical Center · USGeorgetown University · USMayo Clinic in Arizona · USVA Palo Alto Health Care System · US

Funding

Stanford Medicine Center for Longevity and Healthy Aging Research Education CoreP30AG059307 · NIA · STANFORD UNIVERSITY · PI PERIYAKOIL, VYJEYANTHI S, YESAVAGE, JEROME A · 2018 to 2025
$4.9M
SURPASS: (Statin Use and Risk Prediction of Atherosclerotic Cardiovascular Disease in minority Subgroups)K01HL144607 · NHLBI · STANFORD UNIVERSITY · PI RODRIGUEZ, FATIMA · 2019 to 2023
$856k
NHLBI NIH HHS K01 HL144607
6 · The paper itself

Abstract

Coronary artery disease (CAD), the most common manifestation of cardiovascular disease, remains the most common cause of mortality in the United States. Risk assessment is key for primary prevention of coronary events and coronary artery calcium (CAC) scoring using computed tomography (CT) is one such non-invasive tool. Despite the proven clinical value of CAC, the current clinical practice implementation for CAC has limitations such as the lack of insurance coverage for the test, need for capital-intensive CT machines, specialized imaging protocols, and accredited 3D imaging labs for analysis (including personnel and software). Perhaps the greatest gap is the millions of patients who undergo routine chest CT exams and demonstrate coronary artery calcification, but their presence is not often reported or quantitation is not feasible. We present two deep learning models that automate CAC scoring demonstrating advantages in automated scoring for both dedicated gated coronary CT exams and routine non-gated chest CTs performed for other reasons to allow opportunistic screening. First, we trained a gated coronary CT model for CAC scoring that showed near perfect agreement (mean difference in scores = -2.86; Cohen's Kappa = 0.89, P < 0.0001) with current conventional manual scoring on a retrospective dataset of 79 patients and was found to perform the task faster (average time for automated CAC scoring using a graphics processing unit (GPU) was 3.5 ± 2.1 s vs. 261 s for manual scoring) in a prospective trial of 55 patients with little difference in scores compared to three technologists (mean difference in scores = 3.24, 5.12, and 5.48, respectively). Then using CAC scores from paired gated coronary CT as a reference standard, we trained a deep learning model on our internal data and a cohort from the Multi-Ethnic Study of Atherosclerosis (MESA) study (total training n = 341, Stanford test n = 42, MESA test n = 46) to perform CAC scoring on routine non-gated chest CT exams with validation on external datasets (total n = 303) obtained from four geographically disparate health systems. On identifying patients with any CAC (i.e., CAC ≥ 1), sensitivity and PPV was high across all datasets (ranges: 80-100% and 87-100%, respectively). For CAC ≥ 100 on routine non-gated chest CTs, which is the latest recommended threshold to initiate statin therapy, our model showed sensitivities of 71-94% and positive predictive values in the range of 88-100% across all the sites. Adoption of this model could allow more patients to be screened with CAC scoring, potentially allowing opportunistic early preventive interventions.

Identifiers

PMID34075194
PMCPMC8169744
OpenAlexW3164278751

What Socratic holds

Textmetadata
LicenceCC BY
Read underepoch 390

Registered trials

Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the Socratic graph.