Evidence map›Paper›PMID 39755432›Full record

ArticleThe American journal of clinical nutrition2025

Reliability and reproducibility of systematic reviews informing the 2020-2025 Dietary Guidelines for Americans: a pilot study.

Alexandra M Bodnaruc, Hassan Khan, Nicole Shaver, Alexandria Bennett, Yiu Lin Wong, Catherine Gracey, Valentina Ly, Beverley Shea, Julian Little, Melissa Brouwers and 2 more

Abstract read
PubMed Full text at PMC Publisher
In one paragraph

Article in The American journal of clinical nutrition, 2025. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 3 papers, 1 of them a synthesis that pooled it.

0numbers the graph read from it
0cells of the map it votes in
3citing papers in PubMed, 1 pooled it
–field-weighted citation impact
1 · What the graph read from it

What it found

Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.

The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.

2 · The registry

The trial behind it

Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.

Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.

3 · Its place in the literature

Who cites it

3 citing papers in PubMed, 1 synthesis or guideline pooled it.

  1. Effect ofFrontiers in pharmacology · 2025
    Pooled it
  2. Review
  3. Article
4 · The record

Corrections and comments

PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.

5 · Who and what money

Authors and funding

12 authors.

Alexandra M BodnarucSchool of Epidemiology and Public Health, Faculty of Medicine, University of Ottawa, Ottawa, ON, Canada.
Hassan KhanSchool of Epidemiology and Public Health, Faculty of Medicine, University of Ottawa, Ottawa, ON, Canada.
Nicole ShaverSchool of Epidemiology and Public Health, Faculty of Medicine, University of Ottawa, Ottawa, ON, Canada. Electronic address: nicole.shaver@uottawa.ca.
Alexandria BennettSchool of Epidemiology and Public Health, Faculty of Medicine, University of Ottawa, Ottawa, ON, Canada.
Yiu Lin WongSchool of Epidemiology and Public Health, Faculty of Medicine, University of Ottawa, Ottawa, ON, Canada; Clinical Epidemiology Program, Ottawa Hospital Research Institute, Ottawa, ON, Canada; School of Chinese Medicine, Hong Kong Baptist University, Hong Kong SAR, China.
Catherine GraceyDalhousie University, Halifax, NS, Canada.
Valentina LyUniversity of Ottawa, Ottawa, ON, Canada.
Beverley SheaSchool of Epidemiology and Public Health, Faculty of Medicine, University of Ottawa, Ottawa, ON, Canada; Clinical Epidemiology Program, Ottawa Hospital Research Institute, Ottawa, ON, Canada.
Julian LittleSchool of Epidemiology and Public Health, Faculty of Medicine, University of Ottawa, Ottawa, ON, Canada.
Melissa BrouwersSchool of Epidemiology and Public Health, Faculty of Medicine, University of Ottawa, Ottawa, ON, Canada.
Dennis BierChildren's Nutrition Research Center, Baylor College of Medicine, Houston, TX, United States.
David MoherSchool of Epidemiology and Public Health, Faculty of Medicine, University of Ottawa, Ottawa, ON, Canada; Clinical Epidemiology Program, Ottawa Hospital Research Institute, Ottawa, ON, Canada.

Funding

No grant is acknowledged in the PubMed record.

6 · The paper itself

Abstract

backgroundAlthough high-quality nutrition systematic reviews (SRs) are important for clinical decision making, there remains debate on their methodological quality and reporting transparency.

objectivesThe objective of this study was to assess the reliability and reproducibility of a sample of SRs produced by the Nutrition Evidence Systematic Review (NESR) team to inform the 2020-2025 Dietary Guidelines for Americans (DGAs).

methodsWe evaluated a sample of 8 SRs from the DGA dietary patterns subcommittee for methodological quality using the Assessment of Multiple Systematic Reviews 2 (AMSTAR 2) tool and for reporting transparency using the PRISMA 2020 and PRISMA literature search extension (PRISMA-S) checklists. We assessed the quality and reproducibility of the original search strategy of one selected SR using the Peer Review of Electronic Search Strategies checklist. The reporting transparency of the SR's narrative data synthesis was assessed using the Synthesis Without Meta-Analysis (SWiM) checklist. Interpretation bias was evaluated using existing spin bias classifications in systematic reviews.

resultsThe AMSTAR 2 assessment identified critical methodological weaknesses, and all included SRs were judged to be of critically low quality. Overall, 74% of the PRISMA 2020 checklist items and 63% of the PRISMA-S checklist items were satisfactorily fulfilled. We identified several errors and inconsistencies in the search strategy and could not reproduce searches within a 10% margin of the original results. The SWiM assessment identified concerns regarding the reporting transparency of the narrative data synthesis, but the spin bias assessment revealed no evidence of interpretation bias.

conclusionsSeveral methodological quality and reporting concerns were identified, which could lead to reliability and reproducibility issues should a full reproduction attempt be made. However, additional research is needed to confirm the impact of these findings on conclusions statements and their generalizability across the NESR team SRs. This study was registered in the Open Science Framework (https://osf.io/ns6a9/).

Indexed as

Nutrition PolicySystematic Reviews as TopicGuidelines as TopicHumansPilot ProjectsReproducibility of ResultsUnited States

Identifiers

PMID39755432
PMCPMC11747194

What Socratic holds

Textmetadata
LicenceCC BY
Read underepoch 390

Registered trials

None linked

Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the Socratic graph.