ArticlePNAS nexus2025
Risk of bias and low reproducibility in meta-analytic evidence from fast-tracked publications during the coronavirus disease 2019 pandemic.
Article in PNAS nexus, 2025. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Not yet cited in PubMed.
What it found
Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.
The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.
The trial behind it
Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.
Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.
Who cites it
0 citing papers in PubMed.
No citing paper in PubMed yet.
Corrections and comments
PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.
Authors and funding
22 authors.
Funding
No grant is acknowledged in the PubMed record.
Abstract
The fast-tracked publication of coronavirus disease 2019 (COVID-19)-related meta-analytic evidence has undeniably facilitated rapid public health policymaking; however, concerns are mounting that this publication policy may compromise research quality and scientific integrity. To investigate this, we conducted a meta-research study systematically evaluating risk of bias (ROB), transparency, and reproducibility in pandemic-era meta-analyses synthesizing COVID-19-derived mental health problem epidemics. From 98 identified studies-including data from 18.6 million individuals across 94 countries-we observed significant ROBs in publication, with one new meta-analysis published approximately every 5 days at peak output. Despite apparent sample diversity, nearly half of participants were from China, and only 8.9% originated from less economically developed countries. Of these meta-analyses, a substantial proportion (70.6%) showed discrepancies between Preferred Reporting Items for Systematic Reviews and Meta-Analyses-guided reporting and actual research conducts, while 57.1% exhibited high methodological ROBs due to insufficient data sources and lack of sensitivity analysis. Alarmingly, none achieved full computational reproducibility, and fewer than one-fifth were fully replicable. Furthermore, neither publication in high-impact journals, citation performance, nor fast-track publication mode correlated with lower ROBs that we identified above. To address these limitations, we re-estimated global COVID-19-derived mental health epidemics using their individual participant data after minimizing identified ROBs. Our recalibrated meta-analytic findings provide more reliable benchmarks for understanding the pandemic's mental health impact. This study demonstrated that rigorous methodology and scientific integrity must remain central priorities-even under urgent, crisis-driven conditions-establishing a foundation for transparent, reproducible, and unbiased global mental health surveillance during public health emergencies.
Indexed as
Identifiers
What Socratic holds
Registered trials
Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the Socratic graph.