Evidence map›Paper›PMID 36054583›Full record

ArticleResearch synthesis methods2023

Identifying and managing problematic trials: A research integrity assessment tool for randomized controlled trials in evidence synthesis.

Stephanie Weibel, Maria Popp, Stefanie Reis, Nicole Skoetz, Paul Garner, Emma Sydenham

Abstract read
PubMed Full text at PMC Publisher
In one paragraph

Article in Research synthesis methods, 2023. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 44 papers, 18 of them syntheses that pooled it.

0numbers the graph read from it
0cells of the map it votes in
44citing papers in PubMed, 18 pooled it
–field-weighted citation impact
1 · What the graph read from it

What it found

Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.

The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.

2 · The registry

The trial behind it

Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.

Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.

3 · Its place in the literature

Who cites it

44 citing papers in PubMed, 18 syntheses or guidelines pooled it.

  1. Pooled it
  2. Intraperitoneal chemotherapy for gastric cancer.The Cochrane database of systematic reviews · 2025
    Pooled it
  3. Pooled it
  4. Pooled it
  5. Pooled it
  6. Ketamine and other NMDA receptor antagonists for chronic pain.The Cochrane database of systematic reviews · 2025
    Pooled it
  7. Polyphenols for the Prevention or Management of Preeclampsia: A Systematic Review and Meta-Analysis.BJOG : an international journal of obstetrics and gynaecology · 2025
    Pooled it
  8. Pooled it
  9. Pooled it
  10. Pooled it
  11. Pooled it
  12. Pooled it
  13. Pooled it
  14. Pooled it
  15. Nirmatrelvir combined with ritonavir for preventing and treating COVID-19.The Cochrane database of systematic reviews · 2023
    Pooled it
  16. Pooled it
  17. Nirmatrelvir combined with ritonavir for preventing and treating COVID-19.The Cochrane database of systematic reviews · 2022
    Pooled it
  18. Ivermectin for preventing and treating COVID-19.The Cochrane database of systematic reviews · 2022
    Pooled it
  19. Article
  20. Review
4 · The record

Corrections and comments

PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.

5 · Who and what money

Authors and funding

6 authors.

Stephanie WeibelDepartment of Anaesthesiology, Intensive Care, Emergency and Pain Medicine, University Hospital Wuerzburg, Wuerzburg, Germany.ORCID https://orcid.org/0000-0003-1900-2535
Maria PoppDepartment of Anaesthesiology, Intensive Care, Emergency and Pain Medicine, University Hospital Wuerzburg, Wuerzburg, Germany.
Stefanie ReisDepartment of Anaesthesiology, Intensive Care, Emergency and Pain Medicine, University Hospital Wuerzburg, Wuerzburg, Germany.
Nicole SkoetzEvidence-based Oncology, Department I of Internal Medicine, Center for Integrated Oncology Aachen Bonn Cologne Duesseldorf, Faculty of Medicine and University Hospital Cologne, University of Cologne, Cologne, Germany.
Paul GarnerCentre for Evidence Synthesis in Global Health, Liverpool School of Tropical Medicine, Liverpool, UK.
Emma SydenhamCochrane Injuries Group, London School of Hygiene & Tropical Medicine, London, UK.

Funding

Cochrane Complementary Medicine Field: Resource for ResearchR24AT001293 · NCCIH · UNIVERSITY OF MARYLAND BALTIMORE · PI BERMAN, BRIAN M, WIELAND, LISA SUSAN · 2003 to 2022
$9.4M
NCCIH NIH HHS R24 AT001293
6 · The paper itself

Abstract

Evidence synthesis findings depend on the assumption that the included studies follow good clinical practice and results are not fabricated or false. Studies which are problematic due to scientific misconduct, poor research practice, or honest error may distort evidence synthesis findings. Authors of evidence synthesis need transparent mechanisms to identify and manage problematic studies to avoid misleading findings. As evidence synthesis authors of the Cochrane COVID-19 review on ivermectin, we identified many problematic studies in terms of research integrity and regulatory compliance. Through iterative discussion, we developed a research integrity assessment (RIA) tool for randomized controlled trials for the update of this Cochrane review. In this paper, we explain the rationale and application of the RIA tool in this case study. RIA assesses six study criteria: study retraction, prospective trial registration, adequate ethics approval, author group, plausibility of methods (e.g., randomization), and plausibility of study results. RIA was used in the Cochrane review as part of the eligibility check during screening of potentially eligible studies. Problematic studies were excluded and studies with open questions were held in awaiting classification until clarified. RIA decisions were made independently by two authors and reported transparently. Using the RIA tool resulted in the exclusion of >40% of studies in the first update of the review. RIA is a complementary tool prior to assessing "Risk of Bias" aiming to establish the integrity and authenticity of studies. RIA provides a platform for urgent development of a standard approach to identifying and managing problematic studies.

Indexed as

COVID-19Scientific MisconductBiasHumansProspective StudiesRandomized Controlled Trials as TopicCOVID-19 pandemicevidence synthesisgood clinical practicerandomized controlled trialresearch integritysystematic review

Identifiers

PMID36054583
PMCPMC10551123

What Socratic holds

Textmetadata
LicenceTDM
Read underepoch 390

Registered trials

None linked

Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the Socratic graph.