Evidence map›Paper›PMID 29929538›Full record

ReviewImplementation science : IS2018

Enhancing the uptake of systematic reviews of effects: what is the best format for health care managers and policy-makers? A mixed-methods study.

Christine Marquez, Alekhya Mascarenhas Johnson, Sabrina Jassemi, Jamie Park, Julia E Moore, Caroline Blaine, Gertrude Bourdon, Mark Chignell, Moriah E Ellen, Jacques Fortin and 22 more

Abstract readReview
PubMed Full text at PMC Publisher
In one paragraph

Review in Implementation science : IS, 2018. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 27 papers, 1 of them a synthesis that pooled it.

0numbers the graph read from it
0cells of the map it votes in
27citing papers in PubMed, 1 pooled it
–field-weighted citation impact
1 · What the graph read from it

What it found

Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.

The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.

2 · The registry

The trial behind it

Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.

Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.

3 · Its place in the literature

Who cites it

27 citing papers in PubMed, 1 synthesis or guideline pooled it.

  1. Pooled it
  2. Trial
  3. Article
  4. Article
  5. Article
  6. Review
  7. Article
  8. Article
  9. Article
  10. Article
  11. Article
  12. Article
  13. Article
  14. Study within a review (SWAR).Journal of evidence-based medicine · 2022
    Article
  15. Article
  16. Review
  17. Review
  18. Experiences of Using Cochrane Systematic Reviews by Local HTA Units.International journal of health policy and management · 2022
    Review
  19. Article
  20. Article
4 · The record

Corrections and comments

PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.

5 · Who and what money

Authors and funding

32 authors.

Christine MarquezLi Ka Shing Knowledge Institute, St. Michael's Hospital, Toronto, Canada. marquezc@smh.ca.ORCID 0000-0003-3564-5303
Alekhya Mascarenhas JohnsonLi Ka Shing Knowledge Institute, St. Michael's Hospital, Toronto, Canada.
Sabrina JassemiLi Ka Shing Knowledge Institute, St. Michael's Hospital, Toronto, Canada.
Jamie ParkLi Ka Shing Knowledge Institute, St. Michael's Hospital, Toronto, Canada.
Julia E MooreLi Ka Shing Knowledge Institute, St. Michael's Hospital, Toronto, Canada.
Caroline BlaineBMJ Knowledge Centre, London, UK.
Gertrude BourdonCentre Hospitalier Universitaire de Québec (CHUQ), Quebec City, Canada.
Mark ChignellDepartment of Medicine, Faculty of Medicine, University of Toronto, Toronto, Canada.
Moriah E EllenBen Gurion University, Beer Sheva, Israel.
Jacques FortinAgence de la santé et des services sociaux Montérégie, Longueuil, Quebec City, Canada.
Ian D GrahamOttawa Hospital Research Institute, Ottawa, Canada.
Anne HayesOntario Ministry of Health and Long-Term Care, Toronto, Canada.
Jemila HamidLi Ka Shing Knowledge Institute, St. Michael's Hospital, Toronto, Canada.
Brenda HemmelgarnDepartment of Community Health Sciences, Cumming School of Medicine, University of Calgary, Calgary, Canada.
Michael HillmerInstitute of Health Policy, Management and Evaluation, University of Toronto, Toronto, Canada.
Bev HolmesMichael Smith Foundation for Health Research, Vancouver, Canada.
Jayna Holroyd-LeducDepartment of Medicine, Cumming School of Medicine, University of Calgary, Calgary, Canada.
Linda HubertCentre Hospitalier Universitaire de Sherbrooke (CHUS), Quebec, Canada.
Brian HuttonOttawa Hospital Research Institute, Ottawa, Canada.
Monika KastnerInstitute of Health Policy, Management and Evaluation, University of Toronto, Toronto, Canada.
John N LavisMcMaster Health Forum, Department of Health Research Methods, Evidence and Impact, and Department of Political Science, McMaster University, Hamilton, Canada.
Karen MichellSinai Health System, Toronto, Canada.
David MoherOttawa Hospital Research Institute, Ottawa, Canada.
Mathieu OuimetLaval University, Quebec City, Canada.
Laure PerrierInstitute of Health Policy, Management and Evaluation, University of Toronto, Toronto, Canada.
Andrea ProctorOntario Ministry of Health and Long-Term Care, Toronto, Canada.
Thomas NoseworthyDepartment of Community Health Sciences, Cumming School of Medicine, University of Calgary, Calgary, Canada.
Victoria SchuckelMinistry of Health, Victoria, Canada.
Sharlene StaybergAlberta Health, Edmonton, Canada.
Marcello TonelliDepartment of Medicine, Cumming School of Medicine, University of Calgary, Calgary, Canada.
Andrea C TriccoLi Ka Shing Knowledge Institute, St. Michael's Hospital, Toronto, Canada.
Sharon E StrausLi Ka Shing Knowledge Institute, St. Michael's Hospital, Toronto, Canada.

Funding

CIHR 312381
6 · The paper itself

Abstract

backgroundSystematic reviews are infrequently used by health care managers (HCMs) and policy-makers (PMs) in decision-making. HCMs and PMs co-developed and tested novel systematic review of effects formats to increase their use.

methodsA three-phased approach was used to evaluate the determinants to uptake of systematic reviews of effects and the usability of an innovative and a traditional systematic review of effects format. In phase 1, survey and interviews were conducted with HCMs and PMs in four Canadian provinces to determine perceptions of a traditional systematic review format. In phase 2, systematic review format prototypes were created by HCMs and PMs via Conceptboard©. In phase 3, prototypes underwent usability testing by HCMs and PMs.

resultsTwo hundred two participants (80 HCMs, 122 PMs) completed the phase 1 survey. Respondents reported that inadequate format (Mdn = 4; IQR = 4; range = 1-7) and content (Mdn = 4; IQR = 3; range = 1-7) influenced their use of systematic reviews. Most respondents (76%; n = 136/180) reported they would be more likely to use systematic reviews if the format was modified. Findings from 11 interviews (5 HCMs, 6 PMs) revealed that participants preferred systematic reviews of effects that were easy to access and read and provided more information on intervention effectiveness and less information on review methodology. The mean System Usability Scale (SUS) score was 55.7 (standard deviation [SD] 17.2) for the traditional format; a SUS score < 68 is below average usability. In phase 2, 14 HCMs and 20 PMs co-created prototypes, one for HCMs and one for PMs. HCMs preferred a traditional information order (i.e., methods, study flow diagram, forest plots) whereas PMs preferred an alternative order (i.e., background and key messages on one page; methods and limitations on another). In phase 3, the prototypes underwent usability testing with 5 HCMs and 7 PMs, 11 out of 12 participants co-created the prototypes (mean SUS score 86 [SD 9.3]).

conclusionsHCMs and PMs co-created prototypes for systematic review of effects formats based on their needs. The prototypes will be compared to a traditional format in a randomized trial.

Indexed as

Administrative PersonnelDecision MakingEvidence-Based MedicinePolicy MakingSystematic Reviews as TopicCanadaHumansDecision makingEvidence implementationHealth care managersIntegrated knowledge translationKnowledge synthesisKnowledge translationPolicy makersSystematic reviewsUsability

Identifiers

PMID29929538
PMCPMC6014014

What Socratic holds

Textmetadata
LicenceCC BY
Read underepoch 390

Registered trials

None linked

Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the Socratic graph.