Evidence map›Paper›PMID 33786991›Full record

ArticleClinical and translational science2021

Associations between uncertainties identified by the European Medicines Agency and national decision making on reimbursement by HTA agencies.

Lourens T Bloem, Rick A Vreman, Niels W L Peeters, Jarno Hoekman, Menno E van der Elst, Hubert G M Leufkens, Olaf H Klungel, Wim G Goettsch, Aukje K Mantel-Teeuwisse

Abstract read
In one paragraph

Article in Clinical and translational science, 2021. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 9 papers.

0numbers the graph read from it
0cells of the map it votes in
9citing papers in PubMed
–field-weighted citation impact
1 · What the graph read from it

What it found

Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.

The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.

2 · The registry

The trial behind it

Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.

Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.

3 · Its place in the literature

Who cites it

9 citing papers in PubMed.

  1. Article
  2. Regulatory and HTA framework for herbal medicines in Indonesia.Journal of pharmaceutical policy and practice · 2026
    Review
  3. Article
  4. Article
  5. Article
  6. Article
  7. Review
  8. Article
  9. Article
4 · The record

Corrections and comments

PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.

5 · Who and what money

Authors and funding

9 authors.

Lourens T BloemDivision of Pharmacoepidemiology and Clinical Pharmacology, Utrecht Institute for Pharmaceutical Sciences, Utrecht University, Utrecht, The Netherlands.
Rick A VremanDivision of Pharmacoepidemiology and Clinical Pharmacology, Utrecht Institute for Pharmaceutical Sciences, Utrecht University, Utrecht, The Netherlands.
Niels W L PeetersDivision of Pharmacoepidemiology and Clinical Pharmacology, Utrecht Institute for Pharmaceutical Sciences, Utrecht University, Utrecht, The Netherlands.
Jarno HoekmanDivision of Pharmacoepidemiology and Clinical Pharmacology, Utrecht Institute for Pharmaceutical Sciences, Utrecht University, Utrecht, The Netherlands.
Menno E van der ElstDutch Medicines Evaluation Board, Utrecht, The Netherlands.
Hubert G M LeufkensDivision of Pharmacoepidemiology and Clinical Pharmacology, Utrecht Institute for Pharmaceutical Sciences, Utrecht University, Utrecht, The Netherlands.
Olaf H KlungelDivision of Pharmacoepidemiology and Clinical Pharmacology, Utrecht Institute for Pharmaceutical Sciences, Utrecht University, Utrecht, The Netherlands.
Wim G GoettschNational Healthcare Institute, Diemen, The Netherlands.
Aukje K Mantel-TeeuwisseDivision of Pharmacoepidemiology and Clinical Pharmacology, Utrecht Institute for Pharmaceutical Sciences, Utrecht University, Utrecht, The Netherlands.

Funding

No grant is acknowledged in the PubMed record.

6 · The paper itself

Abstract

We aimed to determine whether uncertainties identified by the European Medicines Agency (EMA) were associated with negative relative effectiveness assessments (REAs) and negative overall reimbursement recommendations by national health technology assessment (HTA) agencies. Therefore, we identified all HTA reports from Haute Autorité de Santé (HAS; France), National Institute for Health and Care Excellence (NICE; England/Wales), Scottish Medicine Consortium (SMC; Scotland), and Zorginstituut Nederland (ZIN; The Netherlands) for a cohort of innovative medicines that the EMA had approved in 2009 to 2010 (excluding vaccines). Uncertainty regarding pivotal trial methodology, clinical outcomes, and their clinical relevance were combined to reflect a low, medium, or high level of uncertainty. We assessed associations by calculating risk ratios (RRs) and 95% confidence intervals (CIs), and agreement between REA and overall reimbursement recommendation outcomes. We identified 36 medicines for which 121 reimbursement recommendations had been issued by the HTA agencies between September 2009 and July 2018. High versus low uncertainty was associated with an increased risk for negative REAs and negative overall reimbursement recommendations: RRs 1.9 (95% CI 0.9-3.9) and 1.6 (95% CI 0.7-3.5), respectively, which was supported by further sensitivity analyses. We identified a lack of agreement between 33 (27%) REA and overall reimbursement recommendation outcomes, which were mostly restricted recommendations that followed on negative REAs in case of low or medium uncertainty. In conclusion, high uncertainty identified by the EMA was associated with negative REAs and negative overall reimbursement recommendations. To reduce uncertainty and ultimately facilitate efficient patient access, regulators, HTA agencies, and other stakeholders should discuss how uncertainties should be weighed and addressed early in the drug life cycle of innovative treatments.

Indexed as

Decision Making, OrganizationalDrugs, InvestigationalClinical Trials as TopicCost-Benefit AnalysisDrug CostsEuropean UnionHealth PolicyHumansInventionsReimbursement MechanismsRetrospective StudiesTechnology Assessment, BiomedicalUncertaintyDrugs, Investigational

Identifiers

PMID33786991
PMCPMC8301545

What Socratic holds

Textmetadata
LicenceCC BY-NC
Read underepoch 390

Registered trials

None linked

Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the Socratic graph.