Background: There have been several explorations of factors influencing the reimbursement decisions of the National Institute for Health and Clinical Excellence (NICE) but not of other UK-based health technology assessment (HTA) organizations.
Objective: This study aimed to explore the factors influencing the recommendations of the All Wales Medicines Strategy Group (AWMSG) on the use of new medicines in Wales.
Methods: Based on public data, logistic regression models were developed to evaluate the influence of cost effectiveness, the quality and quantity of clinical evidence, disease characteristics (including rarity), budget impact, and a range of other factors on the recommendations of AWMSG and its subcommittee, the New Medicines Group (NMG).
Results: Multivariate analyses of 47 AWMSG appraisals between 2007-9 correctly predicted 87% of decisions. The results are suggestive of a positive influence on recommendations of the presence of probabilistic sensitivity analyses (PSAs) but, counter-intuitively, a statistically significant negative influence of evidence from high-quality randomized controlled trials (RCTs) [odds ratio 0.059; 95% CI 0.005, 0.699]. This latter observation may be attributed to our strict definition of high quality, which excluded the use of surrogate endpoints. Putative explanatory variables, including cost effectiveness, budget impact, underlying disease characteristics and 'ultra'-orphan drug status were not statistically significant predictors of final AWMSG decisions based on our dataset. Univariate analyses indicate that medicines with negative recommendations had significantly higher incremental cost-effectiveness ratios than those with positive recommendations, consistent with the pursuit of economic efficiency. There is also evidence that AWMSG considers equity issues via an ultra-orphan drugs policy.
Conclusions: Consideration of decision uncertainty via PSA appears to positively influence the reimbursement decisions of AWMSG. The significant negative impact of the presence of high-quality RCTs, and the lack of a significant positive impact of other expected factors, may reflect issues in the plausibility of supporting evidence for medicines that received negative recommendations. Furthermore, it serves to emphasize the difficulties in applying the usual hierarchies of evidence to the HTA process, and in particular to the appraisal of high-cost specialist medicines close to market launch.