Are We Getting the Full Story When It Comes to Surrogate Outcomes in Health Technology Assessments?

We spent this past week in Dublin at the 16th Annual ISPOR European Congress and presented research on six topics. If you’re interested, you can click here to view electronic versions of all six poster presentations. The conference was a great experience for our team; we were able to share knowledge, catch up with our customers, and get a good sense of the types of issues and questions our peers are looking to solve. Over the next few weeks our blog will focus on the research we presented, and we encourage everyone to provide any thoughts or feedback you have on our findings.

This post focuses on our study of surrogate outcomes in health technology assessments (HTAs). The underlying evidence used to support HTA decisions is based on clinical trials conducted by other researchers.

What Is a Surrogate Outcome?

A surrogate outcome is a biomarker intended to substitute for a clinical end point. Surrogate outcomes are generally used when a clinical end point of interest is too difficult to analyze or does not occur often enough to perform meaningful statistical analysis. We believe the agencies may not be reporting the full picture in these reviews.

How to Use Surrogate Outcomes

Before using surrogate outcomes, researchers should confirm that there is a strong correlation with the clinical endpoint. They should also confirm that the surrogate outcome is biologically plausible and reflects changes in the relevant clinical endpoint. Finally, it is important to justify using a particular surrogate outcome as part of providing clinical trial results.

Simply put, if you are using a surrogate outcome in your trial… you are supposed to report it and document that it is appropriate. We looked at how often HTAs followed this practice when they described the underlying research upon which their decisions were based.

A Disconnect?

Our study looked at 1,586 HTAs spanning 31 disease conditions. Each HTA had a primary outcome reported in the review that could be classified as either a surrogate outcome or a clinical endpoint. We analyzed reviews from 13 HTA agencies: AHRQ, CADTH, CCO, CONITEC, DERP, HAS, HIRA, HIS, IQWIG, NICE, PBAC, pCODR and SMC. For HTAs using a surrogate outcome, we looked to see whether the agency reported the outcome as a surrogate and if the validity of the surrogate was discussed.

We found that 73% of HTAs relied on studies that used a surrogate outcome as the primary outcome (1,158 HTAs), but only 11% (130 HTAs) identified it as a surrogate outcome and discussed its validity (only in a handful of cases identified the outcome as a surrogate without discussing its validity). For oncology, 52% of the reviews used a surrogate outcome as the primary outcome, but only 9% discussed its validity. Outside of oncology, 80% used a surrogate outcome as the primary outcome, but only 12% discussed its validity.

These numbers are a bit surprising. Why are HTAs so rarely explicitly identifying surrogate outcomes and discussing their validity?

HTA reviews are based on clinical trials conducted by other researchers. We can assume that most or all of the underlying studies are justifying their use of surrogate outcomes, but our figures show that HTAs are noting surrogate use only about 10% of the time.

From the Source

We are unsure why HTAs rarely mention or discuss the use of surrogate outcomes, which of course leads us to some interesting speculation. Can we assume that this is because the use of surrogate outcomes doesn't in any way factor into the ultimate reimbursement decision? Or is this because surrogate outcomes are not viewed as sufficiently strong enough clinical evidence to warrant any real discussion? Or because the use of surrogate outcomes is so unremarkable that it goes unremarked? Or is this simply an oversight that HTA agencies routinely neglect to do?