Predictive modeling can be a powerful tool for understanding how multiple factors contribute to an event. As outlined in previous posts, Context Matters created a model that used variables from oncology assessments by the Scottish Medicines Consortium (SMC) to identify the most influential variables for a positive reimbursement decision by that agency and used the model to demonstrate the impact that economics (i.e., patient access schemes and ICERs) have in predicting positive SMC decisions for oncology drugs. However, as discussed in our previous post, Understanding Predictive Modeling, not all models are created equal. One measure of a predictive model’s quality is its ability to deliver actionable results and insights. If a model is not able to inform the initial hypothesis and provide a path forward, it is just an academic exercise. But what does it mean for a predictive model to be “useful” and to deliver “actionable results”? What are some of the implications and use cases for predictive models of Health Technology Assessment (HTA) agency decisions?
A predictive model can identify what aspects of a drug are most important to decision-making. Currently, HTA decision-making is opaque. The protocols and submission guidelines of HTA agencies do not always clearly lay out what aspects of a drug they care about most, thus the weight that each clinical and economic component (e.g., the use of active comparator, efficacy gains on the primary endpoint, and the ICER) has on the overall decision is unknown. A predictive model of HTA decision-making can identify what clinical and economic variables are significantly related to positive reimbursement decisions. With a predictive model, it is no longer a guessing game as to what variables matter. This will allow market access teams to highlight within the HTA dossiers the value components and supporting evidence of their drug that are most influential to the agency. The model can also provide benchmarks for drug development teams when they are designing a drug’s clinical trials.
It is often unclear how different HTA agencies compare to each other in terms of what components matter most to them and how these factors contribute to decision-making. For example, HTA agencies often disagree on their reimbursement decisions even when they evaluate the same drug for the same indication using the same evidence base. This creates challenges for manufacturers who are trying to optimize reimbursement and only have a limited amount of clinical and economic evidence at the time of HTA submission. A predictive model can allow for an easy comparison between HTA agencies on the influential components of decisions. For example, SMC might place higher relative weight on improvements in quality of life than NICE. These comparisons can help manufacturers design their clinical trials in order to incorporate the aspects that objectively measure what is important and relevant to the different HTA agencies.
Different disease conditions require different considerations and are often treated differently by the HTA agencies. For example, in our recent webinar, Is Your Oncology Drug HTA-ready?, we presented results that showed that oncology drugs were more likely to be issued negative reimbursement decisions compared to non-oncology drugs. Again, HTA agencies are not forthcoming regarding what aspects matter most to them and how this might differ by disease. A predictive model of HTA decisions can control for disease conditions and identify the clinical and economic components that carry more weight in positive decisions. For example, NICE might be more willing to accept an absence of improvement in quality of life measures for an oncology drug, but not willing to do so for a diabetes drug. Once again, this knowledge will allow manufacturers to better tailor their dossier submissions and clinical trials to the demands of HTA agencies.
Within the HTA agencies, there are often multiple groups or committees that issue the reimbursement recommendations. While these groups follow the same general decision-making principles, it is worth asking if these separate committees are consistent in their decision-making. For example, we evaluated NICE’s reimbursement decisions in relation to which evidence review group (ERG) did the initial technology appraisal. Differences in rates of positive decisions by ERG were identified in this study, which suggested that some ERG evaluations are more likely to lead to positive decisions. There are many factors that contribute to a reimbursement decisions and could explain this association. However, a predictive model can control isolate a single variable (e.g., ERG) to determine if it is related to a specific decision. These types of models can serve as status checks on a specific HTA agency’s evaluation process and determine if an agency’s decision-making is consistent and aligned with their country’s values. A model of HTA decisions can potentially identify areas where the HTA agency may need to change its processes to ensure fairness in decision-making.
These days, regulatory approval does not guarantee reimbursement. The return on investment for a drug can be significantly affected by negative reimbursement decisions across the globe. In order for manufacturers to set internal expectations on a drug’s success, make more informed decisions on what assets to continue to develop, and make informed decisions on what assets to acquire, they need to assign a probability to how well a drug will fare through the reimbursement process. A predictive model of HTA decisions can take the guess work out of these high stakes decisions and provide a manufacturer with a robust and reliable probability of reimbursement given a drug’s clinical and economic profile. For example, if drug X improves overall survival by 3 months, does not impact quality of life, and costs £20,000 a year, we could use a model to determine the probability that the drug would be reimbursed by SMC. We could also use a model to determine how much the probability of a positive decision would change if a component was altered (e.g., a cost of £30,000 per year or overall survival of 2.5 months), which can aid HEOR teams in power studies to meet specific benchmarks.
As you can see, there are many significant benefits to developing a predictive model of HTA decisions. A model of HTA decisions can offer a significant improvement in transparency which benefits both manufacturers and HTA agencies. However, in order to create a robust and useful model, these use cases and implications must be defined at the beginning and at the forefront of the researcher’s mind, when executing the analysis. These use cases are the goals for our predictive model analysis as we continue to build a robust model of SMC decision-making and each additional agency that we currently track.