Predictive modeling can be a powerful tool for understanding how multiple factors contribute to an event. As outlined in previous posts, Context Matters created a model that used variables from oncology assessments by the Scottish Medicines Consortium (SMC) to identify the most influential variables for a positive reimbursement decision by that agency and used the model to demonstrate the impact that economics (i.e., patient access schemes and ICERs) have in predicting positive SMC decisions for oncology drugs. However, as discussed in our previous post, Understanding Predictive Modeling, not all models are created equal. One measure of a predictive model’s quality is its ability to deliver actionable results and insights. If a model is not able to inform the initial hypothesis and provide a path forward, it is just an academic exercise. But what does it mean for a predictive model to be “useful” and to deliver “actionable results”? What are some of the implications and use cases for predictive models of Health Technology Assessment (HTA) agency decisions?
In predictive modeling, it is essential to evaluate how good the model fits the underlying data. The closer the model is to fitting the underlying data, the stronger the model. We recently built a model to predict Scottish Medicines Consortium (SMC) reimbursement decisions for oncology health technology assessments (HTAs). Our binary model, that is the model that attempted to predict a positive versus negative SMC reimbursement decision, was able to correctly predict 75% of the oncology decisions in the data set. By statistical standards, 75% is considered a “good” level of prediction. But what happened in the other 25% of assessments? Examining where the model “got it wrong” is an important tool in understanding the data and knowing how to apply the learnings and insights.
With the rise of “big data,” predictive analytics has become increasingly popular. Predictive modeling, the process of using data to predict an unknown event, allows for predictions to be made based on multiple complex factors. Predictive modeling has been touted as the key that unlocks performance increases in everything from customer service, crime prevention, financial services, to population health. Within the pharmaceutical industry, “big data” shows promise for improving the drug development process (e.g., identification of genetic targets and the development of immunotherapies) and demystifying reimbursement decision-making.