This was the third in a three-part series. To read the other posts, click here:
Last month we wrote a post with the hopes of opening discussion about the FDA’s new “Breakthrough Therapy” designation, and started by looking at how one of the previous FDA fast lanes “Priority” compared to their “Standard” lanes and also to the European Medicines Agency (EMA). In that analysis, we compared FDA data from 2008 and 2011 on “Priority” and “Standard” lanes and also to the EMA data from 2012. Our analysis revealed that over time the FDA’s “Priority” lane actually slowed down.
That prompted some excellent dialogue, with many of the responses focused on whether a few extreme values – “outliers” – were driving our conclusions. Based on that discussion, we decided to dig a little deeper, in a follow-up that focused exclusively on “Priority” reviews.
We found that the presence of a few very long approval times had an appreciable affect on the mean approval time in each year, but these “outliers” did not entirely account for the results. Whether we recalculated the means after removing the outliers or compared median times (which are less sensitive to the presence of extreme values), we found that there was still a lengthening of “Priority” review times between the two years.
For this post, we will focus exclusively on “Standard” reviews.
What’s in a “Cycle Time”?
One important bit of context is that we focus on a different measure of time than the FDA does. The FDA’s “cycle time” is the time it takes to respond to the applicant after any individual submission. While this is a good measure of one aspect of the FDA’s performance, we are actually more interested in the length of the whole approval process as this is the time span that is most pertinent to anyone developing drugs. For instance, if a drug goes through multiple cycles (which sometimes happens) the whole process will take the multiple of any single cycle time. For that reason, our last two posts reported figures for the total time from initial submission to approval. We will continue to focus on this more comprehensive measure of “cycle time” in this blog.
Setting the Standard
The two charts below show how long every standard approval took in 2008 and 2011. Both charts order drugs by total time to approval. Those with the shortest times are on the left; those with the longest times are on the right.
The Envelope Please…
Outliers are not an issue. In both 2008 and 2011 almost half of the approved reviews were approved within a year. However, in both years a substantial proportion of reviews took significantly longer than a year to approve. Within this set, there are no obvious “outliers,” just a range of approval times, some fairly long.
- Between the two years average (mean) approval time dropped from 18.4 to 16.6 months
- Median time – the middle time in the range of approval times – declined only slightly, from 13.0 to 12.6
What Does This Tell Us?
Very little actually. Neither the mean nor the median highlight what is most striking in comparing the two years: Between 2008 and 2011 the longest approval times got significantly shorter.
- In 2008, nearly a quarter (23%) of reviews took two or more years to receive approval, and 11% took three or more years
- Three years later, in 2011, a slightly smaller proportion of reviews took two or more years (20%) but only 4% of reviews took three or more years to receive approval
- In 2008 the longest time to approval was just over five years; in 2011 the longest time was just over four years
The good news is, the worst got significantly better, and that’s real improvement.
And it is instructive. In different ways, both the mean and median summarize the range of approval times in a single number. They are different ways of expressing the overall time it took for drugs to be approved. Often the mean and median are helpful, informative simplifications, but here, neither direct attention to the most important part of the story.
The intention of these last few blog posts was to better understand the total time required for a drug to gain regulatory approval. In light of breakthrough designation, the question on our minds was whether this produced a measurable impact on expected time to approval. We looked at current and past performances of the standard and accelerated paths and found a more complicated story than expected.
This analysis confirmed one of the cornerstones that Context Matters was built on. Drug development is complex. Sometimes a summary misses something critical. Sometimes even a great synthesis isn’t good enough. And sometimes, there is no substitute for knowing the data.