The Potential Cost and Implications of Transparency Without Context in Clinical Trial Data

Data transparency continues to be a topic of interest that has been generating a lot of news lately. The European Federation of Pharmaceutical Industries (EFPIA) and the Pharmaceutical Research and Manufacturers of America (PhRMA) have recently pledged to help make clinical trial data more readily available to researchers. Earlier this year, the European Medicines Agency (EMA) proposed a policy that placed fewer restrictions on who can access the data. The idea behind increasing data transparency is admirable, with its promise of making more useful information available to more people. However, increasing transparency without context can have unintended consequences.

To that end, under the PhRMA/EFPIA pledge “Principles for Responsible Clinical Trial Data Sharing,” disclosure is limited to "qualified scientific and medical researchers" upon request with the intention to avoid releasing non-contextualized raw data. In fact, researchers agree, largely because they too worry that non-contextualized data may simply confuse any subsequent analysis.

Our founder and CEO, Yin Ho, was recently asked to provide her perspective for an article in Specialty Pharmacy Times:

“Without the proper context, transparency on its own is meaningless. In the case of patient-level data, that would be like clearly seeing the individual pixels in a high-resolution picture without knowing the relationship of the pixels to each other. Only when you step back and see how the pixels relate to each other do you gain a full understanding of what the picture is.”

In this post we hope to expand on her quote and also bring up a few issues that may not be getting enough discussion or consideration.

More is Not Always Better

To begin with, transparency isn’t free and achieving it isn’t as easy as simply providing access to raw data. Making data accessible and insightful usually requires connecting disparate sources of information. Maintaining integrity and assuring both quality and consistency is not a cheap or easy process.

We are in the business of providing useful information. To focus on just one piece of our process, when putting data in our platform, we are constantly navigating things like differences in terminology across countries, differentiating foreign currencies, and standardizing the policies of various rule-setting bodies (eg., PBAC vs. NICE). Among other things, this requires having an intimate knowledge of how regulatory and reimbursement agencies across the globe function.

Here’s an example: the multiple myeloma drug Kyprolis (carfilzomib) meets the European Medicines Agency's (EMA) criteria for orphan drug designation, but did not receive the same orphan drug status through the U.S. Food and Drug Administration (FDA). And to make things even more confusing, in the U.S. the brand name is Kyprolis, but it doesn't have a brand name yet in the EU and is only referenced by its generic name, carfilzomib. And… the EU often assigns a different brand name.

So how much transparency would be achieved by simply making these raw facts available? Without a fair amount of detailed information and the knowledge of how to combine them (that is without context) releasing this raw data “into the wild” would almost certainly muddy the waters for many people.

Signal or Noise?

While in principle more access to information is good, we must ask ourselves what kind of additional information will be provided? Will the new data be a useful supplement or will it simply serve to increase "noise"?

If it is “noise,” for example inconclusive results, results from poorly designed studies, or other “chaff,” regulatory agencies and others might actually have a harder time wading through the irrelevant information to make their evaluations. Without the proper context, having more information could actually lead to greater confusion or information overload.

Implications for Trial Design

In addition to the cost and effort, we wonder how pharmaceutical companies will adapt to the request for more clinical trial data. If companies are made to publish every study they fund, the nature of the data is likely to change. They may choose to fund less exploratory analyses that risks going down blind alleys in order to bypass the significant added investment.

Obviously this is a complex subject and there are good arguments on both sides. We wanted to leave you with a few questions to help spur more discussion.

  • What problem is additional transparency going to solve – what do we hope to accomplish?
  • Will companies limit the scope of their studies?
  • Will it change the types of drugs they develop?

Let the discussion begin!