Metrics are essential to determine the performance of a clinical trial. However, defining what those metrics are, how to select them and why they’re valuable is not a straightforward process.
is more easily collected and aggregated. Having standardized metrics makes it easier for all clinical trial stakeholders to understand why these measurements are being used. However, as we will see later on, standardization of metrics can sometimes result in missed KPIs and KRIs are specific to a particular trial.
In this post, we explore the importance of getting the selection of clinical trial metrics correct and how best to ensure that all stakeholders understand what they mean and why they’re being used.
Know What You Want to Answer
Before study staff can decide on which metrics matter, they need to know what answers they want from the research. This strategy helps determine the most important metrics, explains April Schultz at Forte Research. The next step is to base further and more specific metrics on those fundamental metrics. Second-tier metrics include measuring performance and processes.
Schultz says stakeholders should focus on several important factors to determine what metrics to measure. One focus area is enrollment rates, contrasting participant accrual with target enrollment as well as the number of days since the last patient enrolled.
Another is looking at the study’s start-up phase and how it’s progressing towards activation. Pay attention to the time between contract receipt and execution turnaround, as well as the time between submitting to the IRB and being granted approval.
The representativeness of the study sample is also important. Look at patient demographics and diversity to ensure you’re meeting requirements.
Key Metrics to Focus On
Recruiting patients to trials is notoriously challenging, as is retaining them until study completion: 80 percent of studies do not reach their recruitment targets, and half of trials recruit only one patient or none, writes clinical researcher Robert Dannfeld.
One of the most significant metrics to focus on, then, is patient recruitment. Researchers at BMC, stress that patient retention to study completion, data quality and compliance with protocol compliance should also be added to this list.
Developing Study Metrics
There are a few common pitfalls when it comes to developing clinical trial metrics. Stakeholders can sometimes select too many metrics that are often overly complex, which can be time-consuming to manage, explains Ann McGee at PharmaLex Ireland.
Other mistakes include pursuing metrics so doggedly that wrong behaviors creep in such as taking actions without context to validate the metrics. Researchers should rather focus on factors such as operational quality performance, indicators that show compliance with regulations, and the differences between quantitative and qualitative data, she writes.
Choosing the right metrics and defining them appropriately requires careful planning. Metrics should be defined at the beginning of the study and tracked throughout.
Using Analogy To Discuss Metrics
Analogy is an effective means of explaining complex topics. So when it comes to convincing stakeholders about which metrics to use, try talking about the weather. This is what Alex Hsieh, director of predictive analytics at Pfizer, did to explain the organization’s implementation of machine learning and AI technology to predict quality risk and events.
As with weather forecasting, Hsieh says predicting clinical trial risks requires breaking risk into certain categories using existing data, and then elevating or downplaying that risk with any new data that arrives. Inputting new data will affect changes in the predictive model.
Measuring Tumors and Performance
Knowing the size of cancer patients’ tumors is critical to an oncology study’s success, of course, but researchers and study stakeholders need additional metrics too. Measuring study performance and quality is essential, argues James Golando, chief of operations at Median Technologies.
The best way for pharma organizations to measure these metrics is to adopt standardized measurements of operational time, cost and quality. Essential to any successful metrics is the need for agreed definitions of terms, study milestones and performance targets. Standardized metrics help improve planning and predicting a study’s performance.
The need for standardized metrics, of which all stakeholders should be aware, is best underscored by the pursuit of validity and reproducibility of results. In the case of imaging trials, specifically, it’s vital to equip staff with devices calibrated to standard specifications, as well as strategies for gathering images and following protocol.
The key performance metrics in an imaging trial are cycle times, timeliness, quality and efficiency, Golando explains.
Standardization Helps But Is Not Sufficient
A recent survey shows that 83 percent of biopharmaceutical companies use standardized performance metrics for clinical operations and data management. Nearly 70 percent use metrics for quality management, explains business strategist Moe Alsumidaie.
He points to the Metrics Champion Consortium as an example, which has developed its analytics into therapeutic-specific study metrics. These standardized measurements present huge potential benefits to the whole industry. Most notable are clinical trial oversight, risk management and performance.
But Alsimidaie urges caution. In the pursuit of standardization, study-specific KPIs/KRIs might be neglected, leading to the omission of significant study issues such as therapeutic indication, study design, drug or device, and study phase.
Outcome-Based and Goal-Based Metrics
Measuring whether a trial achieves its desired strategic objectives requires that outcome-based metrics are used. But because they are often broad with many different stakeholders contributing to a study’s outcomes, metrics can be difficult to decide upon, according to Envision Pharma Group.
Goal-based metrics, on the other hand, concern specific activities and initiatives that help to bring about the desired outcome. These goal-based metrics should be linked with a complementary outcome-based metric.
The authors of the Envision white paper say that outcome-based and goal-based metrics are related but distinct. They should be considered simultaneously and connect to the study’s objectives. Embedded as sub-categories of both outcome-based and goal-based metrics are quantitative and qualitative metrics. However, outcome-based metrics tend to be qualitative, and goal-based metrics quantitative.
The Value of Site-Based Metrics
Multiple stakeholders can benefit from using site-based metrics. Milestones can be set and progress measured based on performance data from past trials, explains Kristina Lopienski at Forte Research. Similarly, cycle times and progress from previous trials allow for a more accurate prediction of timelines of current trials.
But it’s not just sponsors that benefit from site metrics. Sites themselves can rely on this data to improve their processes. Lopienski says that information helps sites choose trials for which they will be able to deliver and meet sponsors’ needs.
Data from robust metrics also tend to give more weight to requests that sites make to sponsors and other stakeholders.
Metrics that should be focused on include past site performance measured against industry averages, volumetrics (number of trials conducted, cycle-times), and the historic speed to randomize the first study participant.
Relevance Depends on Stakeholders’ Perceptions
Quantifiable outcomes of investigational treatments that are easy to interpret and clinically relevant are fundamental conditions to meet when bringing new therapies to market.
However, what is considered relevant depends on which stakeholder you ask, says molecular geneticist Claudia Dall’Osso and Ian Love, a business insights analyst at Decision Resources Group. Developers and patients may well differ in this regard.
The key is defining clinical endpoints. Developers have traditionally chosen endpoints that are not always relevant to patients. This is changing, of course, with a new focus on patient-centric study designs. As a result, patients’ perspectives are being included drug development.
These patient-centric outcomes are clinical metrics that are relevant to patients and caregivers. This requires understanding the patient journey, needs and pain points. Determining these relevant endpoints, particularly in the treatment of rare diseases, requires drug manufacturers to first identify specific disease manifestations that negatively affect patients.
Next is to determine what amount of change to these diseases patients consider as meaningful. Collaboration with patients and patient advocacy groups, regulatory agencies, payers, and physicians will help to determine the relevant metrics to focus on, write Dall’Osso and Love.
What is most important to a patient may not be the same for sponsors, CROs or researchers when it comes to measuring the performance and success of a clinical trial. However, taking patient-centric approaches to research better aligns the desired outcomes of multiple stakeholders. Choosing the right metrics and ensuring their relevance and meaning are explained will empower all study stakeholders.
Images by: ammentorp/©123RF Stock Photo, wavebreakmediamicro/©123RF Stock Photo, dolgachov/©123RF Stock Photo