The importance of methodological rigor and clinical expertise for actionable insights

Life sciences companies invest millions of dollars in developing novel treatments such as GLP-1-RAs and the expansion of indications to improve patient outcomes and quality of life. While positive press can be beneficial, as additional studies shine light on benefits, risks, and overall safety, if the research is not rigorous, then the pendulum can quickly swing, undermining the credibility of a treatment and a company.

Payers and clinicians alike are discerning and often mistrusting of “headline hype,” and may be quick to write off findings as “self-promotional marketing.” Methodological rigor and clinical expertise applied to the research of new treatments and indications are key to standing up to stakeholder scrutiny. By investing in and applying methodological frameworks and contextual knowledge, life sciences companies are more likely to reap long-term gains with more actionable insights and greater clinical applicability while building stakeholder trust.

Case study: Methodological rigor and clinical expertise in GLP-1-RA research

A recent Carelon Research study illustrates the relevant and long-term value of this approach. The Safety and Epidemiology team recently completed a study examining serious clinical outcomes during GLP-1-RA use for weight management. The study protocol was pre-registered with the Open Science Framework, and the study was designed to mitigate potential biases through use of an active comparator, new-user design with propensity score weighting for 82 clinical and demographic covariates. Importantly, the study was designed with input from clinicians whose insights provided a foundation for study design decisions such as which drugs to select as comparators and which covariates to include in the analysis.

The study used data from the Healthcare Integrated Research Database (HIRD®) which is an integrated U.S. healthcare database, including claims representing more than 90 million people. An important methodological challenge with U.S. claims databases is that people enter and exit the database over time. To address this, researchers often measure clinical outcomes as the number of cases that occurred during the time that patients were members of the health plan during the entire study period. But this approach of calculating an average incidence rate for the study population can mask important information that might be of interest to stakeholders, such as an incidence rate that varies over time.

To uncover results that might be masked by certain measures, such as relative risk, the study team also estimated absolute effect measures, such as risk differences and number needed to treat (NNT) using time to event (e.g., Kaplan-Meier) analyses. This approach is recommended to estimate NNT in situations where study participants are followed for different durations and offers additional clarity to clinicians and policymakers.

In another example of methodologic rigor and clinical insight, the study team found important differences between treatment groups as reflected in non-overlapping propensity score distributions. The team reasoned that as evidence emerged of the benefits of GLP-1-RAs, patients with diabetes or prediabetes were increasingly channeled to GLP-1-RAs in the weight management population. Going forward, the study team excluded patients with diabetes or prediabetes from the weight management analysis and restricted analyses to a more recent period during which users of GLP-1-RAs could be compared to similar users of other weight management therapies. The reported study findings represent a carefully planned analysis conducted by a study team who were observant and agile enough to recognize when an observational study plan needed to be modified to produce informative results.

Lessons for sustainable and actionable results from RWE studies of GLP-1-RAs (or other drugs)

1. Include clinicians with relevant experience in study design and analysis. It is easy to overlook the importance of clinical insight for analytic implementation, but real-world data are not designed like clinical trial data, and clinical insight can help understand unexpected findings and how to meaningfully address them. 
2. Examine underlying assumptions when conducting analyses so that anomalies can be detected and a change in course can be implemented. Propensity score methods are a useful approach for the transparency they provide, including the ability to compare propensity score distributions and examine covariate balance between treatment groups. 
3. Take the time to be thorough in contemplating intermediate results. For instance, even if covariate balance is achieved, propensity score distributions may still suggest important differences between populations that could bias results. . 

The need for expertise and rigorous methods

With large datasets increasingly available that can produce speedy analyses of very large populations, it is important to avoid scientific shortcuts. Scientists using such data should have specialized training and experience in pharmacoepidemiology studies with large databases and a demonstrated track record of producing accurate and informative analyses.

Investing in this level of expertise and the time needed to conduct analyses with care is more likely to produce results that stand up to scrutiny and provide actionable insights that stand the test of time. Long-term credibility is more sustainable than short-term “headline” gains. Through robust methodological practices and clinical collaboration, sustainable success is not only achievable, but likely.

See our latest work presented at ISPE 2025 in Washington, D.C.

Tuesday, August 26, 2025
9:15 AM - 9:30 AM ET
5E-04 - Comparative safety of glucagon-like peptide 1 receptor agonists (GLP-1-RAs) in chronic weight management patients without diabetes: a real-world data study 

Tuesday, August 26, 2025
1:30 PM - 1:45 PM ET
6E-01 - Calculating number needed to treat (NNT) for studies using real world data with variable follow-up times: Kaplan-Meier versus rate-based approach