Assessment of the Power of ITCs to Detect Minimally Important Treatment Effects in Subsequent Head-to-Head Trials

Author(s)

Disher T
EVERSANA, West Porters Lake, NS, Canada

OBJECTIVES: NMAs and other anchored ITCs leverage randomization via comparison of relative effects compared to placebo. In most cases, trials included in ITCs are powered to detect large differences compared to placebo. Differences between any two active therapy are expected to be smaller, and thus this combined with the increased variance of treatment differences estimated via an anchor may mean that ITCs are drastically under-powered. We use a recent Cochrane NMA in Psoriasis to explore the power of ITCs compared to the same comparisons in subsequent head to head trials.

METHODS: We use extracted pairwise comparisons and confidence intervals from a recent Cochran review in Psoriasis and compare the estimate of the standard error of a given pairwise comparison to the hypothesized effect size used for planning subsequent head to head studies. We use this data to conduct a design analysis to assess the power of comparison from the NMA, and describe how much statistically significant results can be expected to have exaggerated magnitude (type-M error) or increased probability of being significant in the wrong direction (type-S error).

RESULTS: ITCs from NMAs are extremely under-powered compared to the hypothesized effect size in subsequent head to head randomized controlled trials. The result is that effect estimates from statistically significant comparisons in NMAs excluding the head to head trial can be inflated by as much as five times, and results that are statistically significant have a greater than 5% chance of being in the wrong direction.

CONCLUSIONS: ITCs are likely extremely under powered which increases the risk that filtering results (by emphasizing those that are significant) could lead to important public health consequences via upwardly biased effect estimates that have higher than nominal probability of being in the wrong direction.

Conference/Value in Health Info

2023-05, ISPOR 2023, Boston, MA, USA

Value in Health, Volume 26, Issue 6, S2 (June 2023)

Code

MSR107

Topic

Methodological & Statistical Research, Organizational Practices, Study Approaches

Topic Subcategory

Artificial Intelligence, Machine Learning, Predictive Analytics, Best Research Practices, Clinical Trials

Disease

No Additional Disease & Conditions/Specialized Treatment Areas

Explore Related HEOR by Topic


Your browser is out-of-date

ISPOR recommends that you update your browser for more security, speed and the best experience on ispor.org. Update my browser now

×