OBJECTIVES: Real-world evidence is playing an increasingly important role in health technology assessment, but is prone to selection and confounding bias. We demonstrate how to conduct a real-world within-study cost-per-quality-adjusted-life-year (QALY) analysis. We combined traditional within-trial bootstrapped regression-baseline-adjustment with causal inference methods, using a Target Trial (TT) framework, inverse probability weights (IPWs), marginal structural models (MSMs), and g-computation, applied to England’s Improving Access to Psychological Therapies (IAPT) mental-health e-records.

METHODS: The ‘Assessing a Distinct IAPT service’ (ADAPT) quasi-experimental-study evaluated an Enhanced-IAPT service Vs. IAPT’s treatment-as-usual. IAPT collects patient-reported PHQ-9-depression and GAD-7-anxiety scores at index-assessment and each treatment session, from which we predicted EQ-5D utilities using a mapping function. Our primary estimands were incremental costs and QALYs for Enhanced Vs. treatment-as-usual at 16-weeks post-IAPT-index-assessment.

We prespecified our TT including eligibility, treatment strategies, assignment procedure, follow-up, outcomes, estimands, and analysis plan. We used stabilised treatment-related and censoring-related IPWs within MSMs to reduce selection and confounding bias due to non-randomised treatment allocation and informative censoring, respectively. Our doubly-robust approach involved MSM-adjusted baseline covariates and g-computation to estimate incremental utilities, costs, and QALYs, with bootstrapped bias-corrected 95% confidence-intervals (95%bCIs) and cost-effectiveness acceptability curves.

RESULTS: Analysis sample: Enhanced, N=5,441; treatment-as-usual, N=2,149. Naïve regression-baseline-adjustment and doubly-robust approaches suggested Enhanced-IAPT dominated treatment-as-usual, with average per-person (95%bCIs) cost-savings of £30.64 (£22.26 to £38.90) or £29.64 (£20.69 to £37.99) and QALYs-gained of 0.00035 (-0.00075 to 0.00152) or 0.00052 (-0.00105 to 0.00277), respectively; probability of cost-effectiveness at £30,000 per QALY was 99% or 95%, respectively. The doubly-robust and naïve results concurred; albeit, the doubly-robust results suggested average QALY gains were higher but less certain. The cost-effectiveness results were driven by the potential for the Enhanced service to provide cost-savings.

CONCLUSIONS: When using real-world data and treatment allocation is non-randomized, the TT framework alongside doubly-robust analyses should be used to reduce selection and confounding bias.