OBJECTIVES: Randomized clinical trials (RCTs) are an important source of evidence for health technology assessment (HTA), but their external validity is often uncertain. Assessment of RCT external validity can facilitate decision making by improving the relevance of RCT evidence to local contexts. Transportability analyses are causal inference methods used to assess the validity of RCTs in external settings. We conducted a simulation study to assess the performance of two novel methods, inverse odds of participation weighting (IOPW) and g-computation, for transporting original RCT findings to real-world data (RWD).

METHODS: Datasets from RCT (study) and RWD (target) populations were generated with different distributions of covariates and effect modifiers and different true average treatment effects (ATEs) for a continuous outcome. We assumed that the treatment and response variables were only observed in the study dataset. IOPW and g-computation methods were each used to estimate the ATE in the target population using its covariate data. Bias, coverage of 95% confidence interval (CI), and relative error of standard error (SE) were used to compare the performance of IOPW and g-computation versus naïve linear regression.

RESULTS: Compared to the naïve estimator, the IOPW and g-computation estimators performed considerably better on all metrics. There was considerably less bias, and the 95% CIs of the target ATE calculated using IOPW and g-computation empirically met the nominal level. The relative errors of SE were small in all methods. The performance of the transportability analysis methods is contingent on specifying the models correctly when using IOPW and g-computation.

CONCLUSIONS: Transportability analysis is an important tool that can facilitate HTA decision making. As seen in our simulation study, transporting results from RCT to real-world settings using IOPW and g-computation methods show great promise. Challenges in the analysis include correctly specifying factors that affect treatment assignment in the RCT and modify the treatment effect.