The Size Paradox: Comprehensive Data and Better Methods Can Yield Significant Results with Smaller Sample Sizes in Real World Data (Size Doesn't Always Matter)

Author(s)

Brook RA1, Beren IA2, Kleinman NL3, Winter DA4, Rosenberg EM4
1Better Health Worldwide, Newfoundland, NJ, USA, 2Workpartners, LLC, Loveland, CO, USA, 3WorkPartners, LLC, Merion Station, PA, USA, 4Workpartners, LLC, Pittsburgh, PA, USA

OBJECTIVES: To understand the sample-size impact of basic vs comprehensive employee-patient descriptive information and regression vs non-adjusted comparisons on outcomes comparisons.

METHODS: Retrospective analysis of WorkPartners' Research Reference Database for employees with Hepatitis-C and controls (non-HCV Employees). Random sampling of HCV-employee-patients and 3X controls (all with ≥1 year continuous eligibility). Unadjusted measures (means,t-tests) were compared with two stage (logistic followed by generalized linear models) stepwise regression controlling for basic descriptive regressors (age, gender, region, Charlson Comorbidity Index [CCI] scores) and comprehensive descriptive regressors (basic metrics plus self-reported race, job-related data [salary, full-/part-time status, exempt-/non-exempt status]). Outcomes included direct (medical, prescription) costs, indirect absence costs (from payroll records) due to sick-leave (SL), short-/long-term disability (STD/LTD), workers’ compensation (WC) and lost-time (from employer records due to SL, STD, LTD, WC). Means, standard errors (S.E.s), and confidence intervals [C.I.s] were compared using sensitivity analysis to identify sample sizes needed between the three methods.

RESULTS: The two stage regressions using comprehensive descriptive components consistently had the smallest S.E.s, the narrowest C.I.s, and the highest likelihood of identifying significant between cohort differences. Significant differences in direct costs were achieved in samples of 50 HCV employee-patients. STD and WC costs required samples of 200 HCV employee-patients. WC days required 500 HCV-employee-patients. For HCV employee-patients, the (range [minimum—maximum],average) S.E. ratios between methods were: unadjusted/basic (1.42—55.40,6.64); unadjusted/comprehensive (1.63—58.64,7.84); basic/comprehensive (0.97—2.27,1.41). For control employees (with 3x the HCV sample), S.E. ratios between methods were: unadjusted/basic (1.16—8.70,2.73); unadjusted/comprehensive (1.27—6.45,2.69); basic/comprehensive (0.54—1.27,1.08). The stepwise process frequently selected CCI, salary, age, and gender.

CONCLUSIONS: Comprehensive descriptive information used in regression models consistently outperformed basic descriptive regression models and non-adjusted methods. Two stage regression better controlled for outliers reducing S.E.s by over 50 times. While larger sample sizes are desirable, results are achievable with smaller sample sizes using comprehensive descriptive data and two stage regression techniques.

Conference/Value in Health Info

2023-05, ISPOR 2023, Boston, MA, USA

Value in Health, Volume 26, Issue 6, S2 (June 2023)

Code

EE571

Topic

Economic Evaluation, Methodological & Statistical Research

Topic Subcategory

Cost-comparison, Effectiveness, Utility, Benefit Analysis, Work & Home Productivity - Indirect Costs

Disease

Infectious Disease (non-vaccine)

Explore Related HEOR by Topic


Your browser is out-of-date

ISPOR recommends that you update your browser for more security, speed and the best experience on ispor.org. Update my browser now

×