The primary purposes of the ISPOR Health Outcomes Metrics Index of Open Source Code are to promote estimator transparency and to clarify alternative approaches in health econometric research. Estimator transparency refers to open source code, where all relevant code is accessible, interpretable, operational, and modifiable for alternative use. Transparency does not imply reproducibility of any particular results, because access to modeling parameters or data may be required for replication. Furthermore, base software (eg, STATA) may need to be purchased and some statistical knowledge may be required to utilize open-source code. Nevertheless, full transparency in health econometric methods is often attainable through the citation or provision of open-source code.

This Index of Open Source Code is composed of brief descriptions and hyperlinks to open-source code that can be downloaded for free [at the time of publication in this index]. Proprietary code and code for in silico experiments (eg, simulations) are excluded from the index.

The Index of Open Source Code is arranged according to Topic [Treatment Effects, Health Costs, Healthcare Utilization, Quality of Life and Utility, Censoring and Survival, Other], Subtopics as given below and Base Software.

 

 Topics Sub-Topics
 Treatment Effects
  • Instrumental Variables
  • Control Functions
  • Propensity Scores
 Healthcare Costs 
 Healthcare Utilization 
 Quality of Life & Utility
  • Psychometrics
  • Stated Preference and Choice
  • Revealed Preference and Choice
 Censoring & Survival 
 Other 
The ISPOR Health Outcomes Metrics Index of Open Source Code was developed by the Health Econometrics Working Group of the Health Outcomes Metrics Special Interest Group,, and will be updated as new open source code is received and reviewed. For more information about the Health Econometrics Working Group or to join this ISPOR Working Group, see the ISPOR Special Interest Group Working Group at http://www.ispor.org/sigs/HOM_Econometric.asp. To submit Open Source Code to be included in this Index, email us the information given below. Any listing in this Index may be removed at the discretion of the Health Econometrics Working Group leaders. While brief descriptions and references are provided, the Group's leaders do not endorse any code and are not expected to provide any technical support.

Disclaimer: The information contained in this website is for general information purposes only. The information is provided by ISPOR and while we endeavor to keep the information up to date and correct, we make no guarantee of any kind about the accuracy, reliability, or suitability with respect to this open source code and free software for any purpose. Through this website you are able to link to other websites which are not under the control of ISPOR. We have no control over the nature, content and availability of those sites. The inclusion of any links does not necessarily imply a recommendation or endorse the views expressed within them.

Healthcare Costs

Deb-dealwithzeros2 - Cost Analysis with Zeros
This ensemble of estimators tackles the mass at zero in cost analysis using a variety of measures. Although it may be run with its exemplar data, the code is easily tailored for other purposes.

Open Source Code Website Location: http://urban.hunter.cuny.edu/~deb/courses/dmn-minicourse.html

Open Source Code Developer(s): Partha Deb, Will Manning, and Edward Norton

Base Software (including Version): Stata

Confirmed functionality of code:No

Confirmation Method(s): N/A

Published Studies using this Code: Deb P, Manning W, Norton E. Minicourse on Modeling Health Care Costs and Use, 2nd Biennial Conference of the American Society of Health Economists, Duke University, June 22-25, 2008

gengam2- generalized gamma regressions
This maximum likelihood estimator produces fits the three parameter generalized gamma (GGM) distribution to right skewed outcomes, and tests for several of the standard alternatives as special cases - OLS with a normal error, OLS for the log normal, the standard gamma and exponential with a log link, and the Weibull.

Open Source Code Website Location: http://faculty.washington.edu/basua/software.html

Open Source Code Developer(s): Anirban Basu

Base Software (including Version): Stata 9

Confirmed functionality of code: No

Confirmation Method(s): N/A

Published Studies using this Code: Manning WG, Basu A, Mullahy J. Generalized modeling approaches to risk adjustment of skewed outcomes. 2003 NBER Working Paper No. t0293. Journal of Health Economics 2005; 24(3): 465-488. [PMID: 15811539]

GFCURE
A parametric regression analysis package for R that can be used to fit the generalized F distribution to cost data. A two part model should be used to remove any zero cost data points first.

Open Source Code Website Location: http://www.math.mun.ca/~ypeng/research/gfcure/

Open Source Code Description & Intent: This ensemble of estimators tackles the mass at zero in cost analysis using a variety of measures. Although it may be run with its exemplar data, the code is easily tailored for other purposes.

Open Source Code Developer(s): Francis Peng

Base Software (including Version): R-statistics

Confirmed functionality of code:Yes

Confirmation Method(s): This code was used to perform a statistical analysis as part of a  published cost of illness study.

Published Studies using this Code: Hubben GA, Bishai D, Pechlivanoglou P, Cattelan AM, Grisetti R, Facchin C, Compostella FA, Bos JM, Postma MJ, Tramarin A. The societal burden of HIV/AIDS in Northern Italy: an analysis of costs and quality of life. AIDS Care. 2008 Apr;20(4):449-55. PubMed PMID: 18449822.

pglm - power GLM using extended estimating equations
This command simultaneously solves the extended estimating equations estimator for parameters in the link and variance functions along with those of the linear predictor in a generalized linear model. The method addresses difficulties in choosing the correct link and variance functions in these models. It decouples the scale of estimation for the mean model, determined by the link function, from the scale of interest for the scientifically relevant effects. Additionally, it estimates a flexible variance structure from the data leading to efficient estimation.

Open Source Code Website Location: http://faculty.washington.edu/basua/software.html

Open Source Code Developer(s): Anirban Basu

Base Software (including Version): Stata

Confirmed functionality of code: No

Confirmation Method(s): N/A

Published Studies using this Code: Basu A, Rathouz P. Estimating marginal and incremental effects on health outcomes using flexible link and variance function models. Biostatistics 2005; 6(1): 93-109. [PMID: 15618530] Basu A Extended generalized linear models: Simultaneous estimation of link and variance functions. The Stata Journal 2005; 5(4): 501-516. Basu A, Arondekar BV, Rathouz P. Scale of interest versus scale of estimation: Comparing alternative estimators for the incremental costs of a comorbidity. Health Economics 2006; 15(10): 1091-1107. [PMID: 16518793]

Healthcare Utilization

XTSCC: Pooled OLS/WLS or Fixed Effects Regression with Driscoll & Kraay Standard Errors
XTSCC produces Driscoll and Kraay (1998) standard errors for coefficients estimated by pooled OLS/WLS or fixed-effects (within) regression. depvar is the dependent variable and varlist is an optional list of explanatory variables.

The error structure is assumed to be heteroskedastic, autocorrelated up to some lag and possibly correlated between the groups (panels). These standard errors are robust to general forms of cross-sectional (spatial) and temporal dependence when the time dimension becomes large. Because this nonparametric technique of estimating standard errors places no restrictions on the limiting behavior of the number of panels, the size of the cross-sectional dimension in finite samples does not constitute a constraint on feasibility – even if the number of panels is much larger than T. Nevertheless, because the estimator is based on an asymptotic theory, one should be somewhat cautious with applying this estimator to panels that contain a large cross-section but only a short time dimension.

The xtscc command is suitable for use with both balanced and unbalanced panels. Furthermore, it can handle missing values.

Open Source Code Website Location: http://econpapers.repec.org/software/bocbocode/s456787.htm

Code Accessed On : 12/29/2013

Open Source Code Developer(s): Daniel Hoechle, University of Basel, daniel.hoechle@unibas.ch

Base Software (including Version): Stata

Confirmed functionality of code: No

Confirmation Method(s): N/A

Published Studies using this Code: Driscoll, J., and A.C. Kraay. 1998. “Consistent Covariance Matrix Estimation with Spatially Dependent Data.” Review of Economics and Statistics 80: 549-560.

Hoechle, D. 2007. “Robust Standard Errors for Panel Regressions with Cross-Sectional Dependence.” Stata Journal 7(3): 281-312.

Roebuck, M.C. and J.N. Liberman. 2009. “Impact of Pharmacy Benefit Design on Prescription Drug Utilization: A Fixed Effects Analysis of Plan Sponsor Data.” Health Services Research 44(3): 988-1009.

 

Quality of Life and Utility- Revealed Preference and Choice

MXL - Mixed Logit Estimation Routine for Cross-Sectional Data
A mixed logit (MXL) model is essentially a standard logit model with coefficients that vary in the population. The routine estimates the distribution of coefficients. MXL does not exhibit independence from irrelevant alternatives as does standard logit, and allows correlation in unobserved utility over alternatives and over time.

Open Source Code Website Location: http://elsa.berkeley.edu/Software/abstracts/train0196.html

Code Accessed On: 1/4/2010

Open Source Code Developer(s): Kenneth Train

Base Software (including Version): Gauss

Confirmed functionality of code: Yes

Confirmation Method(s): The submitted used this code to estimate mixed logit models without and did not find any issue with its functionality

Published Studies using this Code: Revelt D, Train K, "Mixed Logit with Repeated Choices of Appliance Efficiency Levels." Review of Economics and Statistics, Vol. LXXX, No. 4, 647-657, 1998. Brownstone D, Train K, "Forecasting New Product Penetration with Flexible Substitution Patterns." Journal of Econometrics, Vol. 89, No. 1, pp. 109-129, 1999

Treatment Effects- Control Functions

mtreatreg- fits models with multinomial treatments and continuous, count and binary outcomes using maximum simulated likelihood
The model considers the effect of an endogenously chosen multinomial-valued treatment on an outcome variable, conditional on two sets of independent variables. The outcome variable can be continuous, binary or integer-valued while the treatment choice is assumed to follow a mixed multinomial logit distribution. The model is estimated using maximum simulated likelihood and the simulator uses Halton sequences.

Open Source Code Website Location: http://EconPapers.repec.org/RePEc:boc:bocode:s457064

Code Accessed On : 12/29/2013

Open Source Code Developer(s): Partha Deb

Base Software (including Version): Stata 10

Confirmed functionality of code: No

Confirmation Method(s): NA

Published Studies using this Code: Deb P, Trivedi PK. Specification and simulated likelihood estimation of a non-normal treatment-outcome model with selection: Application to health care utilization. The Econometrics Journal 2006; 9(2): 307 - 331.

Treatment Effects- Propensity Scores

psmatch2 - Stata module to perform full Mahalanobis and propensity score matching, common support graphing, and covariate imbalance testing
This macro implements full Mahalanobis and propensity score matching, common support graphing, and covariate imbalance testing. This routine supersedes the previous 'psmatch' routine of B. Sianesi.

Open Source Code Website Location: http://ideas.repec.org/c/boc/bocode/s432001.html

Code Accessed On: 1/4/2010

Open Source Code Developer(s): Edwin Leuven & Barbara Sianesi

Base Software (including Version): Stata

Confirmed functionality of code: Yes

Confirmation Method(s): The submitted used this code to estimate various propensity score matching estimators

Published Studies using this Code: McBean AJ, Yu X, "The Underuse of Screening Services Among Elderly Women With Diabetes", Diabetes Care, 30 (6): 1466 -1472. Austin PC, Manca A, “Using Propensity Score Methods to Analyze Individual Patient-Level Cost-Effectiveness Data From Observational Studies” Health, Econometrics and Data Group (HEDG) Working Papers 08/20, HEDG, c/o Department of Economics, University of York.

Treatment Effects- Instrumental Variables

ivqte - Instrumental Variable Quantile Treatment Effects
This Stata macro implements IV estimators for unconditional quantile treatment effects (QTE) when the treatment selection is endogenous.

Open Source Code Website Location: http://froelich.vwl.uni-mannheim.de/1357.0.html?&L=1

Open Source Code Developer(s): Marcus Frolich & Blaise Melly

Base Software (including Version): Stata

Confirmed functionality of code: Yes

Confirmation Method(s): The submitted used this code to estimate quantile treatment effect estimate

Published Studies using this Code: Frolich M, Melly B. “Estimation of Quantile Treatment Effects with STATA”. Mimeo, University of St. Gallen. Available at http://www.alexandria.unisg.ch/Publications/46580/L-en

Censoring and Survival

Censored - Censored regression and sample selection
This intuitive code provides an empirical application of censored regression and sample selection using in the context of female labor supply, and may be adapted for further applications.

Open Source Code Website Location: http://www.uam.es/personal_pdi/economicas/rsmanga/mef_aplications_0809.html

Open Source Code Developer(s): Thomas Mroz

Base Software (including Version): Stata

Confirmed functionality of code:No

Confirmation Method(s): N/A

Published Studies using this Code: Mroz, T.A. (1987): "The Sensitiviy of an Empirical Model of Married Women's Hours of Work to Economic and Statistical Assumptions", Econometrica, 55, 765-799.

Other

mice: Multivariate Imputation by Chained Equations
Performs multiple imputation for missing data using Gibbs sampling. Methods available include Bayesian normal, logistic and polytomous regression, non-Bayesian normal, linear discriminant analysis and predictive mean matching.

Open Source Code Website Location: http://cran.r-project.org/web/packages/mice/index.html

Code Accessed On : 02/05/2010

Open Source Code Developer(s): Stef van Buuren & Karin Groothuis-Oudshoorn

Base Software (including Version): R version 2.4 or higher

Confirmed functionality of code:Yes

Confirmation Method(s): Have used to perform multiple imputation using the predictive mean matching (pmm) method.

Published Studies using this Code: Van Buuren, S., Brand, J.P.L., Groothuis-Oudshoorn C.G.M., Rubin, D.B. (2006) Fully conditional specification in multivariate imputation. Journal of Statistical Computation and Simulation, 76, 12, 1049–1064. http://www.stefvanbuuren.nl/publications/FCSinmultivariateimputation-JSCS2006.pdf

Van Buuren, S. (2007) Multiple imputation of discrete and continuous data by fully conditional specification. Statistical Methods in Medical Research, 16, 3, 219–242. http://www.stefvanbuuren.nl/publications/MIbyFCS-SMMR2007.pdf

Peter D. Faris, William A. Ghali, Rollin Brant, Colleen M. Norris, P. Diane Galbraith, Merril L. Knudtson, for the APPROACH Investigators, Multiple imputation versus data enhancement for dealing with missing data in observational health care outcome analyses, Journal of Clinical Epidemiology, Volume 55, Issue 2, February 2002, Pages 184-191, ISSN 0895-4356, DOI: 10.1016/S0895-4356(01)00433-4.

(http://www.sciencedirect.com/science/article/B6T84-44XV90D-D/2/6a0b381d04875f15c19f420f29062805 )

NFXP - Nested Fixed Point Maximum Likelihood Algorithm
This code can numerically solve stochastic control problems, much like our decision analytic models, by computing the associated functional fixed point as a subroutine nested within a standard nonlinear maximum likelihood optimization algorithm. His example concerns when to replace bus engines at the Madison Metropolitan Bus Company, and won the 1992 Ragnar Frisch Medal for "best empirical paper in Econometrica in the preceding 5 years".

Open Source Code Website Location: http://gemini.econ.umd.edu/jrust/software.html

Open Source Code Developer(s): John Rust

Base Software (including Version): Gauss

Confirmed functionality of code: No

Confirmation Method(s): N/A

Published Studies using this Code: Rust J "Optimal Replacement of GMC Bus Engines: An Empirical Model of Harold Zurcher" Econometrica 55-5 999-1033. Rust J "Maximum Likelihood Estimation of Discrete Control Processes'' SIAM Journal on Control and Optimization 26-5 1006-1024

Nrocarea - ROC analysis
This code contains two programs related to ROC curves. The first, rocintercept, estimates the optimal sensitivity and specificity (tangency of OOS and ROC curve) and the intercept of the tangent line. The second, testchar, calculates sensitivities, specificities, and stratum-specific likelihood ratios (plus confidence intervals) for the operating points that define the convex hull of the ROC curve. nrocarea also has 3 utility programs used to cumulate test results (cumval), calculate 95% CI for single proportions (pro95ci), and calculate 95% CI for SSLR (lrci). Finally, there is a utility program, nrocdoc, that provides brief documentation for the program.

Open Source Code Website Location: http://www.uphs.upenn.edu/dgimhsr/stat-roc.htm

Open Source Code Developer(s): Henry Glick

Base Software (including Version): Stata

Confirmed functionality of code: No

Confirmation Method(s): N/A

Published Studies using this Code: None

randomForest: Breiman and Cutler's random forests for classification and regression
Uses the random forest algorithm (Breiman and Cutler) for classification and regression. Can also be used as an “unsupervised” (i.e., no dependent variable) classification tool to estimate proximity of observations. Random forests involve the construction of multiple recursive partition models (trees) for classifying individuals or predicting similarity of continuous responses. Trees are constructed from a random subset of predictor variables at each decision point and are pooled into an ensemble (forest) using bootstrap aggregation (bagging).

Open Source Code Website Location: http://cran.r-project.org/web/packages/randomForest/index.html

Open Source Code Developer(s): Fortran original by Leo Breiman and Adele Cutler, R port by Andy Liaw and Matthew Wiener.

Base Software (including Version): R version 2.0 or higher

Confirmed functionality of code: Yes

Confirmation Method(s): Have used random forest classification option successfully.

Published Studies using this Code: Diaz-Uriarte, R.  and Alvarez de Andres, S.  Gene selection and classification of microarray data using random forest.  BMC Bioinformatics 2006; 7(3) http://www.biomedcentral.com/1471-2105/7/3 Polley, EC and van der Laan MJ.  Selecting Optimal Treatments Based on Predictive Factors.  UC Berkeley Division of Biostatistics Working Paper Series 2009, #244.

http://www.bepress.com/cgi/viewcontent.cgi?article=1247&context=ucbbiostat

 

WinBUGS Code for Mixed Treatment Comparison models for events with competing risks
Provides code used to estimate Mixed Treatment Comparison (MTC) models for simple and competing risk event rates. Code is specific to an MTC used to evaluate pharmacological treatments for relapse prevention in schizophrenia, but is easily modifiable. Code can be copied and pasted into a text editor directly from link.

Open Source Code Website Location: http://www.ncbi.nlm.nih.gov/bookshelf/br.fcgi?book=nicecg82&part=appendices.app13

Open Source Code Developer(s): U.K. National Institute for Health and Clinical Excellence

Base Software (including Version): WinBUGS 1.4.3

Confirmed functionality of code:Yes

Confirmation Method(s): Have adapted code for use in other applications.

Published Studies using this Code: National Institute for Health and Clinical Excellence. Schizophrenia: Core Interventions in the Treatment and Management of Schizophrenia in Adults in Primary and Secondary Care (update), CG82. NICE, 2009 http://www.ncbi.nlm.nih.gov/bookshelf/br.fcgi?book=nicecg82