A scoping review of the use of machine learning in health economics and outcomes research: part 1—data from wearable devices and part 2—data from nonwearables
Lee W, Schwartz N, Bansal A, et al. Value Health. 2023;26(1):article in press; and Lee W, Schwartz N, Bansal A, et al. Value Health. 2022;25(12): 2053-2061.
Artificial intelligence and machine learning techniques and their use in healthcare and health economics and outcomes research (HEOR) have been touted as the next big game changer, and applications have popped up in various HEOR areas recently (eg, for use in automatic literature screening for systematic literature reviews, as well as its use in analyzing real-world data to support market access for products). To aid the understanding of the use of machine learning (ML), the ISPOR Good Practices Report guidelines outlined areas where ML could enhance current research (Figure 1).
Figure 1. Conceptual Diagram of Machine Learning Applications in Health Economics and Outcomes Research
The guidelines provide use cases for each application; however, the current status of ML techniques in HEOR has not been comprehensively described. A team of researchers have reported 2 scoping reviews to provide a picture on the use the ML techniques in HEOR and reported them separately for wearable devices and nonwearables.
Main inclusion criteria for the review were the use of an ML technique and an HEOR-related topic, with the latter defined as “focusing on the clinical and economic aspects of health or health interventions.” The review on wearables included the period between January 2016 and March 2021, and considered data from wearable devices (with wireless mobility, wearability, and portability that could include smartphone data), while the nonwearable literature included the period between January 2020 and March 2021. Note that wearables specifically designed for clinical purposes were excluded (eg, continuous glucose monitors).
Both reviews used standard systematic literature review techniques for the search, screening, and data abstraction. They extracted similar information about the ML techniques used (type, outcome variables, performance metrics) and different outcomes for Parts 1 and 2. For wearables, (1) types of wearable devices, (2) types of data measurement (eg, episodic or longitudinal) and (3) the number of observations in the training dataset were reported. For the nonwearable review, (1) disease areas, (2) application purposes, (3) types of model outcomes, and (4) application settings were reported (Table).
Table. Summary Findings
Figure 2. Nonwearables: Types of Model Purpose and Outcomes (n=32)
In terms of the ML techniques applied, both studies found that tree-based models (25% to 31%), and support vector machine approaches (14% to 19%) were most frequently applied—arguably the simplest and oldest ML methods. This may be driven by the ease of use of these approaches and the high importance of relatively easy interpretability in medical research, as opposed to model goodness of fit. Complex ML algorithms like Bayesian networks and super learner methods were much less frequent.
Apart from the techniques, goals of applications were diverging. Unsurprisingly, analyses of data from wearables focused on monitoring general health, while nonwearable studies had a goal of forecasting disease-specific outcomes based on data from electronic medical records. Forecasting with wearable data was rare and mostly published between 2019 and 2021.
The insight the pair of scoping reviews provides is that ML applications are still limited in HEOR applications. The authors point out future directions that may change that, including greater availability of linked data sources and ML applications on randomized clinical trial data. For wearables, their capacity to provide an alternative to self-reporting, questionnaires, and clinical visits, and the capacity of routinely collected nondisease specific metrics to signal progression (psychological stress on cancer, sleep patterns for neurodegenerative disease) are future directions of research.
The papers along with the ISPOR Good Practices Report on ML methods (Padula et al, 2022) provides the conceptual diagram of ML applications provide readers an excellent guidance into ML methods in HEOR. Future work will be required to evaluate the transparency of the analyses using ML for decision makers and more detailed assessment of the benefits of their use.
