Novel Elements of the Value Flower: Fake or Truly Novel?
Sarah Goring, MSc, SMG Outcomes Research, Vancouver, BC, Canada; Louis P. Garrison, Jr, PhD, University of Washington, Seattle, WA, USA; Jeroen P. Jansen, PhD, University of California, San Francisco, CA, USA; PRECISIONheor, San Francisco, CA, USA; Andrew H. Briggs, BA, MSc, MSc, DPhil, London School of Hygiene & Tropical Medicine, London, England, UK
As moderator, Sarah Goring introduces the “value flower.” Lou Garrison reflects on the ISPOR Special Task Force work, provides context for its genesis and recommendations for future work. Jeroen Jansen presents methodological detail for 2 elements of value related to uncertainty and risk preference and argues that future research to understand its relevance and credibility may be worthwhile. Andrew Briggs provides a critique of the novel elements of value, arguing in favor of the cost-per-QALY metric for capturing value.
At ISPOR’s 22nd Annual European Congress in Copenhagen, Denmark, an issue panel convened to discuss the novel elements of value proposed by ISPOR’s Special Task Force on US Value Assessment Frameworks.1, 2 The panelists debated whether the elements were truly novel and relevant for decision making, or whether this news was “fake.” They further discussed how to incorporate these novel elements into cost-effectiveness analyses. Whereas the task force had focused on the US setting, the panelists considered their relevance also in other health technology assessment (HTA) settings.
Sarah Goring: The Value Flower and Elements of Value
The task force introduced the “value flower” (Figure 1) as a type of augmented cost-effectiveness analysis.2 The 12 petals represent elements of value: 2 elements (costs and quality-adjusted life-years [QALYs] gained) represent existing cornerstones of value in health; 2 elements (adherence and productivity) are relatively common, yet inconsistently incorporated into cost-effectiveness analysis; and the remaining 8 are considered potentially novel.
Figure 1. The Value Flower.
Adapted from Lakdawalla et al.2 QALY indicates quality-adjusted life-year.
Several of the more novel elements of value relate to uncertainty. The value of hope captures differences in patients’ risk tolerance: whereas some may value a treatment with high variability in outcomes (with the hope that they may be among the few who respond very well), others may prefer a treatment with less variability around expected outcomes. The value of knowing pertains to the pairing of a new treatment plus a new test that can accurately predict who will respond; it captures the additional value stemming from that knowledge. Real option value is more forward-looking. In therapeutic areas on the brink of a major shift in treatment efficacy (think of the era slightly before curative drugs for hepatitis C), patients and payers may be willing to pay more for a life-extending therapy to enable the possibility of receiving the curative medicine when it becomes available. Alternatively, they may forego inferior treatments—at some risk—to wait for the curative treatment.
Insurance value also reflects the benefit of reducing uncertainty through access to a health plan’s benefit package. It captures the value to healthy individuals of being protected from the physical and financial burden of a particular illness due to the availability of a new therapy/technology. This is particularly relevant for severe diseases, of which healthy individuals tend to be more fearful. In this way, the severity of disease value element is related. Research shows that incremental gains in utilities of say, 0.1, may be considered more valuable when applied to more severe disease; however, this type of weighting is not captured in traditional cost-effectiveness analysis.2
Equity in healthcare is of paramount value to society yet tends not to be explicitly captured in traditional cost-per-QALY metrics. Several attempts to formally quantify and evaluate impacts of health interventions on equity in an HTA setting have already been made, and the task force identified this as an important area for future work.3
The last 2 value elements are types of system-level “externalities”: fear of contagion (germane to the current pandemic) and scientific spillovers (research and development activities can add broadly to our collective knowledge).
Lou Garrison: Reflections on the ISPOR Special Task Force on US Value Frameworks
Beginning around 2014 and in the short span of a couple of years, 5 “value frameworks” for new medicines were launched or promulgated in the United States. These frameworks seemed to be creating some confusion, so ISPOR formed a Special Task Force in 2016 to review them from a health economics perspective. Peter Neumann, Dick Willke, and Lou Garrison were coleaders of this group of 10 distinguished US and ex-US economists—supported with feedback from a multidisciplinary review panel. The task force published its report in February 2018 and published an article in November 2019 with reflections on the task force effort, making observations and a key recommendation.4
"The important US Second Panel of Cost-Effectiveness argues that some elements, such as productivity, are best monetized in the numerator of the cost-effectiveness ratio rather than in the denominator."
The key takeaway was that this ISPOR Special Task Force “recommended further methods development and testing of alternative approaches that build on a cost-per-QALY metric, including augmented cost-effectiveness analysis and multicriteria decision analysis in support of deliberative processes.” The augmented cost-effectiveness analysis value flower identified some “potential novel elements of value,” but they were not intended as an exhaustive list and/or a definitive list of what is “novel.” All of these elements were identified in prior research. Indeed, the task force endorsed the conventional cost-per-QALY metric as a foundational concept to be built upon and augmented. A key point from a US perspective is that this endorsement goes against legal prohibitions against the use of the QALY in the US Medicare program.
The task force also argued that the cost-per-QALY metric has limitations in theory and in practice that need to be addressed—hence, the need for more work on augmented cost-effectiveness analysis and multicriteria decision analysis as 2 alternative approaches. The task force conceived of value from an economic perspective defined as what individuals would be willing to pay or to forego (“opportunity cost”) in order to have insurance coverage for more healthcare. This is in contrast with “clinical value” in the regulatory sense of the net balance of the benefits of a new medicine versus its risks. Clearly, clinical value is key to “health gain”—one of the 2 key drivers of value in innovative medicines, with the other being “net cost,” which is also affected by clinical value. The QALY measures health gain in terms of length of life and quality of life. In theory, some might argue the QALY is fully comprehensive if health-related quality of life is defined as “everything that is not length of life.” In practice, our estimation of health state utility, via, for example, the EQ-5D or time trade-off technique cannot—given the cognitive limitations of humans—consider everything that matters. Hence, the important US Second Panel of Cost-Effectiveness argues that some elements, such as productivity, are best monetized in the numerator of the cost-effectiveness ratio rather than in the denominator.5 The same rationale for monetization in augmented cost-effectiveness analysis may apply to the novel elements to include in estimating the expected net monetary benefit of specific interventions.
In characterizing the 5 frameworks, the task force also emphasized “decision context” and perspective. Figure 2 illustrates a “cascade” of decision contexts—from the health plan level to clinical guidelines and pathways and on down to shared clinical decision-making—that attempts to clarify key differences in the focus of the 5 value frameworks. This task force focused primarily on the issue of inclusion in the health plan’s benefit package, ie, what is generally considered as HTA for formulary inclusion. In the United States, this approach is best exemplified by the Institute for Clinical and Economic Review. The task force recognized that multiple stakeholders are involved in each of these decision contexts and that the decisions can be analyzed from any of their perspectives. However, for the insurance benefit package, the most relevant perspective is that of the premium-paying plan member (who is also a potential patient).
Figure 2. Decision contexts and recent value frameworks: a cascade from the plan level to the patient level.
Adapted from Garrison et al.8
From the perspective of plan members, the task force identified 2 sets of potential elements that are not well-captured in the cost-per-QALY metric. The first—and perhaps more novel group—has to do with the handling of uncertainty. Conventional cost-effectiveness analysis does not directly account for risk aversion and the impact of uncertainty on the plan member or patient well-being. Covering a new medicine in the benefit package provides “insurance value”—both in terms of financial risk protection and health risk protection. Increasing the premium to provide coverage for a new medicine can give the plan member peace of mind that is not conventionally captured in the QALY. Other uncertainty-related elements—such as the value of hope, real option value, and the value of knowing—have been defined in theory and estimated in empirical research. But there is clearly much more to be done to sort them out.
The task force also presented a multi-criteria decision analysis (see Figure 3) as an alternative approach to weighing the set of attributes that matter in value assessment, including such factors as equity, scientific spillovers, and health system readiness. Both multicriteria decision analysis and augmented cost-effectiveness analysis were seen as tools to be used to support the deliberative processes about coverage that health plans typically engage in. Further theoretical and empirical work is needed to sort out which potentially novel elements are distinct and practically important. Arguably, in most situations, the expected QALY gain should be the key driver of the value of innovative medicines to patients under either augmented cost-effectiveness analysis or multicriteria decision analysis; however, for ultrarare, health-catastrophic conditions, the interaction of insurance value, value of hope, and disease severity could argue for greater value and, in effect, a higher cost-effectiveness threshold.6,7
Figure 3. Multicriteria decision analysis: an input to a deliberative health technology assessment process.
Source: Garrison et al.4 QALY indicates quality-adjusted life-year.
Jeroen Jansen: Relevant Petals of the Value Flower to Acknowledge Risk Preferences
Conventional cost-effectiveness analysis is used to quantify the value of a health technology for a particular target patient population. Assuming a willingness to pay for a unit of health (eg, QALY), the expected net monetary benefit, defined as the health outcomes expressed in monetary terms minus net costs, can be calculated for the compared technologies. The intervention with the greatest expected net monetary benefit is deemed to have the greatest value. Standard cost-effectiveness analysis assumes that consumers are risk neutral in health.9 However, ignoring risk aversion may underestimate the value of interventions for more severe diseases and overestimate it for mild diseases.6,10
Two additional components of value related to risk preferences have been labeled “insurance value” and “value of hope” in the value flower.2 Unfortunately, these labels may lead to a misunderstanding of the concepts and downright rejection of their relevance (eg, “studying the value of insurance is only relevant for the United States where there is no universal coverage”; “incorporating hope is the antithesis of rational cost-effectiveness analysis-based decision making“). However, the formal integration of the value that a new health technology provides by affecting the uncertainty that individuals face regarding health outcomes in the cost-effectiveness analysis framework is useful to consider.
"The intervention with the greatest expected net monetary benefit is deemed to have the greatest value. Standard cost-effectiveness analysis assumes that consumers are risk-neutral in health."
Availability of an efficacious (and safe) intervention for a certain disease does not only provide value to patients, but also provides value to healthy individuals at risk for that disease.6,10 An efficacious intervention provides some degree of protection against the physical risk of the disease. For example, the vaccines that are now available for SARS-COV-2 mean that we all have less to fear from it, even though not all of us would develop COVID-19. In addition, an efficacious intervention converts an uninsurable physical risk (getting sick) into a financial risk, which can be mitigated by health insurance.
Conventional value of a new intervention relative to standard of care from the perspective of a healthy individual at risk of the disease can be calculated as the marginal value to sick patients multiplied by the probability of getting sick.10,11 We account for risk by scaling the ex post (that is, after the health disorder is realized) value of the intervention by the probability of falling ill. This simple approach implies that individuals are risk neutral. If we want to incorporate risk aversion, we need to adjust the conventional estimate of value with the reduction in physical risk and the increase in financial risk. These 2 elements form the “insurance value” with no health insurance, which can be calculated as the net monetary benefit multiplied with the variance of the probability of disease and a constant based on the marginal rate of substitution between “well” and “sick” states.10, 11 The marginal rate of substitution relates to the consumer’s degree of risk aversion and intuitively reflects the amount of money a consumer would give up when healthy in exchange for gaining an additional dollar when sick. It rises when the consumer faces greater risks from illness. If there is access to health insurance, the increase in financial risk is partly or completely offset and we can calculate the “insurance value” with health insurance. In Figure 4, we see the results of an example of cost-effectiveness analysis of biologics for rheumatoid arthritis.11 Results are expressed as net monetary benefit for 2 values of marginal rate of substitution (2.5 and 1.2), with the former representing a greater degree of risk aversion. An intervention that has value ex post reduces risk ex ante because the reduction in physical risk more than offsets the increase in financial risk.
Figure 4. Incorporating “insurance value” in an augmented cost-effectiveness analysis to quantify the value of biologics for healthy individuals at risk of rheumatoid arthritis.
Source: Obtained with IVI-RA model.11 MRS indicates marginal rate of substitution.
The concept underlying “insurance value,” (ie, an increase in expected utility with a reduced variance) also applies to quantifying the ex post health impact of a health technology. Health outcomes will likely vary across patients, even when we focus on clinically homogeneous subgroups. Consider 2 competing interventions with the same average survival—but one with much greater variability in survival—would be considered equivalent in a conventional cost-effectiveness analysis. For individual patients, however, this variability in outcomes represents uncertainty, and some may prefer one treatment over the other. Risk-averse individuals prefer the intervention with the reduced variability in treatment effects. If the first intervention results in an average 1.5 QALYs but with considerable variability across individuals in the population, the certainty equivalent might be 1.2 QALYs, whereas the second intervention with the same average QALYs yet less variability might be worth 1.4 certainty equivalent QALYs. The certainty equivalent is the number of QALYs that a patient would need to obtain to be indifferent between the comparator and the alternative treatment strategy.10
This brings us to “value of hope.” The distribution of health outcomes is not only characterized by its variance but also by the skewness. Empirical research suggests that interventions may also have value when they provide an increase in the positive skew of the distribution of treatment effects.12 Even for risk-averse individuals, interventions for severe disease that increase the “right tail” of the distribution may still be incrementally valuable despite an increase the variation in outcomes.10 Patients with cancer may prefer the hopeful treatment option with the longer right-tail survival curve at the expense of a higher risk of dying earlier.12
The recently presented framework by Lakdawalla and Phelps, where risk aversion related to the distribution in outcomes is integrated in the augmented cost-effectiveness analysis framework, facilitates investigating the impact on estimates of value, its relevance, and whether it can be estimated with credibility given the evidence typically available.6
Andrew Briggs: An Antithetical View
Although “fake” and “novel” are hardly antonyms—and it is not being suggested that the additional components of value identified by the task force are fake—it can be argued that neither are they novel or necessary.
The task force is to be lauded for concluding that cost-utility analysis and the QALY should remain front and center to value assessment in the United States. This is a hugely important endorsement of a methodology that has been refined over the last 40 years and is consistent with the original recommendations of the US Panel on Cost-Effectiveness Analysis, who also re-endorsed cost-utility analysis as the central method for assessing value in its 20-year update.5
Nevertheless, the task force did identify a number of additional components of value that they argued fell outside the standard cost-utility framework (Figure 1), although perhaps it is unfortunate that the net cost and QALY components were not represented as the central core of the flower. Representing them as petals alongside 10 other elements of value gives the impression that each could be equally important, but that is most likely not what the authors intended. Many of the elements of value were taken from the literature and by definition are not novel elements—rather they are identified as elements that are rarely included in a conventional cost-effectiveness analysis. Elements such as productivity, adherence, and reduction in uncertainty (the value of knowing) are routinely captured (at least in a societal perspective). Other elements, such as hope and fear, could legitimately affect individual decision making but are nonetheless fraught with difficulties in measurement as they relate to subjective experience and could be manipulated by the context. While both hopes and fears can be rational (as is the case with fear of COVID-19), they can also be groundless (one can “hope against hope”). Furthermore, the value of hope and the value of knowing can cancel each other out. Patients may value the chance of long-tailed survival, but if we develop a companion diagnostic that perfectly identifies patients who will benefit, do we have to remove the value of hope for those who now learn they cannot benefit from treatment?
"The success of the cost-per-QALY approach has largely been because it has simplified the broader welfarist cost-benefit approach."
The most important additional components of value identified relate to equity and severity of disease. Most individuals and societies actively value equity and would not want to see efficiency pursued at the expense of creating further inequality in the population. We also see evidence that severity of disease is important to many individuals and societies. Some of the recent debate about disability in the United States has focused on limitations of the QALY model in fairly reflecting capacity to benefit for disabled patients. In Europe, both Denmark and the United Kingdom have experimented with adjusting the QALY threshold to give greater weight to interventions that are targeted to those with a lower health endowment (ie, greater severity of disease).
If we are to consider these additional elements of value, how should they be incorporated into the decision-making process? At least 4 possible solutions have been proposed. The task force suggested multicriteria decision analysis and augmented cost-effectiveness analysis as 2 approaches that could quantify additional value elements. While intuitively appealing, the devil is in the detail and the vast majority of examples of multicriteria decision analysis in the literature fall short of even the basic requirements required of economic assessment—particularly in relation to the cardinal utility property of the weights between attributes. The augmented cost-effectiveness analysis approach has much to commend it, but does have the problem of lack of comparability in the event that some analysts include additional value elements and others do not. Another approach is that decision-making bodies could adjust their decision thresholds to allow for other elements of value in the way that the Danes and the British have tried for disease severity. The final option is the status quo—to accept, as most jurisdictions that use formal cost-effectiveness methods do—that cost-utility estimates are only an input to, and not a substitute for, a deliberative decision-making process that allows for additional elements of value to be contextualized into the process without the need for formal quantification into the cost-effectiveness calculus.
In conclusion, the additional elements of value identified by the task force are a useful categorization of those things that can be captured in an economic appraisal and reflect the full economic analysis possible when conducting a full “welfarist” cost-benefit analysis. However, it is perhaps worth remembering that the success of the “extra-welfarist” approach in healthcare has been largely because it has simplified the traditional welfarist approach. Augmenting cost-utility analysis to reestablish these elements is equivalent to moving back to a welfarist approach. Far better is to accept the imperfections of cost-utility analysis and accept that a deliberative decision-making process can assess the wider elements of value. With apologies to Churchill, who once commented that democracy was the worst form of government except for all others that have been attempted from time to time, it has been suggested that cost-utility analysis is the worst form of economic appraisal for healthcare decision making—except for all the others that have been attempted from time to time.
Sarah Goring: Concluding Observations
The disparate views amongst the panelists provided an excellent opportunity for discussion and reflection on how value is captured in healthcare decision making. On the one hand, advocates for the STF’s approach argue for attempting the formal quantification—and even monetization—of additional value elements. On the other, skeptics would argue that additional value elements can be handled without formal quantification into the cost-effectiveness analysis calculus, and instead provide the context to a deliberative discussion of value to which cost-utility analysis remains core. Clearly, this is an important debate that we can expect to continue and from which our field will benefit.
Overall, each of the elements appeared to capture some aspect of real—not fake—value; however, their relative importance, their estimation, and how exactly they are incorporated into economic evaluation for healthcare decision making is still open to debate.
References
1. Goring S, Garrison L, Jansen J, Briggs A. Fake or Novel Elements of Value? Breakout session 1 (IP3); ISPOR Europe November 2-6, 2019. Copenhagen, Denmark.
2. Lakdawalla D, Doshi J, Garrison L, Phelps C, Basu A, Danzon P. Defining elements of value in healthcare-a health economics approach: an ISPOR Special Task Force Report [3]. Value Health. 2018;21(2):131-139.
3. Cookson R, Mirelman J, Griffin S, et al. Using cost-effectiveness analysis to address health equity concerns. Value Health. 2017;20(2):206-212.
4. Garrison L, Neumann P, Willke R. Reflections on the ISPOR special task force on US value frameworks: implications of a health economics approach for managed care pharmacy. J Manag Care Spec Pharm. 2019;25(11):1185-1192. doi:10.18553/jmcp.2019.25.11.1185.
5. Neumann P, Ganiats T, Russell L, Sanders G, Siegel J. Cost-effectiveness in Health and Medicine. Oxford, England: Oxford University Press; 2016. Print ISBN-13: 9780190492939.
6. Lakdawalla D, Phelps C. Health technology assessment with risk aversion in health. J Health Econ. 2020:7(72):102346. doi.org/10.1016/j.jhealeco.2020.102346.
7. Garrison L, Jackson T, Paul D, Kenston M. Value-based pricing for emerging gene therapies: the economic case for a higher cost-effectiveness threshold. J Manag Care Spec Pharm. 2019;25(7):793-799. doi: 10.18553/jmcp.2019.18378.
8. Garrison L, Pauly M, Willke R, Neumann P. An overview of value, perspective, and decision context: a health economics approach: An ISPOR Special Task Force Report [2]. Value Health. 2018;21(2):124-130.
9. Garber A, Phelps C. Economic foundations of cost-effectiveness analysis. J Health Econ. 1997;16(1):1-31. doi:10.1016/s0167-6296(96)00506-1.
10. Lakdawalla DN, Phelps CE. Evaluation of medical technologies with uncertain benefits. National Bureau of Economic Research; Published July 11, 2019. https://www.nber.org/papers/w26058.
11. Incerti D, Curtis J, Shafrin J, Lakdawalla D, Jansen J. A flexible open-source decision model for value assessment of biologic treatment for rheumatoid arthritis. Pharmacoeconomics. 2019;37(6):829-843. doi:10.1007/s40273-018-00765-2.
12. Lakdawalla D, Romley J, Sanchez Y, Maclean J, Penrod J, Philipson T. How cancer patients value hope and the implications for cost-effectiveness assessments of high-cost cancer therapies. Health Aff (Millwood). 2012;31(4):676-682. doi:10.1377/hlthaff.2011.1300.
.
