Addressing the Evidence Gap in Evaluating the Health of Children
An Interview with Wendy J. Ungar, MSc, PhD
Section Editor: Marisa Santos, PhD, MD, Instituto Nacional de Cardiologia, Rio de Janeiro, Brazil
This month, I had the pleasure of interviewing Wendy J. Ungar, MSc, PhD, for Value & Outcomes Spotlight's theme that focuses on the valuation of health in children. Dr Ungar is a Senior Scientist in the Child Health Evaluative Sciences Department at the Hospital for Sick Children Research Institute and Professor at the Institute of Health Policy, Management and Evaluation at the University of Toronto in Canada. She holds the Canada Research Chair in Economic Evaluation and Technology Assessment in Child Health. In 2007 she founded TASK (Technology Assessment at Sick Kids), where she and her team conduct research applying health economic methods to child health and are responsible for maintaining the PEDE database, a user-friendly online database of pediatric economic evaluations published since 1980, used by health technology assessment agencies around the world. Her book, Economic Evaluation in Child Health, was published by Oxford University Press in 2010.
VOS: Developing health economic models for children is a challenge. Can you discuss some of the major roadblocks you face in your work?
Wendy Ungar: When conducting economic evaluations in children, it’s not simply a matter of including age as a variable. Differences between child and adult health must be recognized in terms of developmental vulnerability, dependency, unique patterns of health resource use, and unique patterns of morbidity and mortality. These aspects must be considered when designing an economic evaluation in child health. Key challenges to conducting these studies include:
• The inability to measure preferences for health states in infants, toddlers, and very young children, and reliance on proxies
• The need to consider changes in resource use and health state preferences for different age groups as children mature (ie, neonates, infants, toddlers, school children, adolescents)
• Modeling costs and health consequences over the lifetime
• Using different approaches or instruments for generating utilities in different age groups
• Effects of discount rates when upfront costs are high and benefits are deferred or accrue over many decades
• The need to incorporate the costs and consequences of spillover effects on caregivers and family members
VOS: At what age do you believe a child can begin to complete a multi-attribute instrument?
WU: A major problem is that most preference-based instruments used for indirect elicitation of utilities are implicitly designed for adults. They include quality-of-life attributes that are meaningful to adults but not necessarily relevant for children. They may also have used adults to derive the underlying utility weights. There are some instruments, such as the HUI, CHU-9D, and EQ-5D-Y, that are attracting attention for use in children. However, their classification systems may or may not reflect attributes relevant to child health. Furthermore, children may not have been used to establish underlying utility weights. Even with these child-centric instruments, children younger than 8 years of age typically cannot self-assess and a proxy is needed to provide the responses.
" As QALYs cannot be validly or reliably generated for infants, toddlers, and very young children (under 6 years of age), this constitutes a major evidence gap." —Wendy J. Ungar, MSc, PhD
VOS: What are the potential drawbacks of utilizing adult proxy replacements to fill instruments?
WU: While adults (most often a parent) are accurate reporters for a child’s resource use, their proxy responses for dimensions of quality of life have been found to be poorly correlated with a child’s responses in many studies. Parents are better reporters for observable attributes such as physical activity and worse reporters for more abstract attributes related to mood and cognition. Parents may also imbue their proxy responses with their own subjective perceptions of their child’s health state. Proxy responses should not be pooled with self-assessed responses when calculating utility weights.
VOS: How can you build economic models when you don’t have any data from generic instruments like the EQ-5D?
WU: Like all model building, it’s a question of weighing uncertainty against the demand for evidence needed to inform funding recommendations. Health technology assessments (HTAs) and economic evaluations in child health can use cost-effectiveness analysis with natural health outcomes in addition to or instead of a cost-utility analysis when quality of life-years (QALYs) cannot be generated. They may also utilize shorter time horizons. HTA agencies require guidance on alternative modeling approaches and how to evaluate health economic evidence in children when QALYs cannot be generated. Guideline producers must update their guidelines to explicitly consider the methodologic challenges of performing economic evaluations in child health.
VOS: Can you name some study evidence gaps for preference-based measures of children’s health?
WU: As QALYs cannot be validly or reliably generated for infants, toddlers, and very young children (under 6 years of age), this constitutes a major evidence gap. In addition, sound lifetime models for many chronic childhood conditions are lacking. Few health economic evaluations in child health include a societal perspective, which is essential to capture spillover effects such as caregiver productivity losses and caregiving-related QALY losses. Promising research is ongoing to expand the methods used to generate health state utilities for pediatric conditions, such as the use of parent-child dyad elicitation and discrete choice experiments, as well as methods that circumvent the need for these instruments or cost-utility analyses entirely (ie, willingness-to-pay) via discrete choice experiments and net benefit approaches.
"Agencies that produce guidelines around the world must further explicitly recognize the methodologic challenges in conducting economic evaluations in child health and provide guidance on alternative approaches." —Wendy J. Ungar, MSc, PhD
VOS: Should QALY gains by children and adolescents, as compared to adults, have different values or carry more weight, in your opinion?
WU: This is a great question. Many clinical and funding decision-making bodies inherently value health improvements in children highly as a reflection of altruistic societal beliefs that aim to protect the most vulnerable. This has been borne out by numerous stated preference studies revealing that individuals place greater weight on health improvements in children. This can be difficult to operationalize in cost-utility analyses however, especially given what is stated above with regard to our ability to generate valid QALYs in children. The National Institute for Health and Care Excellence in the United Kingdom has done the most work in this area, examining alternative willingness-to-pay thresholds that may favor investments for more vulnerable populations. Another approach is to rely on the multidisciplinary HTA framework used for funding decision making that places as much (or more) weight on the social, legal, and ethical implications of a particular funding decision, as the incremental cost-effectiveness ratio. Further, many HTA agencies engage directly with patients, families, and members of the public so that their values and preferences regarding the population expected to benefit are directly considered in the funding deliberation.
VOS: Are there any specific best practices or challenges to your country/region that you would like to share that may benefit readers?
WU: The second Washington panel updated their guidelines in 2016 to include a societal perspective in the reference case and to explicitly recommend inclusion of caregiver productivity costs as well as costs occurring outside the health sector (ie, education and social and community services), which commonly offer programs that benefit children. Agencies that produce guidelines around the world must further explicitly recognize the methodologic challenges in conducting economic evaluations in child health and provide guidance on alternative approaches.
VOS: Is there any other topic or issue related to health-related quality of life in children that you would like to highlight for our readers?
WU: The news is not all bad! The volume of published pediatric cost-utility analyses, as indexed in our PEDE database, has grown on average by 23% annually since 2003. In 2020, cost-utility analysis was the most common analytic technique in child health economic evaluation (Figure).
More and more health economic researchers are attracted to the field of child health and exciting research is ongoing, examining alternative methods to generate health state utilities, model building, and capturing spillover costs and consequences. For further reading, see the suggested bibliography below.
Suggested Bibliography
Chen G, Ratcliffe J. A review of the development and application of generic multi-attribute utility instruments for paediatric populations. Pharmacoeconomics. 2015;33(10):1013-1028.
Kromm SK, Bethell J, Kraglund F, et al. Characteristics and quality of pediatric cost-utility analyses. Qual Life Res. 2012;21(8):1315-1325.
Kwon J, Kim SW, Ungar WJ, Tsiplova K, Madan J, Petrou S. A systematic review and meta-analysis of childhood health utilities. Med Decis Making. 2018;38(3):277-305.
Kwon J, Kim SW, Ungar WJ, Tsiplova K, Madan J, Petrou S. Patterns, trends and methodological associations in the measurement and valuation of childhood health utilities. Qual Life Res. 2019;28(7):1705-1724.
Lamb A, Murray A, Lovett R. The challenges of measuring and valuing quality of life in preschool children: a retrospective review of NICE appraisals. Children. 2021;8(9):765.
Matza LS, Swensen AR, Flood EM, Secnik K, Leidy NK. Assessment of health-related quality of life in children: a review of conceptual, methodological, and regulatory issues. Value Health. 2004;7(1):79-92.
Oliveira C, de Silva NT, Ungar WJ, et al. Health-related quality of life in neonates and infants: a conceptual framework. Qual Life Res. 2020;29(5):1159-1168.
Ratcliffe J, Huynh E, Stevens K, Brazier J, Sawyer M, Flynn T. Nothing about us without us? A comparison of adolescent and adult health-state values for the Child Health Utility-9D using profile case best-worst scaling. Health Econ. 2016;25(4):486-496.
Stevens K. Valuation of the Child Health Utility 9D index. Pharmacoeconomics. 2012;30(8):729-747.
Stevens K, Ratcliffe J. Measuring and valuing health benefits for economic evaluation in adolescence: an assessment of the practicality and validity of the Child Health Utility 9D in the Australian adolescent population. Value Health. 2012;15(8):1092-1099.
Ungar WJ. Economic Evaluation in Child Health. Oxford, United Kingdom: Oxford University Press; 2010.
Ungar WJ. Challenges in health state valuation in paediatric economic evaluation: are QALYs contraindicated? Pharmacoeconomics. 2011;29(8):641-652.
Ungar WJ, Boydell K, Dell S, et al. A parent-child dyad approach to the assessment of health status and health-related quality of life in children with asthma. Pharmacoeconomics. 2012;30(8):697-712.
