Value Assessment of Combination Digital Health Technologies Under New NICE Evidence Standards Framework

Xenia F. Sitavu-Radu, PhD, MSc, IQVIA, London, England, United Kingdom; Cátia M. Modesto S, PharmD, IQVIA, Lisbon, Portugal

Background

Digital health technologies (DHTs) are defined by the World Health Organization (WHO) as “the field of knowledge and practice associated with the development and use of digital technologies to improve health”¹ and have been increasingly used in clinical practice.² DHTs cover a wide range of technologies, such as mobile apps, genomics, artificial intelligence (AI), machine learning (ML), wearable devices, and telemedicine.^3,4 The implementation of DHTs has the potential to provide significant benefits both to patients and healthcare systems. Specifically, the combined use of drugs/medical devices with a digital component (combination DHTs [cDHTs]), such as an insulin pump monitored by an application or an ingestible sensor embedded in a tablet, has seen a rising trend in the healthcare industry.^3,5,6

One of the current challenges in the DHTs landscape is related to the lack of established and well-characterized regulatory/assessment frameworks and reimbursement pathways. Some progress has been made with several countries (such as France, Germany, and the United Kingdom) establishing reimbursement pathways for DHTs, although with significant variability.⁴

To address the challenges of assessing these innovative technologies under the traditional frameworks, the National Institute for Health and Care Excellence (NICE), the Health Technology Assessment (HTA) body in the United Kingdom, published the Evidence Standards Framework (ESF) for the assessment of DHTs in March 2019 (updated in August 2022). The ESF aims to ensure that new DHTs are clinically effective and add value to the healthcare system, and that these are assessed consistently across the National Health System (NHS). This framework is mostly directed at decision makers and companies who develop the DHTs that will later on be considered for use and commissioning in the health system. However, it does not have a mandatory statute and it does not constitute a NICE evaluation process.^7-9

NICE’s ESF describes the type and level of evidence that different categories of DHTs (defined in 3 tiers, based on the DHT’s potential risk to the user and to the system) should be able to demonstrate and comprises 21 nonmandated standards across 5 areas: (1) design factors, (2) describing value, (3) demonstrating performance, (4) delivering value, and
(5) deployment considerations. The majority of the standards (17 out of 21) are applicable to all DHTs, regardless of tier.^8,9

Figure. Five groups of evidence standards and their applicability to different tiers

This article evaluated the evidence alignment of recently assessed cDHTs with the ESF, as well as identified key assessment criteria employed by NICE, providing insights into the use of the ESF for stakeholders involved in the development and assessment of cDHTs.

Methodology

In order to identify the cDHTs assessed by NICE since the publication of the ESF, a search was conducted using the HTA Accelerator™, a comprehensive database that includes publicly available HTA reports. The search was conducted on March 28, 2023, and restricted to assessments published since 2019 (later updated on July 22, 2024, restricted to assessments published since March 28, 2023). The inclusion criteria consisted of device interventions assessed by NICE. Search results were then hand-selected to include assessments of cDHTs only. Supporting data were extracted for the intervention details, clinical and economic evidence, agency critique, and recommendations. For completeness, a hand-search of published information on NICE’s website for each cDHT was also conducted. Lastly, the available information from each appraisal was compared against each of the 21 standards of NICE’s ESF.

"The implementation of DHTs has the potential to provide significant benefits both to patients and healthcare systems."

Results

In the first search (March 2023), 75 assessments were identified, and 25 assessments were identified in the second search (July 2024), with a total of 9 cDHTs meeting the inclusion criteria, of which 5 received a positive recommendation (with restrictions).^*  See Table 1.

Table 1. Data extracted for cDHTs assessed by NICE since the publication of the ESF

Clinical evidence

All assessments included a systematic review identifying relevant clinical evidence on the cDHTs under evaluation, with randomized controlled trials (RCTs) available for 6 of the 9 assessments (with ongoing RCTs for one of them), and observational/survey data only for the remaining 3 assessments (of which one had ongoing RCTs). Most (8 out of 9 assessments) of the identified evidence included UK-based data. In addition, NICE established an evidence generation plan for 2 of the assessments, which provides information on the main evidence gaps, ongoing studies, and how these gaps can be addressed by real-world evidence (RWE).

Although RWE can be used to demonstrate that cDHTs can meet their claimed benefits in the UK clinical practice, only 3 assessments included this.

Economic evidence

All 9 assessments included an economic analysis, with 7 including a cost-effectiveness analysis, one a cost minimization model, and one a cost comparison analysis, whereas only 3 included a budget impact analysis. Although a budget impact analysis is applicable to all DHTs, a cost-effectiveness analysis is applicable to DHTs with higher financial risk only (those with substantial implementation costs). The potential cost-effective use of NHS resources was noted; however, uncertainties were highlighted in terms of the clinical data (namely the lack of robustness and high risk of bias, uncertainty in the size, and longevity of claimed benefits, lack of data in the United Kingdom or the target population), model inputs or resource use, making it difficult for NICE to draw firm conclusions about the benefits of routine use of the technologies in the NHS. Further data collection relevant to the NHS setting, as well as price reductions, were the most common recommendations.

ESF standards requirements

Among the 21 ESF evidence standards, the ones most commonly aligned to were the availability of clinical evidence and economic analyses, as well as defining the level of professional oversight, the target population, and current and proposed treatment pathway.

Conversely, evidence to support other requirements, such as considering environmental sustainability, establishing good data practices in the DHT design, providing a budget impact analysis, and ensuring transparency about deployment requirements, was sporadically available.

"One of the current challenges in the DHTs landscape is related to the lack of established and well-characterized regulatory/assessment frameworks and reimbursement pathways."

NICE recommendations

The 5 assessments receiving a positive recommendation demonstrated promising benefits for patients and potential cost savings for the NHS, although considerable uncertainties in the clinical and/or economic evidence were highlighted. Thus, these cDHTs were conditionally recommended for use in the NHS, with the collection of further evidence (such as robust evidence generation on the cDHTs clinical benefits, as well as on its testing, interpretation and accuracy, real-world data collection, data collection on resource use, and data relevant for the UK practice) being a common condition in all assessments.

The 4 assessments receiving a negative recommendation were deemed as not having sufficient evidence to support its routine use in the NHS. Recommendations for further research have been made for all assessments, namely referring to accuracy and failure tests, patient experience studies, data collection assessing the impact of implementing a cDHT, and more robust clinical evidence generation.

Conclusion

The publication of the ESF is expected to fill a gap in the innovative and challenging setting of DHTs by supporting both developers and evaluators with a HTA framework and a risk-proportional set of evidence standards.

Nine cDHTs assessments have been published by NICE since the ESF publication, which supported the current analysis. Overall, the analyzed assessments were partially aligned with the recent framework, although some standards (mostly related with the DHT design, deployment, and environmental sustainability) were only sporadically available, which may be due to the sensitive nature of such information. Moreover, while most of the assessments started after the publication of the ESF, it is plausible that the technology development did not fully incorporate the ESF standards into account.

"Although a high variability has been observed in terms of evidence, the availability of robust clinical and economic data, reflecting the cDHT’s accuracy, use in the UK setting and the target population, and its potential for improved outcomes and costs/resource use in the NHS, were common elements among the positively recommended technologies."

Although a high variability has been observed in terms of evidence, the availability of robust clinical and economic data, reflecting the cDHT’s accuracy, use in the UK setting and the target population, and its potential for improved outcomes and costs/resource use in the NHS, were common elements among the positively recommended technologies. Where feasible, RCTs were the gold-standard study designs for the clinical assessment of cDHTs; however, alternative study designs were also considered acceptable in some assessments, with the collection of RWE being encouraged as part of evidence-generation recommendations. Finally, developers should also consider the level of risk under which the cDHT would be assessed when developing their evidence-generation plan, as higher risk tools will require a more rigorous evidence package.

This analysis provides useful insights into the current use of ESF and key criteria involved in successful cDHTs assessments. Moreover, it highlights the variability in the available evidence for cDHTs and the need for close collaboration between stakeholders to ensure clear requirements for the successful integration of cDHTs into healthcare systems are achieved. 

References

1. Global strategy on digital health 2020-2025. World Health Organization. https://www.who.int/publications/i/item/9789240020924. Published August 18, 2021. Accessed August 21, 2024.
2. Mathias R, McCulloch P, Chalkidou A, Gilbert S. Digital health technologies need regulation and reimbursement that enable flexible interactions and groupings. NPJ Digit Medicine. 2024;7(1):148. doi:10.1038/s41746-024-01147-z
3. Yeung AWK, Torkamani A, Butte AJ, et al. The promise of digital healthcare technologies. Front Public Health. 2023;11:1196596. doi:10.3389/fpubh.2023.1196596
4. van Kessel R, Srivastava D, Kyriopoulos I, et al. Digital health reimbursement strategies of 8 European countries and Israel: scoping review and policy mapping. JMIR Mhealth Uhealth. 2023;11:e49003. doi:10.2196/49003
5. Vallejos X, Wu C. Digital medicine: innovative drug-device combination as new measure of medication adherence. J Pharm Technol. 2017;33(4):137-139. doi:10.1177/8755122517704212
6. Combined health technologies: combining pharma, medtech and tech to bring innovations to the patients. Medtech & Pharma Platform. https://medtech-pharma.com/combined-health-technologies/. Accessed August 21, 2024.
7. Unsworth H, Dillon B, Collinson L, et al. The NICE Evidence Standards Framework for digital health and care technologies: developing and maintaining an innovative evidence framework with global impact. Digit Health. 2021;7:20552076211018617. doi:10.1177/20552076211018617
8. Evidence standards framework for digital health technologies: user guide. National Institute for Health and Care Excellence. https://www.nice.org.uk/corporate/ecd7/resources/evidence-standards-framework-for-digital-health-technologies-user-guide-pdf-11696158815685. Published March 1, 2019. Updated August 9, 2022. Accessed August 21, 2024.
9. Evidence standards framework for digital health technologies. National Insitute for Health and Care Excellence. https://www.nice.org.uk/corporate/ecd7/resources/evidence-standards-framework-for-digital-health-technologies-pdf-1124017457605. Published December 10, 2018. Updated August 9, 2022. Accessed August 21, 2024.

* Of note, 1 of the assessments captured in the search (HTE 11, Artificial intelligence technologies to aid contouring for radiotherapy treatment planning: early value assessment) was excluded due to the fact that it assessed 11 artificial intelligence (AI) technologies, which consisted of standalone softwares that did not meet the inclusion criteria for a cDHT. However, 2 out of the 11 technologies assessed (MRCAT Prostate plus Auto-contouring and RayStation) combined the AI functionality with a device, which could therefore fit into the cDHT category. The HTE11 guidance issued a positive recommendation for 9 of the 11 technologies (including the 2 cDHTs above) to be used in the NHS while further evidence is generated.