Critical Considerations in Assessing the Value of Medical Devices: Are Health Technology Assessment Bodies Falling Behind?
Andreas Freitag, MD, MSc, Cytel, London, United Kingdom; Andrea Schmetz, MBA, BIOTRONIK SE & Co KG, Berlin, Germany; and Grammati Sarri, PhD, Cytel, London, United Kingdom
Introduction
Evidence generation activities and healthcare decision making for medical devices (MDs) are associated with unique challenges. Scarcity of high-quality trial data and confounding factors may impact a manufacturer’s ability to demonstrate an MD’s benefit on clinical outcomes.
There has been a recent shift toward higher evidentiary standards in the regulatory setting for MDs. For example, in 2021, the European Union (EU)’s Medical Devices Regulation (MDR) restricted the use of evidence from existing devices, switched to a formal evaluation of the new device, and established increased MD assessment requirements.1
"The gap in the availability and quality of clinical evidence to support medical device claims compared to pharmaceuticals has not been formally recognized by decision makers."
Changes to regulatory laws across jurisdictions will likely impact how an MD is evaluated by health technology assessment (HTA) bodies, resulting in methodological changes in evidence requirements for reimbursement submissions. The EU’s Joint Clinical Assessment (JCA) Programme is one prominent example of how reimbursement processes are being updated to follow suit with regulatory changes.2-4
The recent regulatory and reimbursement trend for MDs to increasingly align with those for pharmaceuticals ignores the fundamental differences in the way MDs and pharmaceuticals are developed and used (Figure 1). This puts manufacturers at risk of being unable to meet these requirements, potentially barring patient access to effective and safe MDs.
Figure 1. Challenge of generating evidence for medical devices
Unique challenges for MD evidence generation
Regulatory bodies agree that evidence generation activities for MDs face unique challenges such as the inability to blind trials, how to assess investigator bias in outcome reporting due to increased training and experience of staff with the device, and the rapid development of software inherent to most high-risk devices that conflict with the time needed to conduct clinical trials. A less explored factor is the impact of continuous recalibration of programmable devices and its influence on clinical outcomes.
Current regulatory frameworks in the United States,5,6 European Union,1 United Kingdom,7-9 and Canada10,11 provide clear guidance (although the level of detail varies) on evidence requirements and considerations of the challenges in generating reliable outcomes for MDs and the impact in decision making (Figure 2). Generally, these guidelines are specifically aimed at high-risk MDs (ie, class III or above), yet they share many similarities with requirements for pharmaceuticals. For example, it is recommended that evidence should be identified from different sources and be further collected as needed. However, it is recognized that conducting randomized controlled trials (RCT) for MDs can be difficult due to ethical and practical reasons; therefore, US and EU frameworks specifically allow for submission of other evidence (eg, uncontrolled studies if safety and effectiveness of the MD can be reasonably established). Data from another manufacturer’s device with similar operational technicalities and mode of action can also be used as supporting evidence if equivalence can be established (although results of a recent study showed that class I recalls in devices seem to beget class I recalls in descendant devices).12
Figure 2. Main themes of medical device evidentiary requirements across regulatory guidance documents
Similar challenges exist in reimbursement settings, with HTA bodies rarely distinguishing between MDs and pharmaceuticals in evidence and submission requirements. However, some positive steps have started emerging; for example, recent UK strategy acknowledges many of the evidence-generation challenges highlighted in this paper to ensure timely access to safe and effective medical technologies.13 An overview of some of these themes in MD evidence generation is discussed below.
1. Timely MD access requires high-quality clinical data
The gap in the availability and quality of clinical evidence to support MD claims compared to pharmaceuticals has not been formally recognized by decision makers. MD manufacturers are generally required to conduct thorough literature reviews and robust primary trial data collection following a similar evidence hierarchy typically used for pharmaceuticals, with emphasis on the need for RCTs. However, considerable challenges in conducting RCTs or real-world studies for MDs may affect the quality and confidence in the clinical data outputs: the inability to follow blinding processes, need for informed consent about invasive devices, or difficulties in protocol standardization between hospital settings. The increase in support of evidence outside RCTs and the development of MD-specific guidance documents are progress; in rare cases, expert opinion and patient testimonies can inform decision making.14,15 MD-specific quality tools are also needed to provide transparency in the bias assessment.
2. Data collection should aim for clinical relevance
The traditional focus on RCT data and the clinical relevance of MD data such as intended benefit for the targeted population, quantifiable benefit linked to product performance, or assessing the direct impact of device-user interaction to patient-relevant outcomes may disregard the nature of research (effectiveness and safety) questions for MDs. For example, MD-specific guidance does not account for continuous improvement to MD specifications and models, unclear target population (devices may be used across different indications), and parallel use of different MDs in clinical practice. Differences in practice across countries, and even between medical centers, are difficult to fully address through a clinical trial. Issues around data transportability and recent methodologies to account for these differences such as linked analysis accounting for product performance have rarely been applied in MD assessments. Therefore, the expansion of real-world evidence (RWE) frameworks for MDs with emphasis on issues around data storage, collection, and analysis will guide manufacturers to sufficiently account for differences in data sources and explore biases that will support manufacturers in demonstrating a device’s clinical relevance for decision making.
3. Overcoming current operational obstacles related to submission timelines and clearer guidelines may improve data quality
Without clear expectations about the accepted quality level for MD assessment, manufacturers may struggle to provide timely, robust evidence at such an early stage in the MD’s lifecycle. Some HTA agencies require submission of the full evidence dossiers for MDs within 2 to 6 weeks following publication of the final scope, whereas timelines for pharmaceuticals tend to be longer (eg, around 60 days in the United Kingdom).16
"The expansion of real-world evidence frameworks for medical devices will support manufacturers in demonstrating a device’s clinical relevance for decision making."
The need for evidence generation through continuous data collection in the postmarketing setting is higher for MDs due to challenges in data collection through the first assessment. This reassessment schedule may be triggered by the risks and benefits associated with each new device. However, current processes do not provide coherent, time-sensitive plans for proactive evaluation of new MD evidence. As part of the MDR, the European Database on MDs (EUDAMED) is being expanded to strengthen surveillance, transparency, and monitoring the safety and performance of available MDs; however, it appears behind schedule in terms of capturing all devices (independent of class).17 Agencies in other jurisdictions need to update processes to address MD-specific considerations, using EUDAMED as an example.18
4. Increasing transparency in decision making will elevate MD evidentiary standards and safeguard consistency in the process
MD evaluation processes vary greatly within and between countries. Given the lack of clarity in decision-making requirements and challenges in generating and robustly synthesizing MD evidence, it is not surprising that HTA and payer decisions lack transparency.
Clarity around the type of evidence needed will streamline the HTA process, allowing manufacturers to weigh evidence generation activities as early as the study design phase which, in turn, will eliminate some uncertainties later. Likewise, with the increased acceptance of RWE in HTA submissions, manufacturers could plan for data collection methods (eg, MD registries, data linkage) that aim to improve the quality of the evidence from these study types, help address MD performance evolution, and capture real-life patient experience. Novel methodologies, such as living systematic reviews, will transparently present the evidence generated across the lifecycle of an MD and its comparators, eliminating the traditional, more static assessment of an MD. Results from pilot programs that encourage an innovative assessment approach for promising MDs (such as the early value assessment in the United Kingdom) may help showcase the potential of gathering RWE for MDs within a live environment while enabling greater access to patients.19 Recognizing and addressing these challenges for MDs will be vital for efforts like the EU JCA to provide acceptable, timely access for patients and incentivize evidence generation for MDs.
"Given the lack of clarity in decision-making requirements and challenges in generating and robustly synthesizing medical device evidence, it is not surprising that HTA and payer decisions lack transparency."
Conclusion
There is a recognition that MDs and pharmaceuticals have little in common regarding evidence generation from operational and methodological standpoints. Regulatory agencies have provided clear guidance to reflect this situation, but HTA bodies are lagging in this regard. It is therefore vital that reimbursement decision makers and other payers consider the development of MD-specific assessment frameworks that truly represent the potential of an MD to improve patient outcomes and contribute to healthcare sustainability.
References
1. European Parliament and European Council. Regulation (EU) 2017/745 of the European Parliament and of the Council of 5 april 2017 on medical devices, amending Directive 2001/83/EC, Regulation (EC) No 178/2002 and Regulation (EC) No 1223/2009 and repealing Council Directives 90/385/EEC and 93/42/EEC. Published 2017. Accessed July 25, 2023. https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX%3A02017R0745-20170505
2. EUnetHTA. Future Model of Cooperation (FMC) – White Paper. Published September 8, 2021. Accessed July 25, 2023. https://www.eunethta.eu/future-model-of-cooperation-fmc-white-paper/
3. EUnetHTA. Joint HTA Work - Joint Scientific Consultation and Joint Clinical Assessment. Accessed July 25, 2023. https://www.eunethta.eu/jointhtawork/
4. European Parliament and European Council. Regulation (EU) 2021/2282 of the European Parliament and of the Council of 15 December 2021 on Health Technology Assessment and Amending Directive 2011/24/EU. Published December 22, 2021. Accessed July 25, 2023. https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX%3A32021R2282
5. Food and Drug Administration. De Novo Classification Process (Evaluation of Automatic Class III Designation)–Guidance for Industry and Food and Drug Administration Staff. Published October 2021. Accessed July 25, 2023. https://www.fda.gov/regulatory-information/search-fda-guidance-documents/de-novo-classification-process-evaluation-automatic-class-iii-designation
6. National Archives and Records Administration. 21 CRF Part 860 - Medical Devices Classification Procedures. 2023. Amended June 22, 2023. Accessed July 25, 2023. https://www.ecfr.gov/current/title-21/chapter-I/subchapter-H/part-860?toc=1
7. Medicines and Healthcare Products Regulatory Agency. Regulating Medical Devices in the UK. Published December 31, 2020. Updated July 1, 2023. Accessed July 25, 2023. https://www.gov.uk/guidance/regulating-medical-devices-in-the-uk
8. Medicines and Healthcare Products Regulatory Agency. Medical Devices: Conformity Assessment and the UKCA Mark. Published December 31, 2020. Accessed July 25, 2023. https://www.gov.uk/guidance/medical-devices-conformity-assessment-and-the-ukca-mark
9. Medicines and Healthcare products Regulatory Agency. Clinical Investigations of Medical Devices – Guidance for Manufacturers. Published May 2021. Accessed July 25, 2023. https://assets.publishing.service.gov.uk/government/uploads/system/uploads/attachment_data/file/1097796/Guidance_for_mfrs_on_clinical_investigations-May_2021.pdf
10. Health Canada. Health Technology Assessment and Optimal Use: Medical Devices; Diagnostic Tests; Medical, Surgical, and Dental Procedures. Published November 2015. Accessed July 25, 2023. https://www.cadth.ca/sites/default/files/pdf/HTA_OU_Topic_ID_and_Prioritization_Process.pdf
11. Minister of Justice (Canadian Government). Medical Devices Regulation. Last amended February 22, 2023. Accessed July 25, 2023. https://laws-lois.justice.gc.ca/PDF/SOR-98-282.pdf
12. Kadakia KT, Dhruva SS, Caraballo C, Ross JS, Krumholz HM. Use of recalled devices in new device authorizations under the US Food and Drug Administration’s 510 (k) pathway and risk of subsequent recalls. JAMA. 2023;329(2):136-143.
13. UK Department of Health & Social Care. Medical Technology Strategy. Published February 3, 2023. Accessed July 25, 2023. https://www.gov.uk/government/publications/medical-technology-strategy
14. National Institute for Health and Care Excellence. Medical Technologies Evaluation Programme Methods Guide. Published August 21, 2017. Accessed July 25, 2023. https://www.nice.org.uk/process/pmg33/resources/medical-technologies-evaluation-programme-methods-guide-pdf-72286774205893
15. National Institute for Health and Care Excellence. NICE Health Technology Evaluations: The Manual. Published January 31, 2022. Accessed July 25, 2023. https://www.nice.org.uk/process/pmg36/resources/nice-health-technology-evaluations-the-manual-pdf-72286779244741
16. National Institute for Health and Care Excellence. Changes we’re making to health technology evaluation. Accessed July 25, 2023. https://www.nice.org.uk/about/what-we-do/our-programmes/nice-guidance/nice-technology-appraisal-guidance/changes-to-health-technology-evaluation
17. European Commission. EUDAMED–European Database on Medical Devices. Accessed July 25, 2023. https://ec.europa.eu/tools/eudamed/#/screen/home
18. MedTech Dive. European Commission targets spring of 2024 for fully functional Eudamed database. Published July 11, 2022. Accessed January 31, 2023. https://www.medtechdive.com/news/european-commission-eudamed-timeline-2024/626917/
19. National Institute for Health and Care Excellence. Early Value Assessment (EVA) for Medtech. Accessed February 2, 2023. https://www.nice.org.uk/about/what-we-do/eva-for-medtech
