Using Patient Preference Information in Medical Device Regulatory Decisions: Benefit-Risk and Beyond
As the use of patient preference information grows in the healthcare industry, ISPOR and the US Food and Drug Administration (FDA) Center for Devices and Radiologic Health cosponsored a virtual meeting in September on “Using Patient Preference Information in Medical Device Regulatory Decisions: Benefit-Risk and Beyond.” Featuring experts from the FDA, ISPOR, medical device manufacturers, health economics, healthcare, and patient groups, the 4 sessions in the Summit clarified what patient preference information is and presented case studies for the use of patient preference information in decision-making processes, methodologic issues for patient preference information, and future opportunities for the implementation and use of patient preference information beyond the regulatory space.
“We do not want to forget that qualitative assessments can also have a role in patient preference information and provide valuable information for decision making.” —Brett Hauber, PhD
Background on Patient Preference Information
Brett Hauber, PhD, Senior Economist/Senior Fellow at RTI Health Solutions and Anindita “Annie” Saha, Director, Partnerships to Advance Innovation and Regulatory Science, Center for Devices and Radiologic Health, discussed what patient preference information is and is not, and what the FDA is looking for from manufacturers when they submit patient preference information as part of their application.
Although most of the discussions at the seminar focused on how patient preference information can be used in generating quantitative assessments, “We do not want to forget that qualitative assessments can also have a role in patient preference information and provide valuable information for decision making,” Hauber stated. The “relative nature” of preference is important, because it not only includes the “good” things that are desirable, but the “bad” things that are acceptable. “Both of these are components of preferences that matter,” Hauber said. In looking at preferences, the focus needs to be on the features, both positive and negative, that differ among the alternatives.
According to Saha, patient input can help inform product design, clinical trial development, and also be used to identify specific patient populations that prefer the benefit-risk for specific treatments or to communicate treatment preferences. Additionally, patient input can raise or confirm problems that may exist with specific products and bring to light new considerations to inform FDA’s thinking on current issues.
Above all, submission of patient preference information is voluntary and does not have to be part of every medical device application as it may not be relevant to all device types, Saha said. She continued, “Patient preference information could be useful when usage or decisions by patients or healthcare professionals are preference-sensitive. Some examples of preference-sensitive (decisions) include where there might be a direct patient interface, where the device could directly affect health-related quality of life, for certain lifesaving high-risk devices, or maybe in an area with a new technology.”
Saha suggested that manufacturers interested in including patient preference information approach the FDA through a presubmission to discuss the regulatory relevance, research question, survey participants, survey design, and analysis approach.
"Patient input can help inform product design, clinical trial development, and also be used to identify specific patient populations that prefer the benefit-risk for specific treatments or to communicate treatment preferences." —Annie” Saha
Case Studies Demonstrating Use of Patient Preference Information
Dan Harfe, Vice President, Regulatory, Quality and Strategy, Smith+Nephew (S+N), described a study performed by S+N’s ENT business in which preference testing was used very early on in the regulatory process, when designing a protocol for a pivotal study for a combination product premarket approval. The medical product was an alternative to tympanostomy under general anesthesia for treating young children with otitis media (inflammation of the middle ear). The combination product (device-drug system) enables tympanostomy placement in a doctor’s office using local anesthesia. While avoiding the problems of pediatric surgery under general anesthesia and the stress and worry this gives to parents, the alternative procedure introduced its own challenges. “Toddlers typically do not like you to do things to them,” Harfe noted. And while tympanostomies under general anesthesia have a virtually 100% success rate, the alternative would have a lower rate of success, a common characteristic of pediatric procedures when general anesthesia is not used.
The question that arose was whether or not parents would accept a lower rate of success with the novel in-office tympanostomy procedure as compared to the traditional tympanostomy using general anesthesia. To determine an acceptable success rate, S+N conducted qualitative interviews, followed by a preference study, enrolling parents. Completing the preference study and negotiating the acceptability of the data with FDA took longer than initially anticipated. “I suspect that would have gone a lot quicker and smoother if we had engaged with the agency ahead of time,” Harfe said. He noted 4 lessons from the experience. First, make sure to send your study to the right experts at the FDA. Second, treat your preference study like a clinical study from a timeline, budget, and statistical rigor perspective. Third, work with the right external partners if you do not have the expertise within your own company. And fourth, engage your work with the FDA early in the process. While it may seem like a straightforward decision to gain alignment with FDA ahead of running a preference study, for a start-up with limited financial resources and time, it is not always a simple choice to make.
Christine Poulos, PhD, Senior Economist and Global Head, Health Preference Assessment, RTI Health Solutions, described her experiences using patient preference information to support a premarket approval for PneumRx endobronchial coils, an emphysema product that could provide an alternative to lung volume reduction surgery. The preference study was conducted to support the product’s benefit-risk assessment, so it was performed later in the development process. The study used a discrete choice experiment, with the research protocol submitted to the FDA as a formal pre-submission before the survey pretest was completed. There was an in-person meeting to discuss the FDA’s comments on the presubmission, with no additional engagement until the study was completed and the results were submitted to the FDA. After that, there was a period of interactive review before the panel meeting.
Patients did not understand the clinical endpoints in the pivotal trial, which made it difficult for them to state a preference. Ultimately, the company was able to develop a patient preference information study based off a secondary patient-reported quality-of-life measure, which generated some noteworthy results. However, the advisory panel had fundamental concerns about the overall efficacy of the coils and ultimately voted to reject the premarket approval on that basis. Two stated lessons were: 1) manufacturers need to strike a balance between the level of engagement with the FDA during the study development period and the time it adds to the study timeline; and 2) there are still no guidelines in the literature or regulatory guidance for how to adapt a patient-reported outcome measure for use in a patient preference information study.
Todd Snell, Senior Vice President, Quality Assurance, Regulatory and Clinical Affairs, NxStage Medical of Fresenius Medical Care North America, reviewed NxStage’s experiences using patient preference information to expand the labeled indications for use of their home hemodialysis system. At the time the application was developed, home hemodialysis was underutilized. When the system, NxStage System One, was originally developed, the labeling stated that all treatment “must be observed by a trained and qualified person considered to be competent by the prescribing physician.” Snell pointed out that for many patients needing treatment 10 to 15 hours a week, also needing a competent observer was another burden that most could
not meet.
Exploratory discussions with the FDA revealed that the company needed to identify risk tolerance thresholds for experienced home hemodialysis patients who would be willing to perform solo home hemodialysis and also determine if experienced patients would perform it after considering the benefits and risks. The company found a surprisingly high risk tolerance among these patients for things such as death and needle dislodgment. Ultimately, the company was able to get the updated labeling approved, with patient preference information providing a way for NxStage to move its product forward. Snell says that when developing patient preference information studies, manufacturers should know their audience, understand their device’s risks, seek feedback prior to the study, and engage early with the FDA to leverage some of the tools the agency has in order to better communicate with patients.
Barry Liden, Vice President, Patient Engagement, Edwards Lifesciences, talked about the study Edwards Lifesciences performed for severe aortic stenosis. Anecdotally, patients voiced concerns about aortic valve replacement, which requires open-heart surgery, expressing a preference for transcatheter valve replacement, which does not require open-heart surgery and which patients considered vastly superior. The Centers for Medicare and Medicaid Services (CMS) and government payers outside the United States were looking only at clinical endpoints, primarily all-cause mortality at 12 months; in this, the 2 procedures were somewhat equivalent.
Edwards set out to design a patient preference information study to inform reimbursement, hoping to bring to the table “qualitative data that could help inform their decision-making process,” Liden said. To determine the attributes to study, the company looked at other patient preference studies, sought extra consultation with patients and clinicians, and did a clinical literature review. Instead of a discrete choice experiment, Edwards opted for an adapted swing-weighted study design.
Ultimately, the company found that patients were willing to tolerate a very high amount of risk to receive the benefit. When Edwards took the data to the CMS, while the agency appreciated the information and thought it was very helpful, “they really struggled with how to apply it to a coverage decision,” Liden said. The lesson learned, Liden said, was “talk to the people that you are going to be using the data with before you start the study.”
Edwards published the data from their preference study and it was picked up in a literature review by Ontario Health, a health technology assessment (HTA) body in Canada, which reviewed this as a part of their overall assessment. Using this patient preference data along with the clinical and economic evidence, Ontario Health made a recommendation to cover this therapy under an expanded indication to low-risk patients.
In the Q&A after the session, Liden said while it was just luck that Edwards’ study was picked up by Ontario Health, there are HTAs around the world quite interested in patient preference information, and ISPOR had done an assessment of which HTAs are looking at patient preference evidence and what kind of data they want to see. “Examples are CADTH in Canada, NICE in the United Kingdom, and Germany’s HTA. Coming to the table with preferential data, quantitative data, is even more robust and helpful to their decision-making process,” Liden said.
“We have more tools in that toolkit than just the DCE. And whatever the analysis that is chosen, we want to be mindful that it should be robust, it should address the research question, and it should be relative to the relevant medical [or] regulatory decision.” —David Gebben, PhD
Methodologic Issues in Patient Preference Information Studies
David Gebben, PhD, Assistant Professor, Calvin University, and formerly of Center for Devices and Radiologic Health, examined considerations for choosing a method to gather patient preference information. He recommended the document that the Medical Device Innovation Consortium has produced that summarizes some of the qualitative steps to be considered in a survey. Steps include identifying the relevant research question; defining the study results of interest; defining the preference elicitation method and study design; and making sure that the research question is aligned with the study’s objective.
The discrete choice experiment (DCE) method “is probably the most familiar, and the one that is probably the most commonly used,” Gebben said, because it allows for the evaluation of multiple attributes at once and can inform endpoint selection prior to clinical trials as well as benefit-risk analysis. Its drawback is that it is “cognitively burdensome” because respondents are evaluating multiple things at once. Alternatively, the threshold technique can be used in the same ways as DCE, but unlike DCE, it only evaluates one attribute at one time. Other methods include: (a) best-worst scaling, which could be used to inform the prioritization of the endpoint selection, especially earlier on in the product life cycle where it is uncertain which endpoints are the priority, and (b) swing weighting, which can be used with rare or hard-to-reach populations. “We have more tools in that toolkit than just the DCE,” Dr Gebben said. “And whatever the analysis that is chosen, we want to be mindful that it should be robust, it should address the research question, and it should be relative to the relevant medical [or] regulatory decision.”
Ryan Fischer, Senior Vice President, Community Engagement, Parent Project Muscular Dystrophy, discussed its experience with using different methodologies and preference research through its BRAVE initiative, the goal of which was to better quantify and understand how patients and caregivers think and feel about emerging therapies and living with Duchenne muscular dystrophy to better communicate to regulators and other stakeholders the preferences of patients and caregivers. “Patients are involved from the start to the finish developing the instruments, the research questions, attributes, and helping to interpret the results,” Fischer said.
Juan Marcos Gonzalez, PhD, Assistant Professor, Department of Population Health Sciences, Duke University School of Medicine, emphasized that preference data, not preference methods, must be evaluated to determine whether patient preference information is fit-for-purpose. While methods might have some inherent properties, decisions made during study implementation can have far greater impact on the fit of patient preference information. Evaluating fit requires considering at least 3 aspects of a patient preference information study, which he called “the 3 legs of a stool.” Gonzalez said, “We need to consider whether we ask the right questions to patients, whether we are making reasonable assumptions about the answers we get from the patients, and whether the data we collect supports the assumptions we are making about patients’ answers.” For example, instruments should consider both positive and negative framing of the preference elicitation questions. In addition, questions must be incentive-compatible to increase the chance that responses are preference revealing. Some important assumptions about patients’ responses include the form of the measurement error in patient preference instruments, and the type of preference heterogeneity in the data. Finally, support for the assumptions can be obtained within studies through response consistency checks, and across studies through meta-analyses.
In sharing what Janssen and the IMI PREFER public-private partnership have done with data from preference studies, Bennett Levitan, MD, PhD, Senior Director, Benefit-Risk Assessment, Global R&D Epidemiology, Janssen R&D Pharmaceutical Companies of Johnson & Johnson, said that the application of preference study results to clinical data is not always clear or straightforward. He outlined 3 broad classes of approaches to applying preference data: (1) assessments based on the preference study independently (eg, maximum acceptable risk); (2) assessments in which clinical and preference data are depicted together (eg, plots depicting both preference weights and rates); and (3) assessments in which the clinical and preference data are combined into summary metrics (eg, net clinical benefit, choice share). Levitan described a variety of approaches and how they vary in clarity, complexity, incorporation of population heterogeneity, software requirements, complexity of communication, and relevance to different types of decision makers. “In general, I recommend using the simplest approach that will address the research question, but often I end up using a combination of approaches,” he said, adding, “ … the real-world applications, taking into account heterogeneity, the variance, uncertainty are not always as straightforward as we would like.”
Implementing Patient Preference Information Beyond the Regulatory Space
First to tackle the implementation, collection, and use of patient preference information beyond the evaluation of product-level benefit-risk in the regulatory space was Dean Bruhn-Ding, Vice President, Regulatory Affairs and Quality Assurance, CVRx, Incorporated. Bruhn-Ding chairs a novel working group for the Medical Device Innovation Consortium, a project that was a first-of-its-kind collaboration, in which 6 industry sponsors collaborated on a patient preference information study with patients, the FDA, and Duke University preference experts to provide valuable heart failure patient preference information for all to use. The Heart Failure Patient Preference Study was developed to inform on a potential heart failure clinical trial design and provide a regulatory reference for the FDA. “Our challenge as a medical device industry is to use patient preference information studies across the medical device life cycle so that patient perspectives are infused into the entire ecosystem,” he said.
Ravi Jayadevappa, PhD, Research Associate Professor, Perelman School of Medicine, University of Pennsylvania, presented the results of another study, the Preferences for Prostate Cancer Care (PreProCare) tool, which was intended to help patients with prostate cancer assess their preferences for treatment choice in real clinical settings. The preference assessment intervention is a web-based tool that uses choice-based adaptive conjoint analysis. According to Jayadevappa, the intervention group reported higher satisfaction with their care, and higher satisfaction with their decision, but a lower regret across all timepoints, especially at their 12- and 24-month follow-up visits.
Melissa West, Acting Vice President for Research, Discovery, and Innovation, American Society of Nephrology Alliance for Kidney Health, talked about a patient preference initiative that was introduced earlier this year through a partnership with the FDA to develop a survey for a future wearable renal replacement therapy devices. “We are trying to think early about how can we bring the patients into the process, because there is not a wearable hemodialysis machine or peritoneal dialysis machine on the market right now, but we want to bring in this benefit-risk discussion earlier on in the process to ensure that we all really understand what elements of the product and the attributes are most important to them,” she said. Additionally, the group wants to bring patient preference information to payers.
Louis Jacques, MD, Chief Clinical Officer/Senior Vice President, ADVI, said there are still many questions about how to create policy around patient preference information, especially for payers. “Is one going to do a randomized study of every patient preference outcome before one then graduates into some other bucket where it might be used? I mean, that seems impractical,” Jacques says, “Yet we know from our own history that sometimes what seems intuitively appealing turns out, in fact, to be wrong.”
According to Jacques, payers can get frustrated when a pivotal trial misses a primary endpoint, but “then everyone engages in this post hoc data dredging to say, ‘hey, look, we were better on a subpart of SF-36 that we sort of serendipitously happened to collect.’ Well, congratulations. You now have a hypothesis. Go run a trial with that as your primary prespecified outcome and we might have some more interest in talking to you.”
Manufacturers may want to incorporate discussions about outcomes in the application process for Medicare and other payer coverage in IDE trials. “Medicare loves outcomes data that reflect the beneficiaries’ experience of disease, their experience of priorities, and their response to therapies,” Jacques said.
Conclusion
With nearly 2000 registrants from more than 85 countries, the Virtual ISPOR-FDA Summit 2020 demonstrated a growing interest in the field. Attendees showcased their enthusiasm for patient preference information by actively engaging with thought-provoking questions and discussions. Speakers expressed the importance of incorporating patient perspectives throughout the total product life cycle of medical devices. Thoughtful and well-designed patient preference studies can yield information that can be relevant in not only regulatory decision making, but also in clinical care paradigms and payer considerations. Continued collaborative efforts and future discussions are critical to advance the science and application of patient preference information. View the archived webcast here.
About the Author
Christiane Truelove is a healthcare journalist based in Bristol, PA.
