Prerelease Sessions

Moderator: Lucinda Orsini, DPM, MPH, COMPASS Pathways, Skillman, NJ, USA

Panelists: Barbara Bierer, MD, Multiregional Clinical Trials Center, Brigham and Women's Hospital, Harvard University, Cambridge, MA, USA; David Thompson, PhD, Real World & Late Phase, Syneos Health, Boston, MA, USA; Jeroen Paul Jansen, PhD, PRECISIONheor, Oakland, CA, USA

ISSUE: There are many studies and reviews focused on the ability of real world evidence studies to replicate and reproduce randomized clinical trials. However, we often use RWE to answer questions that an RCT is not designed to answer. Can RWE replicate RCTs and if they do does that really give us comfort that RWE is credible and reliable?

OVERVIEW: RCT replications studies using secondary data sources are currently producing much awaited data to see how well RWE can produce ‘accurate’ results as defined by replicating RCT findings. DUPLICATE is sponsored by the FDA and intends to replicate 30 completed Phase III or IV trials and to predict the results of seven ongoing Phase IV trials using Medicare and commercial claims data. OPERAND conducted by Brown University and Harvard Pilgrim Health Care Institute (funded by unrestricted grants from a consortia of pharmaceutical companies) intends to replicate the ROCKET-AF trial for atrial fibrillation and the LEAD-2 trial for Type 2 diabetes control using claims from commercial and Medicare Advantage plans and electronic medical record data from OptumLabs Data Warehouse. This panel will explore viewpoints from both RWE experts involved in these duplication studies and as RWE experts, and from the experts who understand the strength of evidence that is needed in order to make causal inference or comparative effectiveness/efficacy claims. The moderator will open the session with a 5 minute introduction to the issue and each speaker will have 10 minutes to discuss their view point. There will be interactive use of polling through the zoom platform and at least 20 minutes for audience Q&A or for the panelists to query each other on their viewpoints if on-demand. This session will be of interest to HEOR professionals in life sciences industry, academia, regulatory or payer space who are interested in RWE.

Moderator: Dan Riskin, MD, MBA, Verantos, Menlo Park, CA, USA; 
Panelists: Tina Hernandez-Boussard, M.P.H, Ph.D., M.S., School of Medicine, Stanford University, Stanford, CA, USA; Nancy A Dreyer, MPH, PhD, Real-World Solutions, IQVIA, Cambridge, MA, USA; Keri L Monda, PhD, Center for Observational Research, Amgen Inc, Thousand Oaks, CA, USA

ISSUE: Increasingly, real-world evidence (RWE) is used to make clinical assertions and influence the standard of care through regulatory, reimbursement, and clinical decision-making. High validity RWE is increasingly desirable and required. But, the industry has not yet reached consensus on what it means to produce “research-grade” evidence using “fit-for-purpose” data. As physicians, insurance, and regulators increasingly scrutinize RWE for quality, it is now an industry imperative to understand issues of data quality and design a strategy to produce high-validity evidence.

OVERVIEW: This panel will seek to unpack this complex issue. Dan Riskin, the panel moderator, will set the stage for discussion by highlighting key questions, including: What are the differences between traditional and advanced RWE? What factors influence credibility of RWE? Where are there still gaps in data accuracy and population generalizability? Tina Boussard will describe scientific efforts to use advanced data sources and how this influences data quality. Keri Monda will address how the pharmaceutical industry is innovating in advanced RWE while maintaining and even growing efforts in traditional RWE. Nancy Dreyer will comment on advances in evidence validity and outline current thinking on research-grade RWE in the era of the 21st Century Cures Act. The moderator will ensure that at least 15 minutes of the session is devoted to soliciting audience interaction and feedback on the competing viewpoints.

Moderator: Alexandra Chambers, BSc, MA, Novartis, Dorval, ON, Canada

Panelists: Paul Wheatley-Price, BSc, MBChB, FRCP (UK), MD, Ottawa Hospital Cancer Centre, Ottawa, ON, Canada; Sarah Berglas, BSc, BA, CADTH, Ottawa, ON, Canada; Christina Sit, BSc, Lung Cancer Canada, Toronto, ON, Canada

ISSUE: Innovative therapies are continually being developed, and Health Technology Assessments (HTA) agencies are evolving to keep up with the pace of innovation. A key tenant of HTA is the engagement of clinicians and patients who provide context and experience with the disease and therapy under review. Are we optimizing the perspectives of these key stakeholders into the HTA processes as novel therapies continue to enter the health system?

OVERVIEW: HTAs are designed to inform decision making through a multi-stakeholder approach considering many perspectives. For many years, the health care system has committed to focusing on patient-centred care, and that priority should also apply to HTA. The Canadian Agency for Drugs and Technologies in Health (CADTH) has grown to become a leader in patient engagement in HTA, where now there is a call for patient and clinician input for every review. Yet today’s treatment paradigms have been transformed by innovation and novel clinical trial designs, and the pace of new drug discovery continues to increase, putting pressure on the HTA process to further evolve. Real-world insights and the treatment values that patients and clinicians share can be enhanced. For this session, a diverse group of panelists will examine the strengths and opportunities that exist to meaningfully incorporate the experiences that patients and clinicians into HTA.

Moderator: Arnaub Chatterjee, MHA, MPA, Acorn AI at Medidata, a Dassault Systèmes company, Boston, MA, USA

Panelists: John Concato, MD, MS, MPH, BEng, CDER, FDA, Silver Spring, MD, USA; Andrew Kress, BA, HealthVerity, Philadelphia, PA, USA; Najat Khan, PhD, Janssen R&D Strategy and Operations, Johnson & Johnson, Malvern, PA, USA

ISSUE: The efficacy of new drugs is usually established by clinical trials (CTs) that are conducted with a finite follow-up period at the end of the CT, patients are often no longer followed. Data tokenization allows for patients in a CT to be followed using RWD, which in turn will facilitate accurate understanding of post-clinical trial treatment patterns and healthcare resource utilization, as well as providing information on long-term effectiveness and safety. These benefits would be specifically of importance when: It may not be feasible or ethical to run large and lengthy CTs (as often occurs in oncology and rare disease) or a situation warrants urgency, such as during the COVID-19 pandemic. At the same time, merging data from various sources in a secure manner is challenging due to: Lack of interoperability among existing healthcare systems or concerns regarding privacy and potential vulnerability of patients’ personal and health-related information. An urgent need therefore exists to explore this topic from the perspectives of relevant stakeholders.

OVERVIEW: The panel will discuss the opportunities and challenges associated with combining the data of patients enrolled in the clinical trial with real-world data sources.

Discussion Leaders: David M Phillippo, PhD, Bristol Medical School, University of Bristol, Bristol, BST, UK; Jeroen P Jansen, PhD, School of Pharmacy, University of California San Francisco, San Francisco, CA, USA, and PrecisionHEOR, Oakland, CA, USA; Nicky J Welton, PhD, Bristol Medical School, University of Bristol, Bristol, UK 

PURPOSE: Indirect comparisons and network meta-analysis (NMA) estimate relative treatment effects between competing health technologies, however these methods may lack validity if there are differences in patient populations between included studies. Furthermore, in Health Technology Assessment (HTA) the estimates need to be relevant to the target population for the decision. This workshop will provide an overview of methods for indirect comparisons and NMA that aim to account for population differences. Advantages and disadvantages of the different methods and implications for future HTA submissions will be discussed.

DESCRIPTION: This workshop will begin with an overview of different approaches to population adjustment for indirect comparisons and NMA. If individual patient data (IPD) are available for all studies then IPD-level network meta-regression is the gold standard approach. However, in HTA IPD are typically only available for one or a subset of studies. Available methods in this case include matching-adjusted indirect comparison (MAIC), simulated treatment comparison (STC), network meta-regression, and the recently developed multilevel network meta-regression (ML-NMR). The methods will be compared as to their: validity; flexibility to synthesise networks of any size and availability of IPD; and ability to produce estimates in a given target population of interest. Their performance in simulation studies will be presented. Recommendations on when to use these methods for future HTA submissions will then be discussed based on hypothetical yet representative case-studies characterized by different evidence scenarios. Dr Jansen will introduce current methods for population adjustment in HTA. Dr Phillippo will then describe ML-NMR and compare the methods. Finally, Prof Welton will discuss recommendations for HTA. Presentations will last 40 minutes, followed by an interactive Q&A session, providing the audience an extended opportunity to discuss the issues raised with the panel. This workshop will benefit those involved in preparing or critiquing HTA submissions that utilise population adjustment methods.

Discussion Leaders: Laura Elisabeth Gressler, B.A., M.S., Pharmaceutical Health Services Research, University of Maryland, School of Pharmacy, Baltimore, MD, USA; Elise Berliner, PhD, Center for Outcomes and Evidence, Agency for Healthcare Research and Quality, Rockville, MD, USA 

PURPOSE: Funding and activity in health innovation may not be aligned with the areas of highest public health burden and costs to the system. This workshop will focus on an attempt to develop of a reliable and valid model that helps to forecast, identify, and prioritize critical health and innovation gaps.

DESCRIPTION: The Department of Health and Human Services (HHS) and FasterCures convened a group of health experts and data scientists from public and private sectors to develop an objective, data-based methodology to identify critical health innovation gaps in order to inform future programs and policies. The multidisciplinary group aimed to develop a reliable and valid model to help identify critical health innovation gaps as measured by three domains: public health impact, healthcare costs, and innovation activity. The workshop will outline the process for developing the metrics from data sources and methodology for an analytic model and data analyses. Results from the pilot study will also be presented. The first discussion leader will present in 15 minutes the overarching goal of the project and the methodologies employed by each of the subgroups. The second presenter will then present for 10 minutes the detailed methodology used within the public health impact subgroup to identify conditions with the highest public health impact for the pilot and calculate research disparities. Workshop participants will then contend with issues in choosing data sources and developing metrics to measure public health burden and contribute feedback on methods of the pilot. Finally, how the data from all three domains were brought together in a multicriteria decision modeling will be presented. The final model can be leveraged to identify the areas that have relatively low innovation activity compared to public health burden and costs and allow the HHS to consider prioritization of resources to these important areas.

Discussion Leaders: Margaret Rehayem, MA, National Alliance of Healthcare Purchaser Coalitions, Washington, DC, USA; Elisabeth Oehrlein, PhD, MS, Research and Programs, National Health Council, Washington, DC, USA; Kimberly Westrich, MA, National Pharmaceutical Council, Washington, DC, USA; Jennifer Bright, MPA, Innovation and Value Initiative, Alexandria, VA, USA 

PURPOSE: This workshop will provide lessons learned from three projects focused on improving employer evidence needs and engagement in value assessment (VA). These findings will inform how current methods can better account for health plan attributes that matter to employers. 

DESCRIPTION: Employers are key healthcare purchasers in the United States. They have a vested interest in ensuring employees have access to timely care that is most likely to meet individual employee and/or family needs. As healthcare costs rise, employers are doubling down on achieving improved value for their dollar. Health care value, from the employers’ perspective, is broader than that considered by many stakeholders and includes the relative impact on clinical outcomes, employee productivity, and organizational value proposition. However, employers have not traditionally been included in discussions about VA. As VA frameworks are increasingly used to inform health care decision-making, it is important that employers and VA bodies engage in meaningful discussions with one another on what is important to them, discuss alignment in perspectives and needs, and identify strategies to improve current approaches. This workshop is relevant for VA bodies, researchers, patients, students, healthcare purchasers, payers and policymakers.

Discussion Leaders: Elizabeth (Nicki) Bush, MHS, Global Patient Outcomes and Real World Evidence, Eli Lilly and Company, Indianapolis, IN, USA; Elizabeth Tschosik, PhD, Patient-Centered Outcomes Research, Genentech, South San Francisco, CA, USA; Bray Patrick-Lake, MFS, Evidation Health, San Mateo, CA, USA; Jennifer Goldsack, MS, MBA, DiMe, Boston, MA, USA 

PURPOSE: To support the audience in selecting, developing, and deploying fit-for-purpose, meaningful digital clinical measures across clinical research, patient care, and public health.

DESCRIPTION: The global COVID-19 pandemic has brought new urgency to the drive toward devising a common framework for the development and deployment of high quality, patient centric, digital clinical measures. Multiple digital technologies for symptom tracking and contact tracing are currently in use and/or development, and clinical researchers are now designing new decentralized clinical trial strategies using telemedicine and remote patient monitoring to minimize pandemic-related disruption of the clinical research enterprise. Adoption of a comprehensive "how-to" roadmap to synthesize best practices in the digital health field is essential for guiding these efforts and safeguarding public health.

Discussion Leaders: Mary Beth Ritchey, PhD, FISPE, Med Tech Epi, LLC, Philadelphia, PA, USA; Danica Marinac-Dabic, MD, PhD, MMSc, FISPE, CDHR/OCEA, U.S. Food and Drug Administration, Silver Spring, MD, USA; Christopher Tieche, PhD, Medtronic, Chicago, IL, USA; David Wamble, PhD, Bristol Myers Squibb, Lawrence Township, NJ, USA 

PURPOSE: In the world of medical device surveillance, the processes of signal management (detection, discernment and evaluation) are complex and driven by dynamic convergence of device-, provider-, setting- and patient- related characteristics. This workshop will focus on identification of actionable factors for building the (1) capacity of device ecosystem to ascertain and characterize the signal; (2) efforts to utilize novel methods including artificial intelligence/machine learning (AI/ML) and blockchain; (3) transparent and timely communication between stakeholders; and (4) value of patient-centric, interdisciplinary approaches that integrate the medical product ecosystem.

DESCRIPTION: Varying levels of unfamiliarity with real-world data (RWD), study designs, and analytic approaches appropriate for signal management has resulted different levels of trust across the medical device ecosystem, underscoring the need for education and training. This workshop will engage workshop participants in building an ecosystem-driven model for signal management, using examples from paclitaxel-coated devices for the treatment of peripheral arterial disease and diagnostics for identification of biomarkers for oncology treatments. Dr. Richey will discuss methodological capacity and identify needs, some early successes and introduce early- and long-term opportunities. Dr. Marinac-Dabic will present the integration of digital health solutions including blockchain and AI/ML algorithms to advance the value of RWD in active surveillance. Dr. Tieche will highlight the need for responsible transparency and communication across health care sectors and stakeholder perspectives. Dr. Wamble will detail the importance of characterizing new and ongoing risk-benefit balance from a market access perspective. The workshop participants will work interactively in four groups to identify tactical approaches in each of the four domains discussed (i.e., methodologic capacity, integration of digital solutions, communications, and interdisciplinary approach). Interested participants will then also be invited to join the post-workshop mirror stakeholder group to engage in further discussions on this topic.

Moderator: Homero Antonio Monsanto, PhD, Center for Observational and Real-World Evidence (CORE), Merck & Co., Inc., Cidra, PR, USA

Panelists: Claudia Brabata, MSc, Medtronic, USA, Miami, FL, USA; Diego F Guarin, MD, MPH, MA, Merck, Kenilworth, NJ, USA; Jose Y. Santiago, PhD, Abbott, USA, Des Plaines, IL, USA

ISSUE: The SARS-CoV-2 infection (COVID-19) has had profound health, social and economic effects across the world. By mid-December 2020, four countries in Latin America (Argentina, Brazil, Colombia and Mexico) had more than 1 million cases and were among the top 10 countries in the world in number of deaths due to COVID-19. Health innovations to tackle the pandemic, ranging from vaccines and antivirals to medical devices and diagnostics are advancing at incredible speed adding pressure to provide rapid and broad access to these technologies for those who need in order to manage the pandemic. Health Technology Agencies in Latin America follow explicit value frameworks to assess health technologies. These frameworks have a narrow definition of value, mainly focused in elements like health benefits (e.g. LYG, QALYs) and costs, disregarding other elements. COVID-19-related health technologies illustrate challenges and opportunities to advance the definition of what is value and which other elements should be considered based on international recommendations.

OVERVIEW: This panel will address challenges faced by decision makers to assess the value of COVID-19-related health innovations under the established value frameworks in three countries with established HTA agencies (Brazil, Colombia and Mexico). Experts from health technology producers including pharmaceuticals, medical devices, and diagnostics, will discuss the challenges and opportunities faced in the region highlighting those value elements that are not being considered yet.

Moderator: Michelle Tarver, MD, PhD, Center for Devices and Radiological Health, U.S. Food and Drug Administration, Silver Spring, MD, USA

Panelists: Shelby Reed, PhD, RPh, Preference Evaluation Research Group, Duke Clinical Research Institute, Durham, NC, USA; Olufemi Babalola, MHS, MSc, Center for Devices and Radiological Health, Food and Drug Administration, Silver Spring, MD, USA; Lisa Kaiser, BS, Dynata Healthcare Solutions, Warrington, PA, USA

ISSUE: Patients with a physician-confirmed diagnosis are the ideal respondents for stated-preference research in healthcare/regulatory evaluation. However, there are logistical challenges associated with recruiting patients from clinical sites. Recruitment of patients with a physician-confirmed diagnosis may be associated with longer study duration and greater cost. It has been suggested that national panels of patients provide direct access to willing patient participants which can translate to lower expenses and shorter study timeframes. It is not clear if preference results elicited from a self-reporting cohort are similar to preferences from patients recruited directly from the clinic setting. Are panels a suitable substitute for clinical enrollment to elicit preferences? The answer to this methodological question would potentially lead to greater efficiencies and may encourage the conduct of health preference studies to inform regulatory decisions.

OVERVIEW: This panel will debate the pros and cons of a utilizing a self - reporting cohort to elicit patient preferences. Dr. Tarver will moderate the panel. She will pose questions for the panelists to discuss. Some questions are: Do preference surveys administered to respondents via panels and clinically diagnosed patients deliver similar results? Are samples similar in terms of demographics, disease severity, health literacy, numeracy, and other factors? Does this influence the results? How do vendors who manage panels request information on members’ medical conditions? What are recruitment rates for panel-derived vs. physician-recruited samples? What’s the recruitment duration difference (panel vs. physician diagnosed)? What measures can be taken to optimize the reliability of a self-reported diagnosis? Dr. Reed will discuss methodological considerations in comparing preferences between samples. Olufemi Babalola will present an example from the CDRH that compares preference results from a panel (self-reported) to a clinically diagnosed cohort in the case of fibroids. Lisa Kaiser will highlight best practices around the recruitment into and management of their panel.

Moderator: Jacki Chou, MPP, MPL, PRECISIONheor, Los Angeles, CA, USA

Panelists: Pinar Karaca Mandic, PhD, Medical Industry Leadership Institute, University of Minnesota, Carlson School of Management, Minneapolis, MN, USA; Romilla Batra, MD, MBA, SCAN Health Plan, Long Beach, CA; Lance Waller, PhD, Emory University, Atlanta, GA, USA

ISSUE: The COVID-19 global pandemic is an unprecedented public health challenge requiring significant resources to evaluate the problem and determine the best solutions from a health and economic perspective. Much of the decision making related to COVID-19 have been centered around prevention, which requires forecasting. This issue panel will discuss the data and modeling needs for policy makers, health care decision makers, and the average US resident for the COVID-19 pandemic, specifically, with blue sky considerations for a non-specific future pandemic.

OVERVIEW: This panel will discuss the modeling and real-time data needs to support good decision making during the COVID-19 global pandemic. The panel will consider past challenges in data collection and forecasting, and the solutions that were used, as well as remaining challenges. The panel will also debate the role of the scientist in communicating critical data to decision makers at all levels of government and health system. Ms. Chou will moderate and provide an overview of the pandemic from a US perspective as well as a framework for thinking through the data and modeling challenges for COVID-19. Dr. Karaca Mandic will describe the challenges with real-time data collection and interpretation with the University of Minnesota COVID-19 Hospitalization Tracking Project. Dr. Singh will review advanced modeling techniques that have been or could be implemented in supporting COVID-19 decision making. Finally, Dr. Waller will describe his efforts relating to assessing inequity in spatial access to testing sites between racial and ethnic groups in Atlanta, Georgia.

Moderator: Donna A Messner, PhD, Center for Medical Technology Policy, Baltimore, MD, USA

Panelists: Foluso O Agboola, MBBS, MPH, Institute for Clinical and Economic Review, Boston, MA, USA; Emily Tsaio, PharmD, Premera Blue Cross, Mountlake Terrace, WA, USA; Jill Morrow-Gorton, MD, MBA, University of Pittsburgh (UPMC) Health Plan, Pittsburgh, PA, USA

ISSUE: Panelists will explore the potential value of payer and health technology assessor (HTA) participation in consensus-based core outcome set (COS) development for drug clinical trials, discuss hurdles to their participation and debate the potential benefits (or drawbacks) of COS to coverage and payment decision-making.

OVERVIEW: Gene therapies and biologics promise paradigm shifts for many diseases, though at considerably higher prices. Decision makers assessing the added value of these agents for patients, in relation to their high costs, encounter major challenges: a) heterogenous outcomes hinder comparisons with traditional therapies, or b) effective new therapies impact standards of care such that traditional outcomes lose relevance. COS development may address these and other challenges by bringing together multiple stakeholder perspectives to deliberate and reach consensus on critical outcomes for decision-making. Through this lens, payers and HTA are key stakeholders in these initiatives.

PURPOSE: This workshop intends to share with the audience how by applying the value-based health care (VBHC) framework, the Brazilian Institute for health technology assessment (IATS) is producing contributions for the scientific community and the brazilian stroke care pathway. The Time-driven Activity-based Costing methodology is one of the gold-standards recommendations of the value-agenda. However, its application in a real-world setting is still emergent and frequently registers methodological heterogeneity. Motivated by that, the TDABC in healthcare Consortium (www.tdabcconsortium.com) was created to improve the quality of projects that apply TDABC. The community already involves members from all continents, and in Brazil is working with the national stroke network in a research to evaluate value for the stroke-care pathway. By attending this workshop, the audience will learn why the TDABC is the gold-standard recommendation in the context of VBHC, how it is possible to explore the method to identify and measure cost-saving opportunities and consequently increase value, and how TDABC in addition to the stroke ICHOM standard set is being used to evaluate and increase value for the stroke care pathway. 

DESCRIPTION: The speakers will introduce the TDABC in the context of VBHC (20 min), explain the sequence of eight steps to apply it using real examples (20 min), and present the value analysis for the Brazilian stroke care pathway. The stroke case discussion will explore how the value-analysis performed can guide waste reduction for the stroke care pathway and identify opportunities to increase the quality of care (20 min).

Discussion Leaders: Dalia Dawoud, PhD, Science, Policy and Research Programme, National Institute for Health and Care Excellence, London, UK; Johan Pontén, MA, Dental and Pharmaceutical benefits Agency, TLV, Stockholm, Sweden; Milou Amber Hogervorst, PhD, Division of Pharmacoepidemiology and Clinical Pharmacology, Utrecht University, Utrecht, Netherlands; Ayla Lokhorst, MA, Development, Science and International Affairs, Zorginstituut Nederland (ZIN), Diemen, Netherlands 

PURPOSE: This workshop introduces the sandbox approach and explains its possible applications in health technology assessment (HTA), using methodological and policy challenges around real world data (RWD) for HTA-decisions. Attendees should walk away with insight in why it would be beneficial and how it might work to engage all stakeholders in such a complex environment to test emerging methods, policies and processes.

DESCRIPTION: In 2022, a policy sandbox will be organised in the Horizon2020 funded project HTx (Next Generation Health Technology Assessment) to test new methods to make better use of RWD. There is a growing recognition of the value of RWD in addition to efficacy data from RCTs. However, some hurdles remain in benefiting from the use of RWD. A policy sandbox is a safe environment where the outcomes of the project can be translated into HTA practice while engaging stakeholders. We have seen that it is crucial for the acceptance and uptake of any new method that stakeholders are engaged during development. Despite its potential, this is the first time that a sandbox will be applied in HTA-setting.

Milou Hogervorst will give a background on the willingness to use RWD for HTA decisions in challenging circumstances (8 min.). Secondly, Johan Pontén will elaborate on the identified challenges of using RWD in HTA (10 min.). Thirdly, Ayla Lokhorst will outline useful change models for engaging stakeholders (10 min.). Finally, Dalia Dawoud will present the concept of policy sandboxes and its use as a controlled environment to test new HTA methods (13 min.). Q&A time will be given at the end and questions will be asked on other examples/situations where it would be useful to apply the policy sandbox format (19 min.). This workshop will be very relevant for attendees from various backgrounds including academics, HTA agencies and pharmaceutical companies.

Discussion Leaders: Miriam Kimel, PhD, Patient-Centered Research, Evidera, Bethesda, MD, USA; Naomi Knoble, PhD, Food and Drug Administration, Silver Spring, MD, USA; Carla Dias Barbosa, MSc, Patient-Centered Research, Evidera, London , LON, UK; Laurie Eliason, MPH, Value Evidence and Outcomes, Patient Centered Outomes in Oncology, GlaxoSmithKline, Upper Providence, PA, USA 

PURPOSE: Conducting embedded interviews within clinical trials is a new research paradigm that can provide a variety of benefits, including understanding treatment benefit and safety/tolerability from the patient perspective, and estimating thresholds for meaningful change. Using a mixed method approach, embedded interview data can enrich our understanding of the patients’ experience to illuminate and complement the PRO and biomedical outcomes collected in the trial. This workshop will illustrate how embedded interviews can be utilized to generate in-depth and meaningful patient input through the drug development process.

DESCRIPTION: After a brief introduction of the purpose of this type of research and value for different stakeholders, the speakers will present examples of embedded interview use within clinical trials and address resulting contributions to drug development research in each case study. 
• (10 minutes) Introduction to description, uses and value of mixed methods embedded interview research to stakeholders. (Miriam Kimel)
• (10 minutes) Overview of value-added contributions of mixed method embedded interviews to support regulatory inquiries. (Naomi Knoble)
• (10 minutes) Case study on use of mixed methods assessment of patient treatment experience, including disease changes and tolerability in oncology. (Laurie Eliason)
• (10 minutes) Case study on use of longitudinal mixed methods approach to assess treatment benefit-risk assessment in a rare disease. (Carla Dias Barbosa)

Discussion Leaders: Steven Simoens, MSc, PhD, Department of Pharmaceutical and Pharmacological Sciences, KU Leuven, Leuven, VBR, Belgium; Evelien Moorkens, MSc PhD, Department of Pharmaceutical and Pharmacological Sciences, KU Leuven, Leuven, Belgium; Dalia Dawoud, PhD, Science, Policy and Research Programme, National Institute for Health and Care Excellence, London, UK; Delphine Courmier, PhD, MBA, Global Health Economics, AMGEN, Thousand Oaks, CA, USA 

PURPOSE: To provide guidance to and raise awareness of workshop participants regarding methodological issues surrounding the value assessment of biosimilars

DESCRIPTION: The use of biosimilars faces significant gaps and challenges in terms of their health technology assessment, which have not yet received sufficient attention in the health economic literature and in guidelines issued by many health technology assessment agencies.
This workshop will explain the following methodological issues in the value assessment of biosimilars:

1. The choice of the appropriate technique of economic evaluation (cost-minimization or other) and of the appropriate comparator.
2. The approach to filling the clinical evidence gap when the marketing authorization of the biosimilar has been granted on the basis of extrapolation (and no phase 3 clinical trial data are available).
3. The approach to value assessment of biosimilars in treatment naïve patients and in patients previously treated with biologics/biosimilars.
4. Whether value assessment of biosimilars should account for the potential ‘nocebo’ effect and how.
5. The management of uncertainty and barriers to uptake related to biosimilars and the role of managed entry agreements.
6. The valuation of value-added services. 
7. The societal valuation of expanding access to treatment in terms of capturing QALY gains at population level.

Discussion Leaders: Alan Brnabic, MS, Eli Lilly and Company, Indianapolis, IN, USA; David Madigan, PhD, Columbia University, New York, NY, USA; Anthony Zagar, PhD, Eli Lilly and Company, Indianapolis, IN, USA; Xiang Zhang, PhD, CSL Behring, Philadelphia, PA, USA 

PURPOSE: To present recently proposed methods for comparative effectiveness including the application of machine learning through frequentist model averaging (fMA) and doubly robust methods. Along with novel methods for estimation, we will discuss advances in evaluating the robustness of analyses to the potential for unmeasured confounding.

DESCRIPTION: Evidence from real world data (RWE) is increasingly becoming a vital component of decision health care decision making. However, limitations inherent with causal inference are challenging. One challenge is the need for selecting the ‘right’ statistical models both for treatment selection and outcome analysis. For treatment selection models, propensity scores estimated from logistic regression are the gold standard. However, new approaches using tree-based methods, balancing algorithms, and double scores have been proposed. Penalized regression has become popular for building outcome models and doubly robust methods bring the promise of more robust analyses. There is no consensus on the best approach and simulations suggest there is no single best method for all scenarios. Speakers will cover the above issues as well as future directions of including machine learning for causal inference using targeted maximum likelihood methods (TLME) and fMA. With fMA researchers specify many potential methods and models into the analysis and cross validation determines the better approaches in a data driven fashion. 
Another challenge in comparative effectiveness is addressing bias due to unmeasured confounding. In recent years a suite of methodologies has emerged ranging from methods quantifying the robustness to confounding to methods using external data to adjust the estimated causal effect. The speakers will facilitate an understanding of unmeasured confounding options by presenting different statistical procedures within a unified framework. Tools for selecting and implementing, along with simulations and real data examples will be discussed.

Moderator: Samuel Nussbaum, MD, Innovation and Value Initiative, Alexandria, VA, USA

Panelists: Steven Pearson, MD, MSc, Institute for Clinical and Economic Review, Boston, MA, USA; Annie Kennedy, BS, EveryLife Foundation for Rare Diseases, Washington, DC, USA; Bruce Sherman, MD, FCCP, FACOEM, North Carolina Business Group on Health, Greensboro, NC, USA

ISSUE: In the US, value assessment is rapidly becoming a cornerstone of decision making within contracting and payment negotiation, even as debate continues about whether we have the right tools and methods to support the increasing demand for quantifiable estimates of value. These value assessments give us numbers – but are they answering all of the right questions? And if not, what additional elements and frameworks are needed?

OVERVIEW: This panel will explore the current capabilities and limitations of value assessment methods, consider the most important unmet needs in discerning high and low-value care, and debate the best way forward to answering questions that matter to all stakeholders. Samuel Nussbaum will moderate and provide insight on payers’ needs when making value-driven decisions. Steven Pearson will provide the value assessor’s perspective and describe current real-world use. Annie Kennedy will provide the patient community perspective on the alignment between value assessments and value delivered to patients. Bruce Sherman will provide insights from the perspective of the employer community. Panelists will provide brief remarks, reserving the majority of the session for panel discussion and audience questions.

Moderator: Moderator: Nirosha Lederer, PhD, Aetion Inc, Boston, MA, USA

Panelists: David Thompson, PhD, Real World & Late Phase, Syneos Health, Boston, MA, USA; Lindsay Kehoe, MS, CGC, Clinical Trials Transformation Initiative, Durham, NC, USA; Lou Fiore, MD, VA System, Boston, MA, USA

ISSUE: Since first being described more than 50 years ago, the pragmatic clinical trial (PCT) has been considered the most scientifically rigorous of all real-world research designs. Yet, despite its promise, the PCT design has been relatively underutilized in comparison to other approaches for real-world evidence (RWE) generation, such as prospective observational research and retrospective studies of electronic health records (EHRs) and administrative claims databases. More recently, a variety of organizations have focused attention on the potential to embed clinical trials in real-world practice settings as a means of involving practicing physicians, recruiting real-world patients, and leveraging existing EHR systems for data capture. Is this approach better suited to RWE generation? Should we be talking “embedded” rather than “pragmatic” for real-world clinical trials?

OVERVIEW: In its RWE Program Framework, the US Food & Drug Administration highlighted the benefits of integrating clinical trials into real-world practice settings, indicating that real-world data systems can be leveraged in this fashion to generate regulatory-grade evidence. Other organizations, including the NIH Collaboratory, the Clinical Trials Transformation Initiative (CTTI) and the Australian Clinical Trials Alliance, have workstreams devoted to advancing these objectives. Nirosha Lederer, the panel moderator, will set the stage for the discussion by outlining the key questions, including: What are the defining characteristics of embedded clinical trials? In what ways are embedded trials similar or different from pragmatic trials? Is embedding clinical trials in real-world practice a more practical way of improving their generalizability? David Thompson will highlight similarities and differences between embedded and pragmatic trials. Lindsay Kehoe will describe CTTI’s progress to date to identify practical issues associated with embedded clinical trials. Lou Fiori will provide an overview of case studies of embedded clinical trials undertaken in the US Veterans Administration system. The moderator will solicit audience interaction and feedback on the alternative viewpoints.

PURPOSE: This panel seeks to discuss performance-based risk-sharing agreements (PBRSA) for medical technologies (MedTech); to identify whether taxonomies of PBRSAs for pharmaceuticals are applicable for MedTech; and, through discussion, develop a nascent set of best practices for device-specific PBRSA. 
Mark Sculpher will present a taxonomy based on the the nature of the decisions taken by health systems and the type of evidence needed to support these. Susan Garfield, a consultant active in the area of PBRSA, will bring the industry perspective and provide insights the challenges of planning, execution, and evolution of these plans, and Payam Abrashimi will serve as a representative of a payer perspective into how PBRSA can address the needs of payers and procurement bodies; Richard Charter will moderate and provide an overview of PBRSA.

DESCRIPTION: Procurement bodies have increasingly sought rigorous evidence before accepting premium price products. Recent pandemic challenges have accelerated these trends of more cautious evaluation of MedTech prior to procurement. Yet, regulatory agencies in many instances allow for the commercialization of devices without rigorous trials providing the evidence sought by procurement. In the absence of such data, evidence development through performance-based risk-sharing agreements (PBRSA) have recently generated increased interest, development, and emergent use. 

This panel seeks to discuss recent years’ growing interest in PBRSA for MedTech and the roles, opportunities, and challenges within this landscape of procurement bodies, governments, health technology assessment (HTA) entities, and payers. Topics for discussion include how to deliver PBRSA, what is the role of health economics and outcomes research in reducing the clinical and economic uncertainty, and the role of real-world evidence in risk-share agreements.
Each speaker will speak for approximately 15 minutes. The audience is encouraged to bring their experience to the discussion; open questions highlighted during the panel will be tested with audience polling.

Discussion Leaders: Susan dosReis, PhD, Center for Patient-Driven Values in Healthcare Evaluation, University of Maryland School of Pharmacy, Baltimore, MD, USA; Louis P. Garrison, PhD, The Comparative Health Outcomes, Policy, and Economics (CHOICE) Institute, University of Washington, Seattle, WA, USA; Stacey Kowal, MSc, Health Policy and Systems Research, Genentech, Alameda, CA, USA; Richard Xie, PhD, The Innovation and Value Initiative, Newton, MA, USA 

PURPOSE: The purpose of this workshop is to explore challenges in addressing health inequity as an objective in value assessment (VA), and to demonstrate and discuss the applications of potential methods in the US context using real-world model examples. 

DESCRIPTION: There is system-wide consensus and emphasis on the importance of tackling the widening health disparities in our society. Various promising modeling approaches, most notably the distributional cost-effectiveness analysis (DCEA), have been proposed to support more equitable resource allocation in health care. However, the extent to which these methods can be applied in the pluralistic, fragmented US healthcare system is unclear. 

In this workshop, the speakers will first discuss concepts and measures of health equity, and key challenges in addressing equitable resource allocation using existing methods (~10 minutes). To tackle these challenges, ensuring data representations from the underserved subpopulations is the necessary first step. Using the IVI’s model focusing addressing major depressive disorder (IVI-MDD) as an example, researchers will share strategies on how to recruit individuals of color and lower socioeconomic status in a component patient preference survey (~10 minutes). Discussions on available modeling will include: (1) how to build a Medicaid perspective in a VA model (~10 minutes), and (2) how to leverage US data to conduct DCEA through a case study on the impact of funding inpatient COVID-19 treatments on underlying health equality. (~10 minutes).
Insights from the workshop are applicable across all stakeholders: researchers will gain insight into available sources and methods for US equity assessment; innovators and patient groups can learn about how to encourage the traditionally underrepresented communities to participate in the data generation and collection process for health data; and payers and US HTA bodies can learn about data availability and suggested next steps for integration of equity assessment into decision-making.

Discussion Leaders: Mark Trusheim, MS, BS, MIT Center for Biomedical Innovation, Massachusetts Institute of Technology, Cambridge, MA, USA; Angela Banks, MBA, MA, UnitedHealth Group, Washington, DC, USA; Ron Potts, MD, Kaiser Permanente, Portland, OR, USA; David Strutton, PhD, MPH, Center for Observational and Real-World Evidence, Merck, Philadelphia, PA, USA 

PURPOSE: Based on MIT NEWDIGS LEAPS Project experience, inspire participants to explore Adaptive Reimbursement opportunities to accelerate adoption of RWE-based patient regimen optimization to simultaneously improve quality-of-life and economic outcomes.

DESCRIPTION: Scientific, clinical development and regulatory advancements are fueling unprecedented biopharmaceutical innovation from gene therapies to immunotherapies. These upstream innovations have not been matched by patient-centered downstream innovations that employ payment and other incentive systems to implement patient specific regimen improvements informed by leveraging real-world evidence.

Adaptive reimbursement combines downstream innovations that:
(1) Creatively cover and finance therapies to ensure timely, appropriate access, enhance healthcare system sustainability and reward upstream innovation; and
(2) Reward optimized treatment regimens for each patient through the generation of real-world evidence that reduces uncertainties regarding biopharmaceutical product performance.
This Panel will explore current Adaptive Reimbursement examples from the MIT NEWDIGS LEAPS project, discuss developing future innovations, and describe approaches to overcome the significant implementation challenges. Hear how:
• Durable cell and gene therapy precision financing models could apply to other therapeutic areas
• Adaptive reimbursement mechanisms might fund RWE production platforms (indirectly or directly) by creating financial demand for their results
• Data needed to administer adaptive reimbursement contracts can also be used to de-risk market access, identify unmet medical needs, and monitor patient benefits from therapies
• RWE platforms could fuel the design and implementation of innovative adaptive reimbursement models.

Discussion Leaders: Jacki Chou, MPP, MPL, PRECISIONheor, Los Angeles, CA, USA; Bhramar Mukherjee, PhD, University of Michigan, Ann Arbor, Ann Arbor, MI, USA; Zhilan Feng, PhD, Purdue University, West Lafayette, IN, USA; Anuj Mubayi, PhD, PRECISIONheor, Los Angeles, CA, USA 

PURPOSE: This workshop will focus on advanced modeling tools and how they might be used to inform healthcare decision making, including COVID-19 intervention polices as a recent example. Workshop participants will understand the foundational constructs of simulation and modeling techniques and the reliance of these models on the quality of the data used to inform inputs and assumptions. Participants of this workshop will be able to understand the basics for key types of selected statistical and mathematical models as well as what types of questions these models are suited to answer in health evaluations.

DESCRIPTION: Health economic evaluation is used to support decision makers in areas including public health, epidemiology, infectious diseases and pharmacology. This workshop aims to provide some background and overview on cutting edge modeling and simulation methods to address continuously changing healthcare problems and data. Ms. Jacquelyn Chou will provide an introduction and overview for the workshop, as well as basic principles and objectives for simulations and models (5 minutes). Dr. Bhramar Mukherjee will provide an overview of the basics for statistical models alongside key considerations and limitations when there is delay or underreporting, as in the case of COVID-19. (15 minutes). Dr. Zhilan Feng will walk-through the fundamentals and considerations for key types of mathematical dynamic models including complex agent based models and infectious disease transmission models (15 minutes). Finally, Dr. Mubayi will review the applications of these models for health economics via methods such as decision trees and microsimulation models. He and Ms. Chou will also facilitate an interactive session where participants can help sort research questions into the type of model and data that is best suited to answer that question via polling, as well as submit their own research questions to be sorted into model types by fellow participants via polling (30 minutes).

Moderator: Bill Marder, PhD, IBM Watson Health, Winchester, MA, USA

Panelists: Ami R Buikema, MPH, Health Economics and Outcomes Research, Optum, Eden Prairie, MN, USA; David L. Van Brunt, PhD, MS, BA, Evidence and Analytics, Health Economics and Outcomes Research, AbbVie, Chicago, IL, USA; Khalid Kamal Kamal, M.Pharm., Ph.D., School of Pharmacy, West Virginia University, Morgantown, WV, USA

ISSUE:The pharmaceutical industry funds thousands of research projects each year. Most projects run smoothly but problems do arise. This panel will identify a set of common issues and recommend some best practices for successful engagements.

OVERVIEW: In 2020, the ISPOR Institutional Council (IC) undertook a study to better understand sponsor-vendor interactions. A qualitative survey of IC members was conducted, results of which were utilized to develop a quantitative survey questionnaire. The questionnaire was completed by 158 ISPOR members who were either sponsors or vendors and involved in contract research. Four major issues were identified: 1) managing changes to scope and time, 2) achieving the quality expected for the project, 3) prompt/ timely reply as issues arise, and 4) stability of staffing/leadership (and managing turnover).