About Real-World Evidence

RWD are the data relating to areas such as patient health status and/or healthcare delivery not collected in conventional RCTs. Examples of RWD are electronic health records (EHRs); wearables; medical claims data; surveys; and product, patient, and disease registries.

RWE is the clinical evidence obtained from RWD with regard to the use, potential benefits, and potential risks associated with a medical product.

Both RWD and RWE can be used to supplement RCTs, which remain the gold-standard in assessing a treatment’s safety and efficacy.

RWD and RWE have several limitations, such as: 

• RWD/RWE analyses primarily evaluate association and not causation, therefore are typically answering different questions than RCTs.
• There have been concerns related to biases due to lack of randomization and issues of quality in data collection, which should be closely evaluated.
• The source and type of data used may limit the generalizability of the results and endpoints.
• There could be missing or incorrectly coded data from the source.

Real-World Evidence: Additional Tool to Support Clinical Decision Making

By A. Mark Fendrick, MD
Professor, Departments of Internal Medicine and Health Management & Policy; Director, Center for Value-Based Insurance Design; University of Michigan

The world has become increasingly more digital – a change impacting nearly every aspect of life. This rapid adoption has culminated in unprecedented amounts of health care data – generated from a variety of real-world sources, including electronic health records (EHRs), medical claims and billing data, disease and product registries, digital health apps, prescription data, observational studies, and surveys. Increasingly, information gleaned from peoples’ wrists via wearables or reported to us from homes using monitoring devices results in data. This information, sometimes referred to as “real-world data (RWD),” is defined as health-related information reported and collected in real-world medical settings, outside of traditional randomized controlled trials (RCTs). These and other patient-reported outcomes research data may be a useful supplement to the gold standard RCTs, helping clinicians and their patients, in conjunction with RCTs, practice more personalized patient care and potentially improve outcomes.

At an individual level, wearables and home monitoring systems track fitness, nutrition and other health measurements. Meanwhile, at a clinician level, EHRs have become the norm: in 2017 nearly 86% of U.S. office-based physicians use an EHR system,¹ compared to 42% in 2008.² Rapid digitization during the COVID-19 pandemic has also afforded providers and researchers new mechanisms for engaging with patients, from telehealth to clinical trials, that allow participation remotely using a combination of wearable devices, patient questionnaires, and connected sensors.³

Regulatory bodies, like the U.S. Food and Drug Administration (FDA), have long recognized the value of RWD. While RCTs are still the gold standard for reviewing regulatory drug/device applications, fit-for-purpose RWD are also being considered to potentially help inform certain submissions*.^4,5

There are benefits and challenges with RWD and the evidence generated from such data (known as “real-world evidence” or RWE). It can show how patients use available health care such as services, devices and medications, by obtaining information across diverse patient populations potentially not well captured in clinical trials. Thus, RWD/RWE analyses, which can only evaluate correlation and not causation, can be viewed as a supplement (but not a replacement) to the evidence we have available to us through RCTs. They supplement clinical trials by allowing for a wider variety of treatment scenarios and possible comparisons from the real-world setting than typically available in literature solely based on RCT data. Specifically, RWD can provide more information about the total patient experience or patient journey – particularly important as the health care sector works to address issues of health equity, including inequity in disease prevention, diagnosis, treatment, and care. A select list of challenges and limitations of RWD/RWE includes^4,6,7,8:

• RWD/RWE analyses can only evaluate association and not causation, therefore are answering different questions than RCTs.
• Concerns related to biases in data collection due to lack of randomization and issues of quality in data collection should be closely evaluated.
• The source and type of data used may limit the generalizability of the results and endpoints.
• Missing or incorrectly coded data from the source.

Although RWD often takes the form of aggregated data from clinical practice, individual real-world experience can contribute to RWD and potentially inform shared decision making. An example of this in action: when treating a patient of mine recently, the RCT evidence suggested that he would benefit from a specialty drug. However, the patient refused to be treated with the drug and I couldn’t figure out the reason for this decision. It was only after a second conversation that the patient told me he was homeless and did not have a refrigerator, and the drug required refrigeration. In other words, affordability or belief in the drug’s effectiveness wasn’t why this patient denied the drug – he just couldn’t refrigerate it properly and therefore denied it. Once I uncovered the barrier by collecting RWD from the patient’s experience, we found another treatment option that better fit his situation.

Effectively deploying the power of real-world evidence
All evidence should be evaluated with appropriate scrutiny. As the prevalence of patient health data sources continues to grow, it is essential for clinicians to be prepared to critically assess the quality of these data. There have been incidents of unorthodox research practices, such as data mining and cherry picking of data to show mainly favorable results, which has led some practitioners and others to feel hesitation about the quality and reliability of RWD/RWE analyses.

However, there are useful tools to assessing whether RWE can appropriately support decision making. When assessing RWE studies, clinicians should evaluate carefully important aspects of the RWE, such as^8,9:
• Data Quality: What is the quality of the data source used?
• Transparency of Analytical Research Approach: Was the research design and process transparently communicated?
• Statistical Methods: Were appropriate statistical methods applied?
• Reproducibility: Is the study replicable/reproducible?

All health care stakeholders have a role to play in strengthening our collective capacities to fully realize the capabilities and value of RWE. To ensure analyses are fit to support clinician decision making, study sponsors and researchers must practice transparency around data relevancy and quality characteristics.⁹ This, in turn, can help provide useful data that providers can consider incorporating into the evidence used to inform decision making in health care, whether for an individual patient treatment decision or a quality improvement initiative within a department or hospital.

The benefit of RWD is embedded in its name. The data come from the real-world, where diverse people live, work and play. While RCTs remain the gold standard for assessing safety and efficacy of a treatment, correctly and properly collected and analyzed RWD and RWE offer us an additional way to help inform treatment decisions for our patients.

See below for a graphical representation of how RWD/RWE can help support decision making in health care.

For more information on RWD/RWE, visit ISPOR’s, the Professional Society for Health Economics and Outcomes Research’s, webpage (with sponsorship from the BMS-Pfizer Alliance) at: https://www.ispor.org/strategic-initiatives/real-world-evidence/about-real-world-evidence

*Regarding drug submissions, the FDA has primarily used RWD in its evaluation of drug safety and only in limited circumstances to inform decisions regarding drug effectiveness. Per the 2018 FDA Framework for its Real-World Evidence Program, the program will “evaluate the potential use of RWE to support changes to labeling about drug product effectiveness, including adding or modifying an indication, such as a change in dose, dose regimen, or route of administration; adding a new population; or adding comparative effectiveness or safety information.”⁴

References:
1. U.S. Center for Disease Control and Prevention. Electronic Medical Records. National Center for Health Statistics. Page last updated: 3 March 2020.
2. HealthIT.gov. Office-based Physician Electronic Health Record Adoption. Official Website of The Office of the National Coordinator for Health Information Technology (ONC). Page last updated: 6 August 2021.
3. Izmailova, E.S., Ellis, R. and Benko, C. Remote Monitoring in Clinical Trials During the COVID‐19 Pandemic. Clinical and Translational Science. vol 13, issue 5 p. 838-841. https://doi.org/10.1111/cts.12834
4. U.S. Food & Drug Administration. Framework for FDA's Real-World Evidence Program. December 2018. https://www.fda.gov/media/120060/download
5. U.S. Food & Drug Administration. Examples of Real-World Evidence (RWE) Used in Medical Device Regulatory Decisions. March 2021. https://www.fda.gov/media/146258/download
6. ISPOR.org. Real-World Evidence. 2021. Accessed 6 October 2021. https://www.ispor.org/strategic-initiatives/real-world-evidence
7. Blonde, L, et al. Interpretation and Impact of Real-World Clinical Data for the Practicing Clinician. Advances in Therapy. Vol 35(11). 1763–1774. 2018. https://doi.org/10.1007/s12325-018-0805-y
8. Berger ML, Sox H, Willke RJ, et al. Good Practices for Real‐World Data Studies of Treatment and/or Comparative Effectiveness: Recommendations from the Joint ISPOR‐ISPE Special Task Force on Real‐World Evidence in Health Care Decision Making. Value in Health. 2017;20(8):1003-1008.
https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5639372/
9. Daniel, G et al. Characterizing RWD Quality and Relevancy for Regulatory Purposes. Duke. 1 October 2018.
© 2021 Bristol Myers Squibb Company and Pfizer Inc. All rights reserved
NO-US-2102880 10/21