Published Apr 2009
Citation
Coons SJ, Gwaltney CJ, Hays RD, et al. Recommendations on evidence needed to support measurement equivalence between electronic and paper-based patient-reported outcome (PRO) measures: ISPOR ePRO Good Research Practices Task Force Report. Value Health.2009;12(4):419-429.
Abstract
Background: Patient-reported outcomes (PROs) are the consequences of
disease and/or its treatment as reported by the patient. The importance of
PRO measures in clinical trials for new drugs, biological agents, and
devices was underscored by the release of the US Food and Drug Administration’s
draft guidance for industry titled “Patient-Reported Outcome
Measures: Use in Medical Product Development to Support Labeling
Claims.” The intent of the guidance was to describe how the FDA will
evaluate the appropriateness and adequacy of PRO measures used as
effectiveness end points in clinical trials. In response to the expressed need
of ISPOR members for further clarification of several aspects of the draft
guidance, ISPOR’s Health Science Policy Council created three task forces,
one of which was charged with addressing the implications of the draft
guidance for the collection of PRO data using electronic data capture
modes of administration (ePRO). The objective of this report is to present
recommendations from ISPOR’s ePRO Good Research Practices Task
Force regarding the evidence necessary to support the comparability, or
measurement equivalence, of ePROs to the paper-based PRO measures
from which they were adapted.
Methods: The task force was composed of the leadership team of ISPOR’s
ePRO Working Group and members of another group (i.e., ePRO Consensus
Development Working Group) that had already begun to develop
recommendations regarding ePRO good research practices. The resulting
task force membership reflected a broad array of backgrounds, perspectives,
and expertise that enriched the development of this report. The prior
work became the starting point for the Task Force report. A subset of the
task force members became the writing team that prepared subsequent
iterations of the report that were distributed to the full task force for
review and feedback. In addition, review beyond the task force was sought
and obtained. Along with a presentation and discussion period at an
ISPOR meeting, a draft version of the full report was distributed to
roughly 220 members of a reviewer group. The reviewer group comprised
individuals who had responded to an emailed invitation to the full membership
of ISPOR. This Task Force report reflects the extensive internal
and external input received during the 16-month good research practices
development process.
Results/Recommendations: An ePRO questionnaire that has been
adapted from a paper-based questionnaire ought to produce data that are
equivalent or superior (e.g., higher reliability) to the data produced from
the original paper version. Measurement equivalence is a function of the
comparability of the psychometric properties of the data obtained via the
original and adapted administration mode. This comparability is driven by
the amount of modification to the content and format of the original paper
PRO questionnaire required during the migration process. The magnitude
of a particular modification is defined with reference to its potential effect
on the content, meaning, or interpretation of the measure’s items and/or
scales. Based on the magnitude of the modification, evidence for measurement
equivalence can be generated through combinations of the following:
cognitive debriefing/testing, usability testing, equivalence testing, or, if
substantial modifications have been made, full psychometric testing. As
long as only minor modifications were made to the measure during the
migration process, a substantial body of existing evidence suggests that
the psychometric properties of the original measure will still hold for the
ePRO version. Hence, an evaluation limited to cognitive debriefing and
usability testing only may be sufficient. However, where more substantive
changes in the migration process has occurred, confirming that the adaptation
to the ePRO format did not introduce significant response bias and
that the two modes of administration produce essentially equivalent
results is necessary. Recommendations regarding the study designs and
statistical approaches for assessing measurement equivalence are provided.
Conclusions: The electronic administration of PRO measures offers many
advantages over paper administration. We provide a general framework
for decisions regarding the level of evidence needed to support modifications
that are made to PRO measures when they are migrated from paper
to ePRO devices. The key issues include: 1) the determination of the extent
of modification required to administer the PRO on the ePRO device and 2)
the selection and implementation of an effective strategy for testing
the measurement equivalence of the two modes of administration. We
hope that these good research practice recommendations provide a path
forward for researchers interested in migrating PRO measures to electronic
data collection platforms.
Keywords: effectiveness, evaluation studies, health-related quality of life,
patient-reported outcomes.
Full Content
Log In to View ReportRelated Content
Questions?
For any questions about this report please contact us.