A Report of the ISPOR Performance Outcome Assessments Task Force
Lawrenceville, NJ, USA—July 11, 2023—Value in Health, the official journal of ISPOR—The Professional Society for Health Economics and Outcomes Research, announced today the publication of an ISPOR Good Practices Report providing good measurement practice recommendations specific to the selection and development of performance outcome (PerfO) assessments for use in capturing clinical benefit in treatment trials. These recommendations are essential reading for clinical trialists or anyone designing and conducting studies where PerfO assessment is being considered or has been identified as the appropriate clinical outcomes assessment type to evaluate physical, cognitive, and sensory function, and the impact of functional limitations in those areas on peoples’ daily lives. The report, “Recommendations on the Selection, Development, and Modification of Performance Outcome Assessments: A Good Practices Report of an ISPOR Task Force,” was published in the July 2023 issue of Value in Health.
“In evaluating clinical benefit of new therapeutic interventions, it is critical that the treatment outcomes assessed reflect aspects of health that are clinically important and meaningful to patients,” said author Chris J. Edgar, PhD, Chief Science Officer, Cogstate Ltd, London, United Kingdom. “The US Food and Drug Administration (FDA) has categorized clinical outcome assessments based on 4 distinct data sources: patient-reported outcome assessments, clinician-reported outcome assessments, observer-reported outcome assessments, and performance outcome assessments. Little has been published on evaluating content validity and other measurement properties of this last source—PerfO assessments—especially in the context of drug or device labeling. The focus of this first report of the task force is on PerfO assessments, which reflect physical (eg, mobility), cognitive (eg, working memory), sensory (eg, visual acuity), and other functional skills such as activities of daily living.”
A PerfO assessment is a type of clinical outcome assessment based on standardized tasks actively undertaken by a patient according to a set of instructions. Examples include gait speed (eg, timed 25-foot walk test) and measures of memory (eg, word recall test). A PerfO assessment should be used when the optimal means of measuring the clinical benefit of therapeutic interventions is through the completion of defined/standardized tasks that reflect or are the foundational building blocks of day-to-day activities that are important and meaningful in patients’ lives.
Highlights of the good practice recommendations:
- PerfO assessments are most useful when the concept of interest is best measured via ability to perform standardized tasks. For instance, PerfO assessments may be important when patients are limited in their capacity for self-report (eg, cognitively impaired or pediatric populations) and when heterogeneity could affect the utility of self-report of activities in day-to-day life. While there are PerfO assessment-specific considerations, most of the good practice recommendations established for other types of clinical outcome assessments remain directly applicable.
- In contrast to patient-reported outcome measures—which are direct assessments of the identified meaningful aspects of health—PerfO assessments incorporate the performance of representative standardized tasks aimed at indirectly assessing the meaningful aspect of health. Thus, evidence supporting the relationship between the tasks selected to assess the concept of interest and the meaningful aspect of health it is intended to represent/reflect must be generated or assembled.
- With PerfO assessments, a significant amount of attention must be placed on the feasibility of the selected task in the specific context of use (eg, target patient population) along with the physical environment in which it will be conducted. Due to the relatively greater complexity around standardization of instructions, materials, setting, patient safety, and administrator and/or patient training, the Good Practices Report provides recommendations regarding the critical role of a pilot evaluation.
“The assessment of task performance has a long history in clinical practice and clinical research for diagnosis, establishing natural history of disease, and treatment,” noted Edgar. “The goal of this report is to provide emerging good measurement practice recommendations specific to the selection and development of PerfO assessments for use in capturing clinical benefit in treatment trials. Establishing good practices in their selection or development, validation, and implementation, as well as how they reflect meaningful aspects of health (ie, how patients function in daily life) is critical to ensuring high standards and in furthering patient-focused drug development. This Good Practices Report is critical to ensuring the development of fit-for-purpose PerfO assessments and their appropriate selection and use for the evaluation of clinical benefit.”
The report was accompanied by an editorial from Kevin P. Weinfurt, PhD, Duke University School of Medicine, Durham, NC, USA. His editorial, “Developing, Selecting, and Modifying Performance Outcome Assessments,” also published in the July 2023 issue of Value in Health, provides important background and context that help guide researchers in the use of clinical outcome assessments. “Although there is much to recommend in this report, I believe its greatest contributions concern the relative directness or indirectness of assessments,” said Weinfurt. “The report makes 3 important points: (1) it underscores the utility of a clear conceptual framework; (2) it articulates the tradeoffs between relatively direct or indirect assessments; and (3) it makes the important point that patient input by concept elicitation interviews can provide evidence in support of the content of more direct measures but is less relevant for more indirect measures (for which expert input might be more appropriate to judge the suitability of the tasks). In sum, this new report serves as another example of how the field has benefited tremendously from the ISPOR task forces’ leadership in advancing patient-centric trials and evidence.”
Background on the Task Force
ISPOR has been a leader in developing good practice recommendations in response to the FDA’s 2009 landmark guidance for industry titled Patient-Reported Outcome Measures: Use in Medical Product Development to Support Labeling Claims and subsequent regulatory guidance. The PerfO Assessment Task Force was convened following the requirements outlined in Criteria and Process for Initiating and Developing an ISPOR Good Practices Task Force, which include 2 rounds of formal review and 2 presentations for consensus development. The Task Force’s goal is to improve the evaluation and documentation of content validity and other measurement properties including reliability, construct validity, and ability to detect change for PerfO assessments (report 1) and to provide detailed recommended good practices for clinical trial implementation, allowing for more precise and reliable measurement and interpretation of treatment benefit (report 2, in progress). These good practice recommendations will be critical to researchers who are (1) involved in the design and conduct of clinical trials where PerfO assessment may be the most appropriate clinical outcomes assessment type; (2) intending to evaluate physical, cognitive, sensory and/or daily life function; (3) working with cognitively impaired or pediatric populations; or (4) seeking to operationalize regulatory guidance specific to PerfO assessment. All of ISPOR’s Good Practice Reports, organized by research category, are available here.
###
ABOUT ISPOR
ISPOR—The Professional Society for Health Economics and Outcomes Research (HEOR), is an international, multistakeholder, nonprofit dedicated to advancing HEOR excellence to improve decision making for health globally. The Society is the leading source for scientific conferences, peer-reviewed and MEDLINE®-indexed publications, good practices guidance, education, collaboration, and tools/resources in the field.
Website | LinkedIn | Twitter (@ispororg) | YouTube | Facebook | Instagram
ABOUT VALUE IN HEALTH
Value in Health (ISSN 1098-3015) is an international, indexed journal that publishes original research and health policy articles that advance the field of health economics and outcomes research to help healthcare leaders make evidence-based decisions. The journal’s 2021 impact factor score is 5.156 and its 5-year impact factor score is 6.779. Value in Health is ranked 9th of 88 journals in health policy and services, 18th of 109 journals in healthcare sciences and services, and 50th of 381 journals in economics. Value in Health is a monthly publication that circulates to more than 10,000 readers around the world.
Website | Twitter (@isporjournals)