OBJECTIVES: NICE’s health technology evaluations manual states that clinical and cost-effectiveness evidence must be relevant to patient groups, comparators, perspective, outcomes and resource use in the UK setting. ‘Generalizability’ refers to the applicability of evidence to routine clinical practice, but not to a specific framework for such assessment. We examined how generalizability is considered and addressed in NICE TAs.

METHODS: TAs published between January 2019–May 2024 in ovarian, cervical, endometrial, vaginal and vulval cancers were identified on the NICE website. Data and relevant information were systematically extracted, based on pre-specified search terms and categories related to clinical trial or population generalizability: general population demographics; clinical baseline characteristics of patients; interventions and comparators, including subsequent therapies; and explicit comparability of trial and real-world outcomes. Company submissions (CS), external assessment group (EAG) reviews, public committee slides (PS), and final appraisal documents (FAD) were included.

RESULTS: Seventeen TAs were identified: terminated, incomplete or superseded (n=7); reached final publication/included for review (n=10). Common CS considerations included differences in population demographics and baseline characteristics (e.g. age, fitness, disease characteristics). Prior treatment and subsequent treatment compared with UK clinical practice was commonly discussed. EAGs commented on more categories (60%), fewer categories (30%), or the same number of categories (10%) than CS. Agreement was approximately 45% for issues explored by both CS and EAG. Fewer issues were presented in PS, indicating some resolution of issues during the process. This involved more clinical or real-world data, additional clinical validation, or scenario analyses. 47% of the initial concerns raised by the EAG were included in the FAD.

CONCLUSIONS: Ensuring comparability between trial data and UK clinical practice is a key concern in NICE TAs, which can sometimes be resolved with additional data, validation or uncertainty exploration. Reference to a structured framework to assess data generalizability would support consistency in the reviews.