OBJECTIVES: The new Regulation on Health Technology Assessment (HTAR) will apply from January 2025. For Joint Clinical Assessment (JCA), a large number of Population, Intervention, Comparator, and Outcome (PICOs) may be expected. This study aims to conduct an efficient and comprehensive PICO prediction exercise for an approved oncology Product X by manual PICO scoping and by testing the usability of publicly available generative Artificial Intelligence (AI) tools A and B.

METHODS: AI Tools: Training prompts about PICO framework and PICO prediction were developed. Pre-defined questions were asked to check if one of the pre-trained AI tools can be used for PICO prediction. Manual PICO Scoping: Relevant European and local guidelines were reviewed. Health technology assessment (HTA) reports for Product X were also reviewed to confirm PICOs. Analysis was done for Europe, focusing on France, Germany and Italy.

RESULTS: Varying results from generative AI tools A and B highlighted their limitations. Tool A had difficulties identifying sub-populations and comparators and was restricted to attaching only one PDF for support, hence limiting the research. Tool B resulted in only 8 PICOs based on provided guidelines, and temporarily restricted access to tool’s latest version. Tools often used European guidelines for country-specific requirements resulting in inconsistent number of PICOs. Overall, identifying correct local guidelines and comparators was challenging. Testing HTA reports adds another layer of complexity. Manual scoping resulted in 14 PICOs based on European and local guidelines. A thorough selection and review of sources resulted in a comprehensive PICO prediction and consolidation exercise.

CONCLUSIONS: Lack of well-trained and robust AI tools still makes human involvement necessary for an essential step like PICO prediction. There is a need for a well-trained team who understands JCA requirements, European guidelines, local guidelines, and preferred PICOs by HTA bodies. Maintaining quality and confidentiality remains critical for such exercises.