OBJECTIVES: The impending Joint Clinical Assessment (JCA) regulations will result in the need for precise and comprehensive extraction of Patient, Intervention, Comparison, and Outcome (PICO) elements from a vast number of clinical trials. The recent rise of Generative AI (GenAI) has shown great promise in a wide range of fields including health economics and outcomes research (HEOR) with GenAI models becoming faster, more flexible and more accurate. The objective of this study was therefore to evaluate the feasibility, accuracy, and efficiency of using GenAI for mass extraction of PICOs from PubMed abstracts.

METHODS: Relevant abstracts were identified using the search string corresponding to the PubMed clinical trials filter. PICOs were extracted from all individual abstracts using GPT-4 Omni (GPT-4o). The extraction was performed using the OpenAI batch API, which allowed parallel processing of 682,667 abstracts. To determine the accuracy of GenAI PICO extraction a random subsample was selected for human checking, containing 274 clinical trials.

RESULTS: From the sub sample of 274 clinical trials GPT-4o comprehensively and accurately extracted 269 (98%) of all PICOs. In the remaining 5 cases, the model occasionally missed some outcome elements but always extracted the population, intervention, and comparator accurately. The entire process resulted in the extraction of 682,667 PICOs in less than 3 hours.

CONCLUSIONS: This study highlights the potential of GenAI, specifically GPT-4o, for large-scale, rapid PICO extraction and to our knowledge represents the first instance of PICO extraction at this speed, scale and accuracy. While accuracy is high, with 98% of extractions being fully correct, further human validation may be necessary to ensure further adoption in clinical research settings. This promising application of AI will likely be beneficial for meeting JCA requirements and may potentially open up a new paradigm for how systematic literature review (SLR) is conducted.