OBJECTIVES: The emergence of AI has transformed various fields within healthcare, presenting an opportunity to revolutionize the extraction of data from large documents without human intervention. This research aimed to assess the potential use of LLM in market access when extraction of data from NICE FAD is required.

METHODS: Five NICE FADs published in 2023 were selected to represent a range of disease areas and marketing authorization scenarios (oncology [monotherapy/combinations], add-on therapy, rare diseases/highly specialized technologies). Six topics were defined for extraction: intervention, recommendation, clinical, economic, severity modifier, and differentiators. A script was developed to prompt the LLM (Generative Pre-trained Transformer 4 [GPT-4o]) to extract the pre-defined qualitative and quantitative data for each topic. The data extracted by the LLM was compared to double human extraction for completeness/accuracy.

RESULTS: The LLM achieved 100% success when extracting quantitative data across all topics as well as for the extraction of qualitative data relating to intervention and severity modifier. LLM achieved 96% accuracy overall across differentiators including innovation, end-of-life, equality, and patient/clinical expert input, 94% for clinical data, and 92% for NICE recommendation. In contrast, only 50% success rate was achieved for extraction of qualitative data relating to economic modelling.

CONCLUSIONS: This feasibility study suggests LLM could be used to accurately and efficiently extract all quantitative data and most qualitative data from NICE FAD. However, human intervention is still currently required when extracting economic model-related qualitative data. The feasibility of using current generation LLM to extract FAD data using a written script could result in time savings when analyzing market access trends in NICE decision-making when large numbers of FAD need extracting. However, any efficiency gains are currently offset by the time required to develop and test the script, meaning human extraction is still most efficient where smaller numbers of FAD are being analyzed.