OBJECTIVES: Health economic models (HEMs) are vital for assessing the cost-effectiveness of healthcare interventions. Advances in artificial intelligence (AI), particularly with large language models (LLMs) like the GPT-4o, have shown potential to revolutionize various domains by automating complex cognitive tasks. This study aims to evaluate the feasibility and accuracy of using LLMs to conceptualize HEMs.

METHODS: A proof-of-concept exercise was undertaken and a cost-effectiveness model for an anti-cancer therapy in advanced breast cancer was developed. Publicly available data sources were considered. A Human Intelligence (HI) in-the-loop approach was utilized, combining the strengths of LLMs with the critical oversight and expertise of human health economists. Various reasoning techniques within the LLM, such as Chain of Thought (CoT), Tree of Thought (ToT), and CoT-Self-Consistency, were explored. Additionally, the Retrieval-Augmented Generation (RAG) approach was employed to enhance information retrieval and integration, all implemented in Python.

RESULTS: The LLM recommended a natural history which was further refined using the HI-in-loop approach. The initial recommendation was promising and closely aligned with what human experts might have generated through manual conceptualization. For the model structure, LLM suggested a Markov model with four health states: "Progression-Free Survival" (PFS), "First Progression," "Second Progression," and "Death." Key parameters were identified, and gaps were highlighted.

CONCLUSIONS: The results from this case study demonstrated the potential of LLMs in the conceptualization of HEMs. While LLMs are becoming increasingly adept at reasoning, an open and agile approach is crucial for their usability in HEOR rather than blindly adapting them. Integrating human feedback into the loop not only enhances the accuracy and reliability of the models but also introduces a dependency on highly skilled human modelers, which can be a bottleneck. This highlights the need for a balanced approach to utilizing LLMs in HEOR, ensuring that AI complements rather than replaces human modellers.