OBJECTIVES: Despite its potential, the use of Generative-AI in health economic modeling is still emerging. Leveraging Generative-AI to streamline health economic model development could notably benefit stakeholders by reducing the time and expertise required. This study examines the feasibility and accuracy of Generative-AI in replicating health economic models using a well-established benchmark.

METHODS: We replicated the Markov model for HIV/AIDS from Briggs et al.'s book, "Decision Modeling for Health Economic Evaluation," using a two-step approach. First, we used Python for large language model (LLM) interactions to extract model structure and parameter values from the chapter PDF. We utilized ValueGen.AI, a GPT-4-based platform utilizing multi-agent pipelines with LangGraph, LangChain, and OpenAI libraries for parameter extraction. The extracted data included states, transition probabilities, state and treatment costs, discount rate, cycle length and time horizon. These parameters were then transferred to the R programming language’s Heemod package to construct and run the Markov model. We evaluated life years and costs for each strategy and calculated the incremental cost-effectiveness ratio (ICER). We compared the Generative AI-based model's structure, input parameters, and generated outcomes against the published chapter.

RESULTS: The Generative AI-based model effectively extracted parameters from the chapter. It estimated the average cost of the monotherapy arm to be $48,400, compared to $44,663 reported in Briggs et al. (8% error). Similarly, the model predicted 8.46 life years for the monotherapy arm, compared to 8.47 reported (0.1% error). For the ICER, the Generative AI-based model estimated $6,400 per life-year gained, compared to $6,276 reported (2% error). We repeated the experiment 20 times and error margins remained consistent

CONCLUSIONS: We demonstrate the feasibility of fully and accurately replicating published ‘simple’ Markov-based health economic models using Generative AI. Future research should focus on replicating more complex health economic models.