Prompt Engineering for the Use of Generative Artificial Intelligence (AI) in Health Economic Modeling: Findings From a Targeted Literature Review

OBJECTIVES: As large language models (LLMs) are increasingly employed across research settings, including health economics and outcomes research (HEOR), effective prompt engineering techniques are needed to optimize LLM performance. A targeted literature search was conducted to identify prompt engineering methods and understand applications of generative artificial intelligence (AI) to health economic modeling.

METHODS: Targeted literature searches were conducted using the arXiv database (through 6/14/2024) to identify the types of prompt engineering methods 1) available for research and 2) used in health economic model development. Additional studies were identified through citation searching.

RESULTS: The searches identified 13 relevant papers. Three reviews summarized over 30 unique prompting techniques. Ten additional studies more closely examined 12 of these methods. Of the techniques described in the literature, few-shot and chain-of-thought (COT) have been shown to enhance response quality compared to zero-shot prompting. Few-shot prompting is a method in which the LLM is given one or more examples prior to the desired query. This reduces the model’s reliance on pre-trained knowledge and has demonstrated comparable results to traditional fine-tuning. COT prompting, which involves providing examples of intermediate reasoning steps or directing the LLM to “think step-by-step,” has been shown to improve response quality in arithmetic, commonsense, and symbolic reasoning settings. Variations of this approach, such as structured COT, program-of-thoughts, and chain-of-code, have also been successful in programming applications. None of these techniques have specifically been evaluated in a health economic modeling context.

CONCLUSIONS: A budding literature exists on available prompting methods to enhance the capabilities of LLMs, and this search highlights the need for additional literature on prompt engineering within HEOR. Few-shot and COT prompting may be applicable to economic modeling due to their broad generalizability and minimal user expertise required for implementation. Further research should be conducted to evaluate these techniques for the development of health economic models.