OBJECTIVES: Using large language models (LLMs) such as Generative Pre-trained Transformer 4 (GPT-4) to edit Microsoft Word files could revolutionize the reporting of health economic models. This study aimed to assess GPT-4’s capabilities in automatically updating a Word technical report for a cost-utility model that was used in health technology assessments (HTAs) for muscle-invasive urothelial carcinoma (MIUC).

METHODS: The MIUC model was first manually edited. Then, utilizing GPT-4 and deterministic programming, the Word technical report was automatically updated to reflect the Excel model. GPT-4 updated text in the results and conclusion sections, and all automated edits were captured in tracked changes. Two experienced health economists then blindly assessed the AI-adapted report alongside a manually adapted report that was developed by a third health economist. Accuracy was evaluated based on correct/incorrect changes, correct/incorrect retainment of original text, and instances of missing information.

RESULTS: Both reviewers identified more than 30 instances for scoring. Accuracy was 94.3% for the AI-adapted report and 98.5% for the manually adapted report. The reviewers agreed there were 2 incorrect changes in the AI-adapted report: a rounding error and an incorrect description of a scenario analysis. Qualitatively, the reviewers generally approved of the tone of edits made by GPT-4. However, there were a small number of factually correct edits where the reviewers preferred language chosen by the human health economist.

CONCLUSIONS: This study is a promising early indication that LLMs can be leveraged as a part of a reviewer-friendly pipeline for automatically updating model technical reports in Microsoft Word. The accuracy achieved in our study suggests suitability as a first editor prior to human review. Utilizing AI to adapt technical reports for HTA submissions could accelerate dissemination of health technologies around the world.