OBJECTIVES: To investigate reasons behind discontinuation and switching of biologic treatments for inflammatory bowel disease (IBD) leveraging large language models (LLMs) applied to unstructured clinical notes from electronic health records (EHRs).

METHODS: EHRs (2017-2023) from 4 health systems in the OMNY Health real-world data platform were accessed; patients with an IBD diagnosis code were selected. A two-phase methodological approach utilizing LLMs was employed: (1) a Robustly Optimized Bidirectional Encoder Representations from Transformers Pretraining Approach (RoBERTa)-based model (pretrained on the Stanford Question Answering Dataset 2.0) was used to identify reasons for treatment discontinuation and switching within clinical notes and (2) these initial findings were used as ground truth data to fine tune a Gemini Pro 1.5 model to aid in precision and generalizability in identifying real-world clinical nuances. Model comprehensiveness in extracting relevant information was measured to ensure accuracy in inferring reasons for treatment discontinuation and switching from broader contexts.

RESULTS: Among 10.6 million notes, the model achieved 94.5% accuracy in identifying 7 distinct reasons for treatment alteration across 7 biologics. Reasons for treatment alteration in decreasing order of prevalence (with the range of the top 3 most prevalent biologics) were adverse drug event (16-28%), finance-related reasons (4-24%), patient-related reasons (2-9%), lack of efficacy (1-14%), symptom resolution (1-4%), drug-disease interaction (1-3%), and obstetric (0-2%).

CONCLUSIONS: Our LLM-driven approach successfully extracted and categorized reasons for discontinuing or switching biologics in IBD demonstrating its ability to derive desired insights from unstructured clinical notes. These insights not only advance our understanding of treatment challenges in IBD but also support the development of more effective, personalized treatment strategies aligning with individual patient outcomes. Application of LLMs on a broader scale could be useful in identifying reasons for treatment alteration and other clinical concepts of interest to extract data not traditionally captured in structured EHR fields.