OBJECTIVES: Multiple myeloma is marked by elevated rates of relapse. Timely and large-scale analysis of adverse events is critical due to the dynamic nature of multiple myeloma treatments and the evolving landscape of therapeutic interventions. We aimed for a comprehensive analysis of adverse events on a large scale of multiple myeloma treatment studies utilizing large language models (LLMs).

METHODS: We collected 231 multiple myeloma clinical trial conference abstracts and journal publications (2012-2023) comprising 130 chimeric antigen receptor T cell (CAR-T), 63 bispecific antibody/bispecific T cell engagers (BsABs/BiTE), and 38 antibody-drug conjugate (ADC) studies from conferences and PubMed sites. We developed an LLM based information extraction model and conducted quantitative evaluations of the pipeline using a manually curated gold standard.

RESULTS: Our system achieved 0.955 precision, 0.939 recall, and 0.947 F1 across 70 data entities defined by our clinical trial outcome ontology. It also demonstrated consistent precision, recall, and F1 of 0.969, 0.954, and 0.961 across other cancer types. Examining adverse event-related entities, neutropenia prevailed in all therapies (CAR-T: 13.4%, BsAbs/BiTE: 13.5%, ADC: 12.4%). Cytokine release syndrome rates were notable (CAR-T: 13.0%, BsAbs/BiTE: 9%, ADC: 8.6%). Infections occurred in 4.8% of CAR T studies, with higher percentages in BsAbs/BiTE (12%) and ADC studies (8.6%). Fatal adverse events varied by phases and therapies (CAR-T: 18.7%, 25%, 56%; BsAbs/BiTE: 20%, 40%, 40%; ADC: 42.8%, 14.3%, 42.8% in phases 1, 1/2, and 2&3, respectively). This result indicates increased incidence of fatal adverse events observed during extended follow-up and larger study populations across phases 2 and 3 studies.

CONCLUSIONS: Our study has shown a comprehensive profiling of adverse events across therapies of multiple myeloma studies. The LLM based adverse event profiling allows for timely large-scale dataset analysis and can be generalized to other therapeutic areas, significantly advancing methodologies within health economics and outcomes research.