OBJECTIVES: Expert elicitation is an invaluable method for obtaining expert judgments, particularly in areas with limited evidence. Health technology assessment (HTA) guidelines, such as the NICE Methods Guide, recommend expert elicitation to address evidence gaps. While commonly used protocols such as SHELF, IDEA and MRC protocol in expert elicitation share some similarities, they also differ significantly in design and execution. This project aims to provide practical recommendations for selecting and adapting these protocols, with a focus on elicitation formats based on insights gained from a case study employing various approaches to gather expert opinions.

METHODS: We adapted the SHELF protocol to elicit long-term survival estimates in two workshops: one online with six experts and one face-to-face with five different experts. Additionally, we administered an online survey with three questions (two qualitative and one quantitative) to gather opinions on healthcare resource use.

RESULTS: The entire process, from preparation to report writing, took four months. The project team received an overwhelmingly positive response from the clinical community, prompting the scheduling of two workshops instead of one to maximise expert participation. We found that the online workshop was as effective as the face-to-face workshop and may offer several advantages. The group discussion component in the elicitation exercise proved extremely valuable, enhancing the credibility of the elicited values. Although the preparation of the online survey required less effort, the responses were heterogeneous, making it challenging to draw definitive conclusions.

CONCLUSIONS: Online surveys may not be suitable for eliciting critical model input parameters. Group facilitation and expert interaction are essential in elicitation exercises to ensure the credibility, accuracy, consistency, and transparency of expert judgments. Contrary to concerns in the literature, time constraints and expert availability may not pose significant issues in HTA practice. When conducted properly, online elicitation workshops can also deliver high-quality outcomes.