OBJECTIVES: Psychometric analysis is vital in drug development to obtain reliable and valid clinical outcomes assessments (COAs) for evaluating drug effects, understanding patient experiences, aiding diagnosis, and supporting clinical decision-making. However, communicating results to non-technical decision-makers presents challenges. Interpretation can be difficult, and success criteria are often applied without considering uncertainty in the psychometric results, which can create false certainty regarding the performance of COAs and lead to underpowered trials with misleading results or ineffective COAs. New research has shown that data visualizations can effectively illustrate uncertainty in statistical estimates. This poster presents a method called the quantile dot plot, which uses a frequency-framing visualization to convey the uncertainty of common psychometric results to non-experts.

METHODS: Common psychometric analyses (e.g., factor analysis, reliability, and validity analyses) were performed on a simulated dataset containing a COA, co-validating COAs, and global impression measures. Analyses were conducted using a Bayesian framework, allowing for the visualization of posterior distributions of each result using quantile dot plots, which were color-coded based on general guidelines for interpreting performance. The probability of a specific result is determined by counting the dots in the plot that meet or exceed the result.

RESULTS: The quantile dot plots effectively displayed plausible estimates and levels of uncertainty for each psychometric result. Some plots indicated that while the modal estimate met the performance criterion, there was a higher likelihood of not meeting the criterion than expected, which was determined by counting the dots.

CONCLUSIONS: Quantile dot plots offer an accessible visualization method for communicating complex continuous probabilistic distributions to non-experts and decision-makers, and clearly illustrates the uncertainty of results, which is critical to consider when developing endpoint strategies. By using this visualization method, decision-makers can better understand the reliability and validity of COAs, leading to improved patient care and more accurate clinical decision-making.