OBJECTIVES

RCTs are gold-standard for assessing the effectiveness of an intervention. However, for diagnostic tools, it is possible to ensure that the only difference in outcomes is due to the accuracy of the device by using both devices concurrently in one individual. This same-person design can be more accurate in assessing accuracy than RCTs. However, problems arise where we are assessing a body of evidence that includes same-person designs and parallel-group RCTs. The Cochrane Risk of bias (ROB₂) tool is widely used to assess the quality of RCTs but undervalues the quality of the parallel within-subject designs that are non-randomised. We aimed to learn what assessment tools would be most appropriate for this situation.

METHODS

We conducted a systematic search of MEDLINE, Embase and PubMed in February 2020 for publications using a quality assessment tool for diagnostic/monitoring studies potentially using a same-person design. 9,441 abstracts were double-screened and the 92 that met the inclusion criteria were summarised in an Evidence Map.

RESULTS

The most relevant tool was QUADAS-2, which is designed to assess the risk of bias and applicability of diagnostic accuracy studies and was cited in 38% of publications. The QUADAS-2 score is based on an assessment of patient selection, the index test, reference standard, flow of patients and timing of the tests. An advantage of QUADAS-2 is the tailoring process through which the tool can be adjusted to best fit the review content. The flexibility of the tool allows for comparison of RCTs and same-person designs without devaluing the latter, but does not fully recognise the superiority of the same-person design.

CONCLUSIONS

While QUADAS-2 is an acceptable tool for assessing the quality of same-person diagnostic studies it does not allow a meaningful comparison between RCTs, and is not yet used widely and consistently.