Table 3 Quality criteria for study outcomes for COSMIN. Based on Terwee et al. [29]

Criterion validity

+

Convincing arguments that criterion standard is “gold” AND correlation OR ICC with criterion standard ≥0.70.

?

No convincing arguments that gold standard is “gold” OR no correlations OR ICC have been calculated.

–

Correlation OR ICC with criterion standard < 0.70, despite adequate design and method.

Reliability (intra-rater, inter-rater)

+

ICC or weighted Kappa ≥0.70 OR Pearson γ or Spearman ρ ≥ 0.80 with evidence provided that no systematic change has occurred.

?

No ICC, weighted Kappa, Pearson γ or Spearman ρ is determined OR doubtful design^a.

–

ICC or weighted Kappa < 0.70 OR Pearson γ or Spearman ρ < 0.80, despite adequate design and method.

Measurement error

+

SEM < SDC OR SEM outside the LOA

?

SEM is not defined OR doubtful design^a.

–

SEM ≥ SDC OR SEM equal or inside the LOA