Kappa statistics

Cohens Kappa

1. DATA-BASEMANAGEMENT
Kappa statistics
DR. AMEY DHATRAK

2. Kappa statistics
◦The kappa statistic is frequently used to test interrater reliability.(Measurement of the extent to
which data collectors (raters) assign the same score to the same.)
◦Why we need Kappa???
◦Some studies involve the need for some degree of subjective interpretation by observers and
often measurement differs with different ‘raters.
◦intraobserver variation
◦intrasubject variation
◦For example:
◦Interpreting x-ray results = Two radiologists reading same chest x-ray for signs of pneumoconiosis
◦Two laboratory scientists counting radioactively marked cells from liver tissue
◦Often same rater differs when measuring the same thing on a different occasion

3. Kappa-agreement
◦Without good agreement results are difficult to interpret
◦Measurements are unreliable or inconsistent
◦Need measures of agreement -kappa
◦Remember-
Extent to which the observed agreement exceeds that which would be expected by
chance alone (i.e., percent agreement observed − percent agreement expected by
chance alone) [numerator] relative to the maximum that the observers could hope to
improvetheir agreement (i.e., 100% − percent agreement expected by chance alone)
[denominator].

4. Formula 

++
++
−
−
=
ii
iiii



1

5. Examples:-1
44*60/100= 26.4
31*40/100= 12.4

8. Two raterswith binary measure
15 5
4 35
No
Biomarker
present
No
Biomarker
present
Rater1
Rater
2
Examples:-2 ( slightly different method)

9. Cohen’s Kappa Statistic (κ)
Measures agreement between ratersmore than expected by chance

++
++
−
−
=
ii
iiii



1
represents the marginal probabilities and i= 1,2 the score

10. Two raterswith binary measure
15 5 20
4 35 39
19 40 59
No
Biomarker
present
No
Biomarker
present
Rater1
Rater
2
Marginal Total
Marginal
Total

11. Two raterswith binary measure
15 520
4 3539
194059

ii= (15 + 35)/59
= 0.847

i+
+i= (20x19 + 40x39)/59
2
= 0.557

12. Two raterswith binary measure
15 520
4 3539
194059

ii= 0.847

i+
+i= 0.557

++
++
−
−
=
ii
iiii



1 557.01
557.0847.0
−
−
= 654.0=

14. Confidence intervals for kappa
◦Given that the most frequent value desired is 95%, the formula uses
1.96 as the constant.
◦The formula for a confidence interval = κ –1.96 x SEκto κ + 1.96 x Seκ
◦To obtain the standard error of kappa (SEκ) the following formula
should be used:

17. Thank You