5 EPIDIMIOLOGY GROUP 5-1.pptx Assessinging

It very important to know the assessing of diagnostic and screen test

1. ASSESSING THE VALIDITY AND RELIABILITY OF DIAGNOSTIC AND SCREENING TESTS EPIDEMIOLOGY PRESENTATION KIU GROUP 5 SUPERVISOR: DR.

2. GROUP MEMBERS FAZILA MALENGERA FORTUNATA J KALIBAJUNA NDUWAYESU SEMEKO PIERRE KAMBALE MATABISHI

3. OUTLINE 0. LEARNING OBJECTIVES I. INTRODUCTION II. RELIABILITY AND VALIDITY II. 1. VALIDITY • Se n sitivity • Specificity • Sequential and simultaneous testing • PPV and NPV • PPV and prevalence • PPV and specificity II. 2. R ELIABILITY • Percent agreement • Kappa

4. . LEARNING OBJECTIVES To define the validity and reliability of screening and diagnostic tests. To compare measures of validity, including sensitivity and specificity. To illustrate the use of multiple tests (sequential and simultaneous testing). To introduce positive and negative predictive value. A ddress measures of reliability, including percent agreement and kappa

5. I. INTRODUCTION To understand how a disease is transmitted and develops and to provide appropriate and effective health care, it is necessary to distinguish between people in the population who have the disease and those who do not. It is therefore important to use a reliable and dependable screening tool It can either be physical examination, imaging, investigation and laboratory tool etc…

6. INTRODUCTION… In using a test to distinguish between individuals with normal and abnormal results , it is important to understand how characteristics are distributed in human populations

7. II. R ELIABILITY AND V ALIDITY • Reliability and Validity are terms that assess reproducibility and accuracy of your measures (and your overall study).

8. Definitions • Validity is the degree to which a result (of a measurement or study) is likely to be true and free of bias or systematic errors. • Reliability , or reproducibility , is the amount of agreement or consistency between results of repeated measurement or assessment , of the same sample .

9. • The validity of a test is defined as its ability to distinguish between who has a disease and who does not. Validity has two components: sensitivity and specificity Sensitivity • The sensitivity of the test is defined as the ability of the test to identify correctly those who have the disease Specificity • I s defined as the ability of the test to identify correctly those who do not have the disease. II. 1 VALIDITY OF SCREENING TEST

10. Two-by-Two Table Sensitivity and Specificity Disease No Disease Test Result Positive Negative 80 100 20 800 100 900 True Disease Status Sensitivity = 80 / 100 = 80% Specificity = 800 / 900 = 89%

11. Use of multiple tests It involves the use of ; Sequential testing Simultaneous testing

12. S equential test In sequential (or two -stage) screening, a less expensive , less invasive, or less uncomfortable test is generally performed first, and those who screen positive are recalled for further testing with a more expensive , more invasive, or more uncomfortable test, which may have greater sensitivity and specificity It is hoped that bring - ing back for further testing only those who screen positive will reduce the problem of false positives

13. Sequential test…. Net sensitivity = (315/500)*100 = 63% Thus, there is a loss in net sensitivity Net specificity = { (1710+7600)/9500 }*100 = 98% Therefore, there is a gain in net specificity

14. Simultaneous Testing • High sensitivity testing strategy • Sensitivity – positive on either test • Specificity – negative on both tests Test A + - Test B + -

15. Simultaneous Testing Disease No Disease Positive Test 160 320 Negative Test 40 480 Total 200 800 Results of Screening Test A Sensitivity = 80% Specificity = 6 0% POP 1000 P revalence is 20%

16. Disease No Disease Positive Test 180 80 Negative Test 20 720 Total 200 800 Results of Screening Test B Sensitivity = 90% Specificity = 90% Simultaneous Testing

17. Simultaneous Testing Test Result D D + - 200 180 20 800 80 720 Test A Sensitivity = 80% Specificity = 60% Test Result D D + - 200 800 160 40 320 480 Test B Sensitivity = 90% Specificity = 90% 160 x 0. 90 = 144 or 180 x 0. 80 = 144 160-144 = 16 180-144 = 36 Net sensitivity (16 + 144 + 36) / 200 = 196 / 200 = 98% (160+180) - 144 = 196

18. Simultaneous Testing Test Result D D + - 200 180 20 800 80 720 Test A Sensitivity = 80% Specificity = 60% Test Result D D + - 200 800 160 40 320 480 Test B Sensitivity = 90% Specificity = 90% Net specificity 432 / 800 = 54% (60 x 90) = 54% 480 x .90 = 432 720 x .60 = 432

19. Validity and Combination Testing • Sequential Testing • Sensitivity reduced (more FN) • Specificity improved • Simultaneous Testing • Sensitivity improved • Specificity reduced (more FP)

20. Predictive Value of Test • Positive Predictive Value • The proportion of people with a positive test who have disease • If the test is positive, what is the probability the patient truly has disease? • Negative Predictive Value • The proportion of people with a negative test who are healthy • If the test is negative, what is the probability the patient truly does not have disease

21. Positive Predictive Value = True positives / Total tested positive Disease No Disease Test Result Positive Negative True Positive False Positive False Negative True Negative Total tested positive True Disease Status Total tested negative Positive Predictive Value

22. Disease No Disease Test Result Positive Negative True Positive False Positive False Negative True Negative Total tested positive True Disease Status Total tested negative Negative Predictive Value = True negative / Total tested negative Negative Predictive Value

23. Positive Predictive Values = 80 / 180 = 44% Negative Predictive Value = 800 / 820 = 98% Disease No Disease Test Result Positive Negative 80 100 20 800 100 900 True Disease Status 180 820 Predictive Value

24. Predictive Value and Prevalence Truth Disease No Disease Test Positive 99 495 594 Negative 1 9405 9406 100 9900 10,000 Sensitivity = 99 / 100 =99% Specificity = 9405 / 9900 = 95% PPV = 99 / 594 = 17% NPV = 9405 / 9406 = 99.9% 1% Prevalence Truth Disease No Disease Test Positive 495 475 970 Negative 5 9025 9030 500 9500 10,000 Sensitivity = 495 / 500=99% Specificity = 9025 / 9500 = 95% PPV = 495 / 970 = 51% NPV = 9025 / 9030 = 99.9% 5% Prevalence

25. Predictive Value and Specificity Truth Disease No Disease Test Positive 1000 2700 3700 Negative 6300 6300 1000 9000 10,000 Truth Disease No Disease Test Positive 1000 450 1450 Negative 8550 8550 1000 9000 10,000 Sensitivity = 1000 / 1000=100% Specificity = 8550 / 9000 = 95% PPV = 1000 / 1450 = 69% NPV = 8550 / 8550 = 100% 95% Specificity Sensitivity = 1000 / 1000 =100% Specificity = 6300 / 9000 = 70% PPV = 1000 / 3700 = 27% NPV = 6300 / 6300 = 100% 70% Specificity

26. II.2. R ELIABILITY = Repeatability • Reliability : Is another aspect of assessing diagnostic and screening test. Amount of agreement or consistency between results of repeated measurement or assessment, of the same sample We have the factors that contribute to the variation between test results: • Intra subject variation I ntra observer variation • Inter-observer variation

27. Ways to Measure Reliability • Percent agreement • The proportion of all observations where two (or more) readers agree • Kappa Statistic • Calculates the agreement beyond chance alone Observed agreement E xpected agreement

28. Percent Agreement Observer 1 Observer 2 Definite Possible Unlikely Definite A B C Possible D E F Unlikely G H I Percent Agreement = A + E + I N N = A + B + C + D + E + F + G + H + I

29. Problems with Percent Agreement Observer 1 Observer 2 Definite Possible Unlikely Definite 5 1 1 Possible 2 6 3 Unlikely 1 1 100 Percent Agreement = 5+6+100/120 = 0.92

30. Kappa Statistic • A measure of agreement between observers or measurements of the same categorical variable beyond that agreement due to chance alone Kappa = Observed agreement – expected agreement 100 – expected agreement

31. Kappa Statistic: Observed Agreement Abnormal Normal Abnormal A B A+B Normal C D C+D A + C B + D N Kappa = observed agreement – expected agreement 100 – expected agreement Observed Agreement = % Agreement = (A + D) / N

32. Kappa Statistic: Expected Agreement Obs 2 Abnormal Normal Obs 1 Abnormal A B A+B Normal C D C+D A + C B + D N Obs 2 Abnormal Normal Obs 1 Abnormal ((A+C)/N) * (A+B) ((B+D/N)*(A+B) Normal ((A+C)/N)*(C+D) ((B+D)/N)*(C+D) N Observed Expected

33. Kappa Statistic: Expected Agreement Obs 2 Abnormal Normal Obs 1 Abnormal ((A+C)/N) * (A+B) ((B+D/N)*(A+B) Normal ((A+C)/N)*(C+D) ((B+D)/N)*(C+D) N (((A+C)/N) * (A+B)) + ((B+D)/N) * (C+D)) N Expected Agreement =

34. Kappa Statistic Abnormal Normal Abnormal 26.4 17.6 Normal 18.6 12.4 75 Abnormal Normal Abnormal 41 3 44 Normal 4 27 31 45 30 75 Percent Agreement = 41 + 27 75 = 90.7 Expected Observed Expected Agreement = (26.4 + 12.4) / = 51.7% 75 Kappa = 90.7 –  51.7 100 –  51.7 = 0.81

35. Interpreting Kappa • > 75%: Excellent agreement • 40% –  75%: Fair to good agreement • < 40%: Poor agreement Fleiss (1981)

36. Criteria for a Screening Test • Important public health problem (Morbidity and mortality) • The natural history of the disease should be adequately understood • There should be a latent stage of the disease • There is a safe and effective treatment • The test should be acceptable to the population • The total cost of finding a case should be economically balanced in relation to medical expenditure as a whole • Case-finding should be a continuous process, not just a “once and for all” project • A proven and acceptable test exists to detect individuals at an early, modifiable stage (valid and reliable)

37. REFERENCES Gordi‘s epidemiology, 6 th edition Public Health and Epidemiology at a Glance-Margaret Somerville Basic epidemiology, 2 nd edition A Dictionary of Epidemiology, Last et al.(2001)

38. END THANK YOU FOR LISTENING