Group 1 ROC assignment explaining ROC and AUC

This PPT explains ROC and AUC curves

1. RECEIVER OPERATING CHARACTERISTIC CURVE GROUP 1 EAB 1 FEB 2025

2. OUTLINE Introduction Types of ROC Curves Area Under the Curve (AUC) and Interpretation Practical Example and Visualization Partial AUC and Its Importance Effect of Thresholds Advantages and Disadvantages of ROC Curves Potential Issues in ROC Analysis Comparison with Other Methods

3. Introduction The ROC curve is a statistical tool used to evaluate the performance of a binary diagnostic classification method. Many diagnostic tests provide continuous results, requiring a cut-off value for decision-making. ROC analysis helps determine the optimal cut-off point

4. Introduction Purpose of ROC Curves Assess the overall diagnostic performance of a test. Compare multiple diagnostic tests. Determine the optimal cut-off value to balance sensitivity and specificity.

5. TPES OF ROC Curves Parametric (Binary) Method Forms a smooth curve by expanding sample size and connecting countless points Compare plots at any sensitivity and specificity values AUC may be biased if normality assumptions are incorrect Assumption Normal distribution of data Nahm, F. S. (2022). Receiver operating characteristic curve : overview and practical use for clinicians .

6. TPES OF ROC Curves Non-Parametric (Empirical) Method Produces a jagged or staircase-like curve. Uses all observed data points. No normality assumption required, making it more robust. Easier to compute with unbiased estimates of sensitivity and specificity.

7. Receiver Operating Characteristic (ROC) Curve Rajeev, K. A. I. (2025). Receiver Operating Characteristic (ROC) Curve for Medical Researchers .

8. Area under the curve (AUC) AUC summarizes ( quantifies ) the overall accuracy of a diagnostic test. Values range from 0 to 1: AUC=1 : perfectly accurate test AUC > 0. 8 : Good-Excellent test AUC 0.5: No discrimination (random guessing) AUC < 0.5: Worse than random guessing AUC=0: perfectly inaccurate test

9. Interpretation of the Area Under the Curve Nahm, F. S. (2022). Receiver operating characteristic curve : overview and practical use for clinicians .

10. Practical example and visualization

11. Software SPSS Stata R SAS Python Data Cervical cancer screening: Pap-smear, HPV-test ,Colposcopy

12. PARTIAL AUC Used when specific values of sensitivity and specificity are clinically relevant If ROC curves of two different tests cross at some point-Side-perform differently at key thresholds. In these two cases, partial AUC may be a more meaningful than the overall AUC

13. PARTIAL AUC (ROC) curves with an equal AUC Although the AUC is the same, the features of the ROC curves are not identical. Test B shows better performance in the high false-positive rate range than test A Test A is better in the low false-positive range. In this example, the partial AUC ( pAUC ) can compare these two ROC curves at a specific false positive rate range. Nahm, F. S. (2022). Receiver operating characteristic curve : overview and practical use for clinicians .

14. EFFECT OF THRESHOLDS For the same diagnostic test sensitivity and specificity vary with the thresholds used. Generally: High threshold : good specificity, medium sensitivity Medium threshold : medium specificity, medium sensitivity Low threshold : good sensitivity, medium specificity Extreme low threshold : no specificity, perfect sensitivity

15. common interpretations of AUC the average value of sensitivity for all possible values of specificity the average value of specificity for all possible values of sensitivity the probability that a randomly selected patient with disease has positive test result that indicates greater suspicion than a randomly selected patient without disease when higher values of the test are associated with disease and lower values are associated with non disease.

16. The ROC curve : advantages. Displays all possible cut-off points and one can read the optimal cut-off Independent of diseases prevalence, unlike predictive values (PPV,NPV), therefore, samples can be taken regardless of the prevalence of a disease in the population Allows comparison of multiple tests in one graph Sometimes sensitivity is more important than specificity or vice versa- ROC helps in finding the required value of sensitivity at fixed values of specificity Useful summary of measures can be obtained for determining the validity of diagnostic test such as AUC and partial area under the curve

17. The ROC curve : Disadvantages. The cut-off value for distinguishing normal from abnormal is not directly displayed on the ROC curve and neither is the number of samples. The ROC curve appears more jagged with a smaller sample size, a larger sample does not necessarily result in a smoother curve.

18. Potential ROC issues Lack of gold standard for diagnosis Lack of reproducibility –E.g., disagreement among pathologists Bias in sample selection, spectrum of disease used in evaluating test –Choose sickest patients, healthy controls Problems in ascertainment –Genetic disease may not be manifest Can’t always reliably measure ROC area (Few cases with disease, imbalanced datasets)

19. Solutions to Potential ROC issues Lack of gold standard for diagnosis composite reference standard : combining multiple diagnostic criteria clinical findings, laboratory tests, imaging results, or expert opinions Lack of reproducibility T raining s . Multiple independent raters and measure agreement (e.g., Cohen’s kappa). Implement AI-assisted decision tools to reduce human variability

20. Solutions to Potential ROC issues Bias in sample selection Ensure representative sampling Problems in ascertainment Longitudinal follow-up to assess test performance over time. Implement repeat testing to detect cases that initially appear negative. Can’t always reliably measure ROC area Report confidence intervals for AUC instead of a single point estimate. Use bootstrapping to improve AUC estimation.

21. References Nahm, F. S. (2022). Receiver Operating Characteristic Curve: Overview and Practical Use for Clinicians.