External Quality Control Lecture MD General 2014 Course, Clin Path Ain Shams University, Egypt

Short description of the categories of proficiency testing programs, management of external quality control assays, and evaluation of results

1. Quality Management in Medical Laboratory External Quality Control Dr. Adel A. Elazab Associate Professor of Clinical Pathology Faculty of Medicine Ain Shams University, Cairo, Egypt

2. Inter-laboratory comparisons and other performance evaluations that may extend throughout all phases of the testing cycle, including interpretation of results; determination of individual and collective laboratory performance characteristics of examination procedures by means of interlaboratory comparison External Quality Assessment (EQA)

3. A program in which multiple samples are periodically sent to members of a group of laboratories for analysis and/or identification, in which each laboratory’s results are compared with those of other laboratories in the group and/or with an assigned value, and reported to the participating laboratory and others Proficiency Testing (PT)

4. Introduced into laboratory medicine more than 60 years ago as an educational tool to address observations that results for aliquots of the same sample were different when measured by different laboratories. PT/EQA programs have evolved in scope and sophistication and are now an essential component of a laboratory’s quality management system. PT/EQA is a component of laboratory accreditation requirements PT/EQA programs

5. Preparation Steps for Serum-Based PT/EQA

6. Ideal samples for a PT/EQA program would fulfill a range of criteria: Stable for the conditions under which they will be transported and stored Homogeneous across all the aliquots produced Have analyte concentrations that include the expected clinical range Include appropriate sample types (e.g., urine, whole blood, serum) Available in sufficient volume Inexpensive enough for cost not to be an impediment Behave in clinical laboratory measurement procedures in the same manner as patient samples Samples from a single donor or pooled samples from multiple donors can be used Samples used for PT/EQA

7. Single donor Vs Pooled Samples Single-donor samples: Interfering substance may be present that influences 1 or more of the measurement procedures. Pooled samples: Dilute an interfering substance Limited by interactions of components such as serum proteins or urine complexes from different donors may cause aggregation or precipitation that necessitates further processing and potential modification of the matrix

8. Samples have traceable reference values (when reference values are used) Behave like patient samples (commutability) The laboratory may also consider cost Similarity of PT samples to patient samples Method compatibility with peer groups Size of peer groups Frequency of challenges Timeliness and usefulness of reports Educational content Customer service Selection of PT Programs

9. Categories of PT/ EQA Schemes

10. category 1 programs are limited because of: • Technical aspects such as a lack of reference measurement procedures, absence of certified reference materials, inability to prepare commutable samples; • Practical considerations such as the difficulty of preparing samples covering the full measuring interval and the complicated logistics of preparation and distribution of fresh or frozen samples; • Psychological limitations such as lack of awareness of the quality factors important in PT/EQA or unwillingness to adopt these; • Economic concerns because distributing commutable samples in sufficient quantity and providing target values with reference measurement procedures is expensive

11. PT samples should be tested in the same manner as patient samples, to the extent possible Some laboratories may improperly test PT samples differently from patient samples, by repeat testing of PT samples when patient samples are tested only once, or by having a specific analyst test PT samples rather than rotating PT testing among all the personnel who perform patient testing. There should be no attempt to produce “best” results by replicate analysis or testing immediately following internal QC or recalibration No Special Treatment

12. The core content of the result report should resemble as closely as possible the content of a routine clinical result report If the usual report is deemed inappropriate for a PT report, it may equally be inappropriate for a clinical report A copy of all PT reports should be retained within the laboratory in order to verify the information handling by the PT provider. No Special Treatment (cont.)

13. Sending a set of samples from an organizing body to a group of participating laboratories for measurement of 1 or more analytes present in the samples Samples are intended to simulate the clinical samples usually measured Laboratories are not informed of the analyte concentration or activity in a particular sample Timely schedules for running and reporting results are included Laboratory perform measurements in the same manner as for patient samples Results for the samples are returned to the PT/EQA organizer for evaluation of conformance to the expected results PT Procedure

14. The organizer prepares a report that includes: the results reported by a laboratory the method used for the measurements the target values expected for each analyte evaluation of whether the individual laboratory’s results met the performance requirements Reports may also include evaluation of the performance of the various measurement procedures used by the participants The laboratory evaluates its performance according to the provider report PT Procedure (cont.)

15. Limits or quality standards around the target value are established against which performance can be assessed by: Regulatory: wider like US CLIA, German Rili-BAeK Statistical: ± 2-3 SD Clinically-based: on a difference that may affect clinical decisions or on biological variation Total error limits including bias, imprecision, and analytical nonspecificity can contribute to the variation in a single result Have different limits to separately assess bias and imprecision when replicate samples are included PT/EQA limits are set as a minimum standard to identify results that indicate poor performance. Thus, meeting these standards may not indicate that performance is optimal nor that performance meets all clinical needs ACCEPTANCE CRITERIA FOR PT/EQA RESULTS

16. Determines the accuracy by comparing PT/EQA results to those from a reference measurement procedure or from a designated comparison method or to an all-participant (or all-method) mean/ median. This arrangement is now referred to as accuracy-based evaluation Assess agreement with other measurement procedures and imprecision among all methods as well as within a method group Peer group evaluation provides valuable information to assess quality, verifying that a laboratory is using a measurement procedure in conformance to the manufacturer’s specifications and to other laboratories using the same technology Participant Evaluation of Their Own PT/EQA Results

17. PT/EQA result represents 1 point in time and will occasionally be a random error Repeat the measurement using a stored aliquot of the PT/EQA sample (assuming the measurand was stable on storage) to confirm if the problem has persisted or to conclude that the problem no longer exists and the original unacceptable result was a random event, and therefore no corrective action is indicated. If the repeated result is still unacceptable, the laboratory conducts further investigation to identify the root cause, and then initiates corrective action Handling of Unacceptable PT result

18. • Gather data related to the testing event to include records of calibration, reagent use, QC results, and maintenance procedures; • Obtain other data on assay performance, e.g., previous PT/EQA results and relevant patient data; • Identify the root cause of the error; • Take corrective action and preventive action if indicated; • Monitor the success of the corrective action; • Document the investigation and the corrective action. Handling of Unacceptable PT Result (cont)

19. • Was the testing material received in satisfactory condition? • Was the appropriate sample tested? • Were procedures for sample preparation followed? • Was the appropriate method used for analysis? • Was the method performed according to documented procedures? • Were appropriate reagents and controls used? • Was equipment operated according to documented procedures? Gathering and Reviewing Data

20. • Was equipment appropriately maintained? • Was QC acceptable at the time of testing PT samples? • Were results interpreted appropriately? • Has this problem occurred previously with PT samples? Are data consistent with previous PT distributions? Is there a trend leading to failure or is the current set completely unexpected? • Did repeat testing on the properly stored residual sample produce similar results? • Were patient results acceptable at the time of PT testing? Gathering and Reviewing Data (cont.)

21. Clerical error; Methodologic problem; Equipment problem; Technical problem; Problem with proficiency testing materials; Problem with evaluation of results; and No explanation after investigation. Classifying the Problem

22. • result was not correctly transcribed from the instrument tape or read-out to the report form; • the PT sample was mislabeled; • an incorrect instrument or method was reported on the form; • incorrect units were reported; • a decimal point was misplaced; or • an incorrect reporting code was selected on the report form. 1. Clerical Errors

23. • no written procedure (method) for staff to follow; • procedure steps inadequately, incompletely, or incorrectly described; • problem in manufacture of reagents or reference materials; • imprecision due to result being close to detection limit of method; • imprecision due to variation between reagent lots; • incorrect assignment of calibrator value; • calibration is unstable; 2. Methodologic Problems

24. • inadequate quality control method; e.g., − QC material not relevant to analyte concentration, or − inappropriate QC rules or limits; • result not within measuring range (linearity) for instrument or reagent system; • method is biased; • method lacks sensitivity; • method lacks specificity; • carry-over from previous sample(s); • inappropriate incubation conditions (time, temperature, and/or atmosphere); • method used without validation; • inappropriate reference intervals applied 2. Methodologic Problems (cont.)

25. • obstruction of instrument tubing/orifice by clot or protein; • misalignment of instrument probes; • problem with instrument data processing functions; • problem in manufacture of reagents or reference materials; • problem with instrument settings specified by manufacturer; • automatic pipettor not calibrated to acceptable precision and accuracy; • equipment malfunction; • errors or omissions in programming of equipment software applications; and • scheduled instrument maintenance not performed appropriately. 3. Equipment Problems

26. • failure to follow recommended instrument function checks (e.g., temperatures, blank readings, pressures); • incorrect reconstitution or storage of reference materials or reagents; • improper reconstitution, preparation, or storage of PT material; • delay in testing after reconstitution of PT material, causing evaporation or deterioration; • failure to follow written procedures; 4. Technical Problems

27. • failure to follow PT instructions; • failure to act on QC results that indicate a method problem; • pipetting or dilution error; • calculation error; • contamination of PT sample during processing 4. Technical Problems (cont.)

28. • differences between PT samples and patient samples; • sample deteriorated in transit (if time- or temperature-sensitive) − bacterial contamination − hemolysis − nonhomogeneous sample; • sample contains interfering factors (which may be method-specific). 5. Problem With Proficiency Testing Materials

29. • inappropriate peer group; • inappropriate target value from nonhomogeneous testing material or lingering [“masked”] outliers • inappropriate evaluation interval with an extremely precise method, the acceptable range may be much narrower than needed for clinical usefulness • incorrect data entry by PT provider. 6. Problem With Proficiency Testing Evaluation

30. An investigation fails to reveal an explanation for an unacceptable PT result 19 to 24% of the time 7. No Explanation After Investigation

31. Investigation Form for Discordant Proficiency Testing Findings in Clinical Chemistry

32. Sample Form for Documenting Unacceptable PT Investigations

33. Sample Form for Documenting Unacceptable PT Investigations (cont.)

34. Split-Sample With Another Laboratory Internal Split-Sample Procedures Audit-Sample Procedure Analysis of Manufacturer’s Product Calibrator or Trueness Control Material Analysis of Interlaboratory Quality Control Data Averages of Patient Data Assessment of Laboratory Tests When Proficiency Testing is Not Available

35. Reference Intervals Reevaluation of Interpreted Results Direct Observation of Technique-Dependent Tests Clinical Correlation Studies Government and University Interlaboratory Comparison Programs Analysis of Data From Qualitative Alternative Assessment Procedures Assessment of Laboratory Tests When Proficiency Testing is Not Available (cont.)

36. YOUR VALUABLE QUESTIONS THANK YOU