Laboratory Method Verification, March 2017

This presentation discusses in a comprehensive way Lab method verification and all related concepts and measures

1. Laboratory Method Verification
Ola H. Elgaddar
MD, PhD, MBA, CPHQ, LSSGB,
Lecturer of Chemical Pathology
Alexandria University
[email protected]

2. Quality
Doing the right thing right, from the
first time and every time!

3. Accreditation
An institution or a program meets
standards of quality set forth by an
accrediting agency

5. Validation / Verification??
Method validation:(Manufacturer concern)
Establishingtheperformanceofanew
diagnostictool.
Confirmation,throughtheprovisionof
objectiveevidence,thattherequirementsfor
aspecificintendeduseorapplicationhave
beenfulfilled’(doingcorrecttest)......
ISO9001:2005

6. Validation / Verification??
Method verification: (Lab / user concern)
Aprocesstodetermineperformance
characteristicsbeforeatestsystemisutilized
forpatienttesting.
Confirmation,throughtheprovisionof
objectiveevidence,thatspecified
requirementshavebeenfulfilled’(doingtest
correctly)……ISO9001:2005

8. According to Westgard
ØTheinner,hidden,deeper,secretmeaning
ofmethodvalidationiserrorassessment.
ØHowmucherrormightbepresentinthe
testresultwithinyourlaboratory?
ØCouldthisdegreeoferroraffectthe
interpretationandpossiblypatientcare?
Ifthepotentialerrorislargeenoughtoleadto
misinterpretation,thenthemethodisnot
acceptable.

10. ØRandomerror,RE,orimprecisionis
describedasanerrorthatcanbeeither
positiveornegative,whosedirectionand
exactmagnitudecannotbepredicted,where
thedistributionofresultswhenreplicate
measurementsaremadeonasingle
specimen.
ØUsually,duetoerrorinPipetting

12. ØSystematicerror,SE,orinaccuracyisan
errorthatisalwaysinonedirection,
displacingthemeanofthedistributionfrom
itsoriginalvalue.
ØIncontrasttorandomerrors,systematic
errorsareinonedirectionandcauseallthe
testresultstobeeitherhighorlow.
ØEitherconstantorproportionate
ØUsually,duetoerrorincalibration

16. InternalQualityControl(IQC)isused,ondaily
basis,inthedecisiontoacceptorreject
resultsofpatientssamplesandenablesthe
labtodescribeandmonitorthequalityofits
work.
-Usuallyithas2levels(Sometimes3);
representingthe“Normal”andthe
“Pathological”analytelevel.
-JudgedaccordingtoWestgardMulti-QC
rules

17. ExternalQualityControl(EQC)=
Proficiencytestisused,onmonthlyorBi-
weekly(Orothers)basis,wherelabsfromall
overtheworldjointheprogramandsendtheir
usedMethod/Analyzer.
-Astatisticalcomparisonismadeandeach
labresultiscomparedtotheresultofitspeer
groupineachanalyte.

19. The following items need verification
ØAnalyticalSpecificity:Interferencestudies
ØAnalytical Sensitivity: Calibration curve
Detection limit
ØReportablerange:Linearityexperiment
ØPrecision:Replicationstudy
ØAccuracy:Bias/Recoverystudy
ØReferenceIntervals

21. Analytical Specificity
The ability of an analytical method to detect
“ONLY” the analyte of interest.
Freedom from interference by any element
or compound other than the analyte of
interest

23. Analytical Specificity
-Analytical Specificity is verified using
interference studies.
-Avalidatedmethod,knowntobefreeof
theinterferingsubstanceisused.Aseries
ofsamples containingincreased
concentrationsoftheinterferingsubstance
areanalyzedusingthatmethod,andthe
methodunderstudy,thenbothresultsare
compared

25. Some Automated systems have a “HIL” index

27. Analytical Sensitivity
-Theabilityofananalyticalmethodto
detectalowconcentrationofagiven
substanceinabiologicalsample.Thelower
thedetectableconcentration,thegreater
theanalyticalsensitivity.
-Detectionlimitsstudies

28. Analytical Sensitivity
OR,
-Theabilityofananalyticalmethodto
detect(respondto)achangein
concentrationoftheanalyte.Thesmaller
thedetectablechange(thechangein
concentrationthatcanresultinadefinite
changeinthereportedsignal),thegreater
istheanalyticalsensitivity.
-SlopeoftheCalibrationCurve

30. REMEMBER!!

31. STANDARD
•Itisasolutionofknownconcentration.
•Formedbydissolvingaknownamountof
ananalyteinaspecificvolumeofan
aqueoussolvent.
•Ex:Dissolving100mgglucosein100ml
D.Wgivesastandardofa100mg/dl
concentration

32. CALIBRATOR
•A solution or a device of known
quantitative or qualitative characteristics
(eg: concentration, activity, intensity)
•Used to calibrate or adjust a measurement
procedure.
•Matrix is preserved.

33. •Thecalibratormaterialisreconstituted&
introducedtotheanalyzerbeforeusinga
newlotofreagent,anditsconcentrationis
assigned.
•Thecalibratoristreatedlikesamplesand
theabsorbanceofthedevelopedcoloris
determined.

34. Using the provided concentration, the analyzer
constructs a calibration curve
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
0 50 100 150 200 250

35. •Ifthegivenconcentrationis100mg/dl,
theabsorbanceofthispointisdetermined.
•Byprovidingtheanalyzerwiththerange
wherethismethodislinear,thecalibration
curveisextendedsothatanysample
concentrationlyingwithinthatlinearity
rangecanbededucedfromthiscurve

36. Detection Limits
1.Lower Limit of Detection (LLD)= Limit of
the blank (LOB)
2.Biologic Limit of Detection (BLD)= Limit of
Detection (LOD)
3.Functional Sensitivity (FS)= Limit of
Quantification (LOQ)

38. Detection Limits
1.Lower Limit of Detection (LLD)= Limit
of the blank (LOB)
Mostanalyticalinstrumentsproduceasignaleven
whenablank(reagentwithoutanalyte)is
analyzed.Thissignalisreferredtoasthenoise
level.TheLLDistheanalyteconcentrationthatis
requiredtoproduceasignalgreaterthantwo(or
three)timesthestandarddeviationofthenoise
level.

39. Detection Limits
1.Lower Limit of Detection (LLD)= Limit
of the blank (LOB)
LimitofDetection(LLD)isestimatedas,themean
oftheblanksampleplus2or3timestheSD
obtainedontheblanksample:
LLD=mean(blk)+Zs(blk)
wheretheZ-valueisusually2or3

40. Detection Limits
2.Biologic Limit of Detection (BLD)= Limit
of Detection (LOD)
-Itisthelimitofblank,aftertheadditionofthe
analyteofinterest(Itslowestconcentration)
-BLDisestimatedastheLLDplus2or3timesthe
standarddeviationobtainedfromthe"spiked"
sample
BLD=LLD+Zs(spk)
wheretheZ-valueisusually2or3
-ResultsbetweenBLD&LLDshouldbereported
withoutquantitation

41. Detection Limits
3.Functional Sensitivity (FS)= Limit of
Quantification (LOQ)
-Thelowestconcentrationoftargetcompounds
thatcanbequantifiedconfidently,thatmeetssome
pre-specifiedtargetsofimprecision,commonly
CV=20%
-Severalspikedconcentrationsmustbestudiedto
determinetheprecisionprofileatthelow
concentrationrangeandtoselectthelowest
concentrationatwhicha20%CVisobtained.

43. To Sum Up!
1.LoBisthehighestapparentanalyte
concentrationexpectedtobefoundwhen
replicatesofablanksamplecontainingno
analytearetested.
2.LoDisthelowestanalyteconcentrationlikelyto
bereliablydistinguishedfromtheLoBandat
whichdetectionisfeasible.
3.LoQisthelowestconcentrationatwhichthe
analytecannotonlybereliablydetected,butat
whichsomepredefinedgoalsaremet.TheLoQ
maybeequivalenttotheLoDoritcouldbeat
amuchhigherconcentration.

45. Reportable Range
=
Analytical Measurement Range
It is the range of numeric results a method
can produce without any special specimen
pre-treatment, such as dilution.

46. ØItshouldbeverified,forthemanufacturer’s
claim,usingthelinearityexperiment.
ØItisperformedusingeithercalibrators,
proficiencysamples,orsamples
ØSerialdilutionswillbemadecoveringthe
wholeanalyticalmeasurementrange,and
reachingascloseaspossibletotheclaimed
valuesofthemanufacturer.
ØEachdilutionistobeprocessedinduplicate
toremovetheelementofimprecision.

47. ØTheobservedmeasures(ontheX-axis)are
plottedagainsttheexpectedmeasures(on
theY-axis),andalinepointtopointgraphis
constructedforeachanalyte.
ØThelineisjudgedvisuallyforitslinearity
andaccordingtoeachexperiment,the
analyticalmeasurementrangeofeach
methodisverified,whereanypatientresult
obtainedinthefuture,outsidetheverified
range,cannotbereleasedwithoutfurther
processing(Dilutionorconcentration).

48. Dilution Expected conc Observed conc
D1 8.27 8.27
D2 4.56 4.7
D4 2.5 2.6
D6 1.5 1.52
D7 1 1.07
D8 0.5 0.68
Correlation: r 0.99988684
Analytical range: fT4 : 0.1 -12 ng/ dl

49. 0
1
2
3
4
5
6
7
8
9
0 2 4 6 8 10
Observed measure
Expected measure

53. Precision
Closeness of agreement between quantity
values obtained by replicate measurements
of a quantity, under specified conditions.

55. Precision
ØPrecisionshouldbeassessedusing
qualitycontrolmaterial(Aminimumof2
levels),orpooledserum(Ofminimumtwo
concentrationlevels)
ØEachleveloftheQCmaterial/pooled
serumismeasured5timesperday(Within
run),for5days(inbetweenruns)

56. Precision
ØThemeasuresobtainedfromthisprecision
studyaretobecollected,andthemean,SD,
andCVarecalculatedforeachparameter,
fortheusedQClevels/pooledserum.
ØTheobtainedCVs,arestatistically
comparedtothemanufacturer’sclaimusing
ANOVAtestofsignificance,todetermineif
thereisasignificantdifferentbetweenthe
obtainedCVs(andtheirverification
intervals),andthemanufacturer’sclaimata
certainCI;usually95%

58. Accuracy
Closenessoftheagreementbetweenthe
resultofameasurementandatruevalueof
themeasurand.

59. Trueness
Closenessoftheagreementbetweenthe
replicatesofresultofameasurement,and
atruevalueofthemeasurand.

60. ØThedifferencebetweenthemeanof
replicatesofameasurement,anditstrue
valueistheBIAS.
ØTheCLSIcalculationofbiasisbasedon
theresultsof7–11PTsamples;eachis
measuredinduplicate,andthencompared
tothetruevalue(Peer’smean)using
studentT-test.

62. A concept!!!

63. A concept!!!

64. A concept!!!

65. Why is Z-score used in PT results??

66. Biasisverifiedforthetestedmethodwhen
thereisnosignificantdifferencebetween:
qMeanZ-scoreofPTresults(7–11)isnot
significantlydifferentfromZero
qMeanZ-scoreofPTresults(7–11)isnot
significantlydifferentfrompeer’smean

68. Reference intervals
ØRememberthatwearejust“Verifying”the
referenceintervalsstatedbythe
manufacturerorpublishedintheliterature,
and“transferring”themtothelabusingthe
methodunderstudy.
Ø“Establishment”ofreferenceintervalsis
anotherissue.

71. Reference intervals
ØAcceptabilityofthetransfershallbe
assessedbyexamining20reference
individuals,fromoursubjectpopulation,and
comparingtheobtainedtestresultstothose
ofthemanufacturer/Literature.
ØThose20individualsshouldbeselectedin
suchawaythatwillsatisfytheexclusionand
partitioningcriteria.
ØThetestresultsshouldbeexaminedto
makesurethatnoneoftheresultsappears
tobeanoutlier.

72. Reference intervals
ØThemanufacturer's/Literaturereference
intervalsareconsideredverifiedifnomore
thantwoofthe20testedsubjects'values(or
10%ofthetestresults)falloutsidethose
ranges.

73. Reference intervals
ØExclusion/partitioningcriteriainclude:
age,sex,fastingstatus,diseasehistory,
drughistory,previoussurgeries,andtimeof
thecycle/pregnancyforfemales.

76. Method Comparison
ØAccordingtoCLSI,atleast40samples
shouldbeassayedonbothmethodsunder
examination(twofieldmethods),orbetween
onetestedmethodandareferencemethod.
ØSeveralstatisticalapproachescanbe
used,oneofthemistocalculatethe
correlationcoefficient“r”
Ø“r”shouldbemorethanorequal0.95

78. Total Error
ItisthesummationofbothRandomand
Systematicerror.
Itiscalculatedasfollow:
TE=Bias(%)+2CV
ItiscomparedtoBiologicalVariation(orany
otherspecifications)forTotalAllowable
Error

80. Uncertainty
ØItisanintervalaroundareported
laboratoryresultthatspecifiesthelocationof
thetruevaluewithagivenprobability
ØIttakesintoconsiderationboththe
imprecision(SD),andtheinaccuracy(Bias)
ØItiscalculatedfromthedataof6month
minimum

81. What performance characteristics
are usually validated?
ØReportablerange(Linearity)
ØPrecision(orimprecision)
ØAccuracy(orinaccuracy,bias)
ØReferenceinterval