Defect removal effectiveness

Anna University, Final CSE, Software Quality Management

1. DefectRemovalEffectiveness Defect
 Removal
 Effectiveness
SoftwareQualityManagement Software
 Quality
 Management
 
Unit 3
GRAAld G
. 
R
oy 
A
ntony 
A
rno
ld
Asst. Professor / CSE
GRAA

2. f
l
i
f
h
i
•De
f
ectremova
l
isoneo
f
t
h
etopexpenses
in
any software project and itgreatly affects
schedules.
•Effective defect removal can lead to
reductionsinthedevelopmentcycletimeand g
ood
p
roduct
q
ualit
y
.
g
p
qy
•It is important for all development
organizations
to
measure
the
effectiveness
of
organizations
to
measure
the
effectiveness
of
theirdefectremovalprocesses.
GRAA

3. •Fagan(1976)defined error detection efficiency as:
Errors found by an Inspection
Total errors in the product before inspection
X 100%
•Jones'sdefinition (1986), stated here, is very similar to Fagan's:
–Removal Efficiency = 
Defects found b
y
 removal  o
p
eration
yp
Defects present at removal operation
X 100%
Defects found
Defects found + Defects not found (found later)
X 100%
•IBM Houston received the first NASA Excellence Award for Quality 
and Productivityin 1987
GRAA

4. •
OneofthefourmetricsIBMusedtomanagequalityis One
 of
 the
 four
 metrics
 IBM
 used
 to
 manage
 quality
 is
 
the early detection percentage, which is actually 
inspection defect removal effectiveness
Numberofmajorinspectionerrors
Early Detection Percentage =
Number
 of
 major
 inspection
 errors
Total number of errors
X 100%
•where total number of errors is the sum of major 
inspection errors and valid discrepancy reports 
(ditithhiftkitt (di
screpancy repor
t
 is 
th
e mec
h
an
ism 
f
or 
t
rac
ki
ng 
t
es
t
 
defects).
GRAA

5. GRAA

6. •
TheeffectivenessmeasurebyDunn(1987)differslittlefrom The
 effectiveness
 measure
 by
 Dunn
 (1987)
 differs
 little
 from
 
Fagan's and from Jones's second definition. 
•Dunn‘s definition is:
Numberofdefectsfoundbyactivity
Effectiveness of activity (development phase ) =
Number
 of
 defects
 found
 by
 activity
Number of defects found by subsequent activities
X 100%
•This metric can be tuned by selecting only defects present at 
the time of the activit
y
 and susce
p
tible to detection b
y
 the 
y
py
activity.
GRAA

7. •Daskalantonakis(1992) describes the metrics used at Motorola for 
softwaredevelopment software
 development
.
Numberofpre
‐
releasedefects
Total Defect Containment Effectiveness (TDCE) =
Number
 of
 pre
‐
release
 defects
Number of pre‐release defects + Number of post‐release defects
Number of Phase ierrors
Phase Containment Effectiveness (PCE
i) =
•
Wherephase
i
errorsare
problemsfoundduringthatdevelopment
Number of Phase ierrors + Number of phase idefects
Where
 phase
 i
errors
 are
 problems
 found
 during
 that
 development
 
phase in which they were introduced, and 
•Phase idefects are problems found later than the development 
phase
inwhichtheywereintroduced.
phase
 in
 which
 they
 were
 introduced.
GRAA

8. ()
Defects removed 
(
at the step
)
Defects existing on step entry + Defects injected during development of the step
X 100%
GRAA

9. GRAA

10. Bd
il
td
ii d
b
th
Dtt
•
B
ase
d
onaspec
ia
l
s
t
u
d
ycomm
iss
ione
d
b
y
th
e
D
epar
t
men
t
of Defence, Jones estimates the defect removal
effectiveness for organizations at different levels of the
()
developmentprocesscapabilitymaturitymodel
(
CMM
)
:
–Level1:85% –
Level
2
:
89
%
Level
2
:
89
%
–Level3:91%
–Level4:93%
l
–
Leve
l
5:95%
•These values can be used as comparison baselines for
organizations
to
evaluate
their
relative
capability
with
organizations
to
evaluate
their
relative
capability
with
regardtothisimportantparameter.
GRAA

11. •Basedonhistoricalandrecentdatafromthree
software engineering organizations at General
Dynamics Decision Systems, Diaz and King (2002) reportthatthephasecontainmenteffectivenessby CMMlevelasfollows:
–Level2:25.5% –Level3:41.5%
–Level4:62.3%
–Level5:87.3%
GRAA

12. PhaseInserted
Cumulative%ofDefects
Phase
 Inserted
Cumulative
 %
 of
 Defects
 
removed throughAcceptance 
Test
Requirements 94% Top
‐
LevelDesign
95%
Top
Level
 Design
 
95%
DetailedDesign 96% CodeandUnit
Test
94%
Code
 and
  
Unit
Test
94%
Integration Test 75% SystemTest
70%
System
 Test
70%
Acceptance Test 70%
GRAA