software-testing-second-edition-pages.pdf
ShivakumarM3
10 views
40 slides
Aug 10, 2024
Slide 1 of 40
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
About This Presentation
software testing vtu syallabus 2016-17
Slide Content
204 Software Tsing: À Cafimanÿ Approach, Second Editon
system-level threads. To end on a pun, the definitions of the next two chapters
‘il te these threads together
References
Atrest, NEW, New Paradigms for Software Devciopment, IEEE. Computer Socety Press
‘Washington, D.C. 1986.
Bochum, BW, A spiral model for sofware development and enhancement, AE Computer,
Vol. 21, No. 6, FRE Computer Saciey Press, Washington, D.C. May 1988, pp. 61-72.
Toppe, Andrew et al, Send Metodo: Merging Model, Techniques, and CASE.
MeGrav-Hil, New York, 1993
Chapter 13
en
Integration Testing
In September 1999, the Mars Climate Orbiter mission failed after successfully
traveling 416 million miles in 41 weeks, Ic disappeared just as it was to begin
orbiting Mars. The faut should have been revealed hy integration testing: Lockheed
‘Mania Astronautics used acceleration data in English units (pounds), while the
Jet Propulsion Laboratory did its calulations with metric units (newton), NASA
ansounced 4 $50,000 project to discover how this could have happened (Fordahl,
1999). They should have read this chapter
Crafispersons are recognized by two essential characteristics; they have a deep.
knowledge of the tools oftheir rade, and they have a similar knowledge of the
medium in which they work so that they understand thei tools in terms of how
they work with the medium. in Parts I and ill, we focused on the tools
(techniques) available to the testing craftsperson. Our goal there was 10 understand
testing techniques in terms of their advantages and imitations with tespect 10
particular types of software. Here, we shift our emphasis 10 the medium, with
the goal of improving the testing cralsperson's judgment through a better under-
standing of the medium. We make a deliberate separation here: this chapter and
the next address testing for software that has been defined, designed, and
developed with the traditional models for function, data, control, and structure.
Testing for objectoriented software is deferred to Part V, We continue our
refinement of the simple automated teller machine (SATA) system in this chapter
and use it to ¡lustre three distinct approaches to integration testing. For each
approach, we begin wih is basis and then discuss various techniques that use
the base information. To continue the crafisperson metaphor, we emphasize the
advantages and limitations of each integration testing technique.
13.1 A Closer Look at the SATM System
In Chapter 12, we described the SATM system in terms ofits output screens (Figure
126), the actual terminal (Figure 12:7), its contest and partal data Now diagrams
205
system-level threads. To end on a pun, the definitions of the next two chapters
‘il te these threads together
References
Atrest, NEW, New Paradigms for Software Devciopment, IEEE. Computer Socety Press
‘Washington, D.C. 1986.
Bochum, BW, A spiral model for sofware development and enhancement, AE Computer,
Vol. 21, No. 6, FRE Computer Saciey Press, Washington, D.C. May 1988, pp. 61-72.
Toppe, Andrew et al, Send Metodo: Merging Model, Techniques, and CASE.
MeGrav-Hil, New York, 1993
Chapter 13
en
Integration Testing
In September 1999, the Mars Climate Orbiter mission failed after successfully
traveling 416 million miles in 41 weeks, Ic disappeared just as it was to begin
orbiting Mars. The faut should have been revealed hy integration testing: Lockheed
‘Mania Astronautics used acceleration data in English units (pounds), while the
Jet Propulsion Laboratory did its calulations with metric units (newton), NASA
ansounced 4 $50,000 project to discover how this could have happened (Fordahl,
1999). They should have read this chapter
Crafispersons are recognized by two essential characteristics; they have a deep.
knowledge of the tools oftheir rade, and they have a similar knowledge of the
medium in which they work so that they understand thei tools in terms of how
they work with the medium. in Parts I and ill, we focused on the tools
(techniques) available to the testing craftsperson. Our goal there was 10 understand
testing techniques in terms of their advantages and imitations with tespect 10
particular types of software. Here, we shift our emphasis 10 the medium, with
the goal of improving the testing cralsperson's judgment through a better under-
standing of the medium. We make a deliberate separation here: this chapter and
the next address testing for software that has been defined, designed, and
developed with the traditional models for function, data, control, and structure.
Testing for objectoriented software is deferred to Part V, We continue our
refinement of the simple automated teller machine (SATA) system in this chapter
and use it to ¡lustre three distinct approaches to integration testing. For each
approach, we begin wih is basis and then discuss various techniques that use
the base information. To continue the crafisperson metaphor, we emphasize the
advantages and limitations of each integration testing technique.
13.1 A Closer Look at the SATM System
In Chapter 12, we described the SATM system in terms ofits output screens (Figure
126), the actual terminal (Figure 12:7), its contest and partal data Now diagrams
205
206 Software Tsing: A Crafsmans Approach, Second Edition
Figures 128 and 12.9), an entiy/eationship model ofits data (Figure 32.10), inte
state machines describing some ofits behavior Figures 12.11 and 12.12), and à
paral funcional decomposition (Figure 1233). We also developed a pseudocode:
description of the main program and two units, VlitePIN and Ged?
‘We begin here by expanding the functional decomposition hat was sated in
Figure 12.12, the numbering scheme preserves the levels of the components in
that figure, For easier reference, each component that appears in our analysis is
given a new Ghoner) number, these numbers are given in Table 131. Che only
reason for this isto make the figures more readable.)
Table 13.1_ SATM Units and Abbreviated Names
Unt Level
Number Number Uni Name
SATA System
Device Sense & Control
Door Sense & Control
Get Door Status
Control Door
Dispense Cash
Slot Sense & Control
WatchCardslot
Get Deposit Slot Status
Control Card Roller
Control Envelope RTE —
Read Card Strip
Central Bank Comm,
Get PIN for PAN,
Get Account Status
Post Daily Transactions
Terminal Sense & Control
Screen Driver
Key Sensor
Manage Session
Validate Card
Validate FIN
GeiPIN
Close Session
New Transaction Request
Print Receipt
Post Transaction Local
Manage Transaction
Got Transaction Type
Get Account Type
Report Balance
Process Deposit
Process Withdrawal
Integration Testing 207
Figure 13.1. SATM functional decomposition tee.
‘he decomposition in Table 13.1 is pictured as a decomposition tree in Figure
13.1. This decomposition isthe Bass far the usual view of integration testing. It
is imporant to remember that such decomposition is primary a packaging
ppanttion of the system. As software design moves into more detail, the added
information lets us refine the functional decomposition tree into a unit calling
graph The unit calling graph is the directed graph in which nodes are program
Units and eus correspond TO program Gils; that I, unit A calls Unie D, 2
directed edge runs from node A to node B. We began the developinent of the
call graph for the SATM system in Chapter 12 when we examined the calls made
by the main program and the ValidatePIN and GetPIN modules, That information
is captured in the adjacency matrix given in Table 13.2. This matrix was created
wih a spreadsheet; this turns out to be a handy too! for testers.
‘The SATM call graph is shown in Figure 13.2 Some ofthe hierarchy is obscured
(0 reduce the confusion in the drawing. One thing should be quite obvious:
drawings of call yraphs do not scale up well. Both the drawings and the adjacency
matrix provide insights to the tester, Nodes with high degree will be important
10 integration testing, and paths from the main program (node 1) to the sink
nodes can be used to identify contents of builds for an incremental development.
13.2 Decomposition-Based Integration
Most textbook discussions of integration testing only consider integration testing
based on the functional decomposition of the system tested. These approaches
are al based on the functional decomposition, expressed either as a tree (Figure
13.) or in textual form. These discussions inevitably center on the order in which
modules are to be integrated. Four choles are available: from the top of the tree
downward (op down), from the bottom of the tree upward (bottom up), some
combination of these (sandwich), or most graphically, none of these (the big
bang). All these integration orders presume that the units have been separately
Figures 128 and 12.9), an entiy/eationship model ofits data (Figure 32.10), inte
state machines describing some ofits behavior Figures 12.11 and 12.12), and à
paral funcional decomposition (Figure 1233). We also developed a pseudocode:
description of the main program and two units, VlitePIN and Ged?
‘We begin here by expanding the functional decomposition hat was sated in
Figure 12.12, the numbering scheme preserves the levels of the components in
that figure, For easier reference, each component that appears in our analysis is
given a new Ghoner) number, these numbers are given in Table 131. Che only
reason for this isto make the figures more readable.)
Table 13.1_ SATM Units and Abbreviated Names
Unt Level
Number Number Uni Name
SATA System
Device Sense & Control
Door Sense & Control
Get Door Status
Control Door
Dispense Cash
Slot Sense & Control
WatchCardslot
Get Deposit Slot Status
Control Card Roller
Control Envelope RTE —
Read Card Strip
Central Bank Comm,
Get PIN for PAN,
Get Account Status
Post Daily Transactions
Terminal Sense & Control
Screen Driver
Key Sensor
Manage Session
Validate Card
Validate FIN
GeiPIN
Close Session
New Transaction Request
Print Receipt
Post Transaction Local
Manage Transaction
Got Transaction Type
Get Account Type
Report Balance
Process Deposit
Process Withdrawal
Integration Testing 207
Figure 13.1. SATM functional decomposition tee.
‘he decomposition in Table 13.1 is pictured as a decomposition tree in Figure
13.1. This decomposition isthe Bass far the usual view of integration testing. It
is imporant to remember that such decomposition is primary a packaging
ppanttion of the system. As software design moves into more detail, the added
information lets us refine the functional decomposition tree into a unit calling
graph The unit calling graph is the directed graph in which nodes are program
Units and eus correspond TO program Gils; that I, unit A calls Unie D, 2
directed edge runs from node A to node B. We began the developinent of the
call graph for the SATM system in Chapter 12 when we examined the calls made
by the main program and the ValidatePIN and GetPIN modules, That information
is captured in the adjacency matrix given in Table 13.2. This matrix was created
wih a spreadsheet; this turns out to be a handy too! for testers.
‘The SATM call graph is shown in Figure 13.2 Some ofthe hierarchy is obscured
(0 reduce the confusion in the drawing. One thing should be quite obvious:
drawings of call yraphs do not scale up well. Both the drawings and the adjacency
matrix provide insights to the tester, Nodes with high degree will be important
10 integration testing, and paths from the main program (node 1) to the sink
nodes can be used to identify contents of builds for an incremental development.
13.2 Decomposition-Based Integration
Most textbook discussions of integration testing only consider integration testing
based on the functional decomposition of the system tested. These approaches
are al based on the functional decomposition, expressed either as a tree (Figure
13.) or in textual form. These discussions inevitably center on the order in which
modules are to be integrated. Four choles are available: from the top of the tree
downward (op down), from the bottom of the tree upward (bottom up), some
combination of these (sandwich), or most graphically, none of these (the big
bang). All these integration orders presume that the units have been separately
Table 13.2_Adjacency Matrix for the SATM Call Graph
208 Sabre Tsing: A Cuma’ Approach Second Ei regain Testing 209
Figure 152. SAT call graph
i teste, thus, the goal of decomposition based integration is 10 tet the interfaces
ex among separuely tested une.
ne Ne can dispense with the big bang approach most cas: in this view of
se ES integran, al the uni are compiled together and tested at once. The drawback
to this is that when (not 18) a flue is observed, few cues are available 10 help
x Pop Isolate the location(s) of the ful. (Recall the distinction we made in Chapter 1
a Ki between faults and failures.)
5 x 13.2.1 Top-down Integration
7 x Top-down integration begins with the main program (he root of he tee), Any
lower level uni that is called by the main program appears as a “sub,” where
2 x ‘subs are pieces of Uhrowsaway code that emule a called un. 1 we performed
= top down integration testing forthe SATM syste, the fst step would be (0
E = develop stubs for all the units called by the main program — WtchCarüsit,
. xx CConiol Gard Rois, Sereen Driver, Validate Car Validate PIN, Manage Transaction
x and New Transaction Request, Generally esters have to develop the stubs, and
some imagination is required. Here are two examples of stubs:
ls Tesco GAIN ANAM, EspecarIn STUB
IEPAN = 1123 Mon PIN = 2876
= x PAN 1236 Then PIN 975.
WPAN EE Then PIN =D
= xx Ba
à ae Bruce Key Sea) STUB
dat: KoyStokes STACK OP 5.8
Kor POP (Reyes)
2
B
“
15
16
ÉCRÉDREREE
m
7
208 Sabre Tsing: A Cuma’ Approach Second Ei regain Testing 209
Figure 152. SAT call graph
i teste, thus, the goal of decomposition based integration is 10 tet the interfaces
ex among separuely tested une.
ne Ne can dispense with the big bang approach most cas: in this view of
se ES integran, al the uni are compiled together and tested at once. The drawback
to this is that when (not 18) a flue is observed, few cues are available 10 help
x Pop Isolate the location(s) of the ful. (Recall the distinction we made in Chapter 1
a Ki between faults and failures.)
5 x 13.2.1 Top-down Integration
7 x Top-down integration begins with the main program (he root of he tee), Any
lower level uni that is called by the main program appears as a “sub,” where
2 x ‘subs are pieces of Uhrowsaway code that emule a called un. 1 we performed
= top down integration testing forthe SATM syste, the fst step would be (0
E = develop stubs for all the units called by the main program — WtchCarüsit,
. xx CConiol Gard Rois, Sereen Driver, Validate Car Validate PIN, Manage Transaction
x and New Transaction Request, Generally esters have to develop the stubs, and
some imagination is required. Here are two examples of stubs:
ls Tesco GAIN ANAM, EspecarIn STUB
IEPAN = 1123 Mon PIN = 2876
= x PAN 1236 Then PIN 975.
WPAN EE Then PIN =D
= xx Ba
à ae Bruce Key Sea) STUB
dat: KoyStokes STACK OP 5.8
Kor POP (Reyes)
2
B
“
15
16
ÉCRÉDREREE
m
7
210 Sohware Testing: A Crafmans Approx, Second ion
In the stub for GetPINfurPAN, the tester replicates a able look-up with just
a few values that will appear in test cases. In the stub for KeySensor, the tester
must devise a sequence of pon events that can occur once each time the
KeySensor procedure is called. (Here, we provided the keystrokes to partially
enter the PIN "8876. hut the user hit the cancel button before the fourth digit)
practice, the effort to develop stubs is usually quite significant. There is good
reason to consider stub code as part of the software development and maintain
it under configuration management,
Once all the stubs for SATM main have been provided, we test the main
program as if it were a stand-alone unit. We could apply any of the appropriate
functional and structural techniques and look for faults, When we are convinced
thatthe main program logic is correc, we gradually replace stubs with the actual
code. Figure 133 shows part of the top-down integration testing sequence for
the SATM decomposition in Figure 13.1. At the uppermost level, we would hive
stubs for the four components in the first-level decomposition. There would be
four integration sessions; in each, one component would be actual (previously
unit tested) code, and the other three would be stubs. Top-down integration
follows a breadth-frst traversal of the functional decomposition tree. Two add
tional integration levels are shown in Figure 133.
Figure 13.3. Top-down integration.
Integration Testing au
Figure 134 Bottom-up integration.
Even this can be problematic. Would we replace all the stubs ar once? If we
did, we would have a "small bang’ for units with a high outdegree. If we replace
fone sub at a time, we retest the main program once for each replaced sub,
‘This means thal, for the SATM main proyam example here, we would repeat
Ais integration test eight times (once for each replaced stub, and once with all
the stubs)
13.2.2 Bottom-up Integration
Bottom-up integration is a “mirror image" to the top-down order, with the
difference that stubs are replaced by driver modules that emulate units at the
next level up in the tee. (See Figure 13.4.) In bottom-up integration, we start
‘ith the leaves of the decomposition tee (unis like ControlDoor and Dispense-
Cash), and test them with specially coded drivers. Less throw-away code exists
in drivers than there is in stubs. Recall we had one stub for each child node
in the decomposition tree. Most systems have a fairly high fan-out near the
leaves; so in the bottom-up integration order, we will not have as many drivers
This is panially offset by the fact that the driver modules will be more
complicated,
In the stub for GetPINfurPAN, the tester replicates a able look-up with just
a few values that will appear in test cases. In the stub for KeySensor, the tester
must devise a sequence of pon events that can occur once each time the
KeySensor procedure is called. (Here, we provided the keystrokes to partially
enter the PIN "8876. hut the user hit the cancel button before the fourth digit)
practice, the effort to develop stubs is usually quite significant. There is good
reason to consider stub code as part of the software development and maintain
it under configuration management,
Once all the stubs for SATM main have been provided, we test the main
program as if it were a stand-alone unit. We could apply any of the appropriate
functional and structural techniques and look for faults, When we are convinced
thatthe main program logic is correc, we gradually replace stubs with the actual
code. Figure 133 shows part of the top-down integration testing sequence for
the SATM decomposition in Figure 13.1. At the uppermost level, we would hive
stubs for the four components in the first-level decomposition. There would be
four integration sessions; in each, one component would be actual (previously
unit tested) code, and the other three would be stubs. Top-down integration
follows a breadth-frst traversal of the functional decomposition tree. Two add
tional integration levels are shown in Figure 133.
Figure 13.3. Top-down integration.
Integration Testing au
Figure 134 Bottom-up integration.
Even this can be problematic. Would we replace all the stubs ar once? If we
did, we would have a "small bang’ for units with a high outdegree. If we replace
fone sub at a time, we retest the main program once for each replaced sub,
‘This means thal, for the SATM main proyam example here, we would repeat
Ais integration test eight times (once for each replaced stub, and once with all
the stubs)
13.2.2 Bottom-up Integration
Bottom-up integration is a “mirror image" to the top-down order, with the
difference that stubs are replaced by driver modules that emulate units at the
next level up in the tee. (See Figure 13.4.) In bottom-up integration, we start
‘ith the leaves of the decomposition tee (unis like ControlDoor and Dispense-
Cash), and test them with specially coded drivers. Less throw-away code exists
in drivers than there is in stubs. Recall we had one stub for each child node
in the decomposition tree. Most systems have a fairly high fan-out near the
leaves; so in the bottom-up integration order, we will not have as many drivers
This is panially offset by the fact that the driver modules will be more
complicated,
mm Safıare Testing: A Calsman's Approsch Second Ein
13.2.3 Sandwich Integration
Sandwich integration is a combination of top-down and bottom-up integration. IF
we think about iin terms of the decomposition tree, we are rely only doing
big bang integration on a subarce (see Figure 13,9. There will be less sub and
driver development effor, but this will be offset to some extent by the added.
«ificuly of faut isolation that is a consequence of big bang integration. (We
could probably discuss the size of a sandwich, from dainty finger sandwiches to.
Dagwood-syle sandwiches, but not now.)
13.2.4 Pros and Cons
With the exception of big bang integration, the decompostion-based approaches
ave all intuitively clear, Build with tested components. Whenever a failure is
observed, the most recently added unit is suspected. Integration testing progress
is easily tracked against the decomposition tree, (Ifthe tree Is smal, i is a nice
‘ouch 10 shade in nodes as they are successfully integrated) The top-down and
Dotom-up terms suggest breadth-Firs traversals of the decomposition tree, but
this is not mandatory. (We could use full-height sandwiches to test the tree in a
dept first manner)
‘One of the most frequent objections to functional decomposition and waterfall
development is that both are artificial, and both serve the needs of project
management more than the needs of software developers, This holds tue also
for decompostion-based testing. The whole mechanism is that units are integrated
‘ith respect to structure; this presumes that correc behavior follows from indi
vidually correct units and correct interfaces, (Practitioners know better.) The
development effort for stubs or drivers is another drawback 10 these approaches,
And this is compounded by che retesting effor, Here is formula that computes
the number of integration test sessions for u given decomposition tree (a test
session is one ser of tests for a specific configuration actual code and stubs):
Sessions = nodes leaves + edges
“The SATM system has 42 integration testing sessions, which means 42 separate
sets of Integration test caes,
For top-down integration, (nodes ~ 1) stubs are needed, and for bottom-up
Integration, (nudes ~ leaves) drivers are needed, For the SATM system, this Is 32
stubs and 10 drivers
13.3 Call Graph-Based Integration
One of the drasebacks of decomposition based integration is that the basis isthe
functional decomposition wee. if we use the call graph instead, we mitigate this
deficiency; we also move in the direction of structural testing. We are in a position
10 enjoy the investment we made in the discussion of graph theory. Because the
‘all graph is a directed graph, why not use it the way we used program graphy?
213
Figure 13:5. Sandwich integration.
This leads us to two new approaches to integration testing: we will refer to them
as pair-wise integration and neighborhood integration,
13.3.1 Pair-Wise Integration
‘The idea behind pair-wise integration isto eliminate the stub/éiver development
cffor. Instead of developing, stubs and/or divers, why not use the actual code?
At first, this sounds like big bang integration, but we resirict a session to only a
pair of units in the call graph. The end result i that we have one integration test
session for each edge in the call graph (40 for the SATM call graph in Figure
132). This is not much of a seduction in sessions from either top-down or bottom
up (42 sessions), but i is a drastic reduction in stub/diver development. Four
pair-vise integration sessions are shown in Figure 13.6.
Figure 13.6 Pairwise integration.
13.2.3 Sandwich Integration
Sandwich integration is a combination of top-down and bottom-up integration. IF
we think about iin terms of the decomposition tree, we are rely only doing
big bang integration on a subarce (see Figure 13,9. There will be less sub and
driver development effor, but this will be offset to some extent by the added.
«ificuly of faut isolation that is a consequence of big bang integration. (We
could probably discuss the size of a sandwich, from dainty finger sandwiches to.
Dagwood-syle sandwiches, but not now.)
13.2.4 Pros and Cons
With the exception of big bang integration, the decompostion-based approaches
ave all intuitively clear, Build with tested components. Whenever a failure is
observed, the most recently added unit is suspected. Integration testing progress
is easily tracked against the decomposition tree, (Ifthe tree Is smal, i is a nice
‘ouch 10 shade in nodes as they are successfully integrated) The top-down and
Dotom-up terms suggest breadth-Firs traversals of the decomposition tree, but
this is not mandatory. (We could use full-height sandwiches to test the tree in a
dept first manner)
‘One of the most frequent objections to functional decomposition and waterfall
development is that both are artificial, and both serve the needs of project
management more than the needs of software developers, This holds tue also
for decompostion-based testing. The whole mechanism is that units are integrated
‘ith respect to structure; this presumes that correc behavior follows from indi
vidually correct units and correct interfaces, (Practitioners know better.) The
development effort for stubs or drivers is another drawback 10 these approaches,
And this is compounded by che retesting effor, Here is formula that computes
the number of integration test sessions for u given decomposition tree (a test
session is one ser of tests for a specific configuration actual code and stubs):
Sessions = nodes leaves + edges
“The SATM system has 42 integration testing sessions, which means 42 separate
sets of Integration test caes,
For top-down integration, (nodes ~ 1) stubs are needed, and for bottom-up
Integration, (nudes ~ leaves) drivers are needed, For the SATM system, this Is 32
stubs and 10 drivers
13.3 Call Graph-Based Integration
One of the drasebacks of decomposition based integration is that the basis isthe
functional decomposition wee. if we use the call graph instead, we mitigate this
deficiency; we also move in the direction of structural testing. We are in a position
10 enjoy the investment we made in the discussion of graph theory. Because the
‘all graph is a directed graph, why not use it the way we used program graphy?
213
Figure 13:5. Sandwich integration.
This leads us to two new approaches to integration testing: we will refer to them
as pair-wise integration and neighborhood integration,
13.3.1 Pair-Wise Integration
‘The idea behind pair-wise integration isto eliminate the stub/éiver development
cffor. Instead of developing, stubs and/or divers, why not use the actual code?
At first, this sounds like big bang integration, but we resirict a session to only a
pair of units in the call graph. The end result i that we have one integration test
session for each edge in the call graph (40 for the SATM call graph in Figure
132). This is not much of a seduction in sessions from either top-down or bottom
up (42 sessions), but i is a drastic reduction in stub/diver development. Four
pair-vise integration sessions are shown in Figure 13.6.
Figure 13.6 Pairwise integration.
24 Sofa Testing: A Cratsman’s Approach, Second Eon
ES
Figure 137. Neighborhood integration.
13.3.2 Neighborhood Integration ——
We can let the mathematics cary us sil farther by bowowing the notion of a
ncighborhood from topology. (This is not too much of a stretch — graph theory
is a branch of topology.) The neighborhood of a node in a graph isthe set of
nodes that are one edge away from the given node. In a directed gmph, this
includes all te immediate predecessor nodes und all the immediate successor
nodes (notice that these correspond to the set of stubs and drivers of the node).
The neighborhoods for nodes 16 and 26 are shown in Figure 137. The 11
neighborhoods for the SATM example (based on the call graph in Figure 13.2)
are listed in Table 133,
‘We can always compute the number of neighborhoods for a given call graph, -
Each interior node will have one neighborhood, plus one extra in case leaf modes
are connected directly 1 the root node. (An interior node has a nonzero indegree
and a nonzero outdegree.) We have:
Interior nodes = nodes ~ (source nodes + sink nodes)
Neighborhoods = interior nodes + source nodes
which combine 10:
Neighborhoods nodes ~ sin nodes PA
Neighborhood integration yields a drastic reduction in the number of ine
gration test sessions (down (0 11 from 40), and it avoids stub and driver
Integration Tsing ns
Table 13.3 _SATM Neighborhoods
Mode Predecesor Successor
16 1 sur
7 100mm
15 vo #5
» 1 m3
» 2 M
2 2 us
2 2 4156825
E 2 412340
5 2m os
2 1. 23,24,26,27,25
1 na 7,2, 2, 16,17, 19,22
development The end result i that neighborhoods are essentially the sandwiches
that we slipped past In the previous section. (ti slighay different, because the
base information for neighbothouds is the call graph, not the decomposition
ce.) What they share with sandwich integration is more significan. neighbor:
¡00d integration testing has the faul isolation difficulties of "medium bang”
integration,
13.3.3 Pros and Cons
The call graplebased integration techniques move away from a purely structural
basis toward a behavioral basis, thus, the underlying assumption is an improve
ment. These techniques also eliminate the stub/driver development effort. In
addition to these advantages, call graph-based integration matches well with
developments characterized by builds and composition, For example, sequences
af neigiborhoods can be used to define builds. Alternatively, we could allow
adjacent neighborhoods (0 merge (nto villages?) and provide an orderly, compo-
Sition-hased growth path. Al! this supports the use of neighbochood-hased inte-
ration for systems developed by life cycles in which composition dominates,
The biggest drawback to call graph based integration testing is the faule isolation
problem, especially for large neighborhoods. A more subtle but closely related
problem occurs. What happens if (when) a fault is found in a node (un) that
appears in several neighborhoods? (For example, the sercen driver unit appears
in 7 of the 11 neighborhoods ) Obviously, we resolve the fault but this means
changing the units code in some way, which in um means that all he previously
tested neighborhoods that contain the changed node need to be retested,
Finally, a fundamental uncertainty exists in any structural form of testing: the
presumption that units integrated with respect 0 strucural information will exhibit
correct behavior, We know where we are going: we want system-level threads
‘of behavior to be correct. When integration testing based on ell graph information
is complete, we stil have quite a leap to get (0 system evel threads, We resolve
this by changing the basis from call graph Information to special Forms of paths
ES
Figure 137. Neighborhood integration.
13.3.2 Neighborhood Integration ——
We can let the mathematics cary us sil farther by bowowing the notion of a
ncighborhood from topology. (This is not too much of a stretch — graph theory
is a branch of topology.) The neighborhood of a node in a graph isthe set of
nodes that are one edge away from the given node. In a directed gmph, this
includes all te immediate predecessor nodes und all the immediate successor
nodes (notice that these correspond to the set of stubs and drivers of the node).
The neighborhoods for nodes 16 and 26 are shown in Figure 137. The 11
neighborhoods for the SATM example (based on the call graph in Figure 13.2)
are listed in Table 133,
‘We can always compute the number of neighborhoods for a given call graph, -
Each interior node will have one neighborhood, plus one extra in case leaf modes
are connected directly 1 the root node. (An interior node has a nonzero indegree
and a nonzero outdegree.) We have:
Interior nodes = nodes ~ (source nodes + sink nodes)
Neighborhoods = interior nodes + source nodes
which combine 10:
Neighborhoods nodes ~ sin nodes PA
Neighborhood integration yields a drastic reduction in the number of ine
gration test sessions (down (0 11 from 40), and it avoids stub and driver
Integration Tsing ns
Table 13.3 _SATM Neighborhoods
Mode Predecesor Successor
16 1 sur
7 100mm
15 vo #5
» 1 m3
» 2 M
2 2 us
2 2 4156825
E 2 412340
5 2m os
2 1. 23,24,26,27,25
1 na 7,2, 2, 16,17, 19,22
development The end result i that neighborhoods are essentially the sandwiches
that we slipped past In the previous section. (ti slighay different, because the
base information for neighbothouds is the call graph, not the decomposition
ce.) What they share with sandwich integration is more significan. neighbor:
¡00d integration testing has the faul isolation difficulties of "medium bang”
integration,
13.3.3 Pros and Cons
The call graplebased integration techniques move away from a purely structural
basis toward a behavioral basis, thus, the underlying assumption is an improve
ment. These techniques also eliminate the stub/driver development effort. In
addition to these advantages, call graph-based integration matches well with
developments characterized by builds and composition, For example, sequences
af neigiborhoods can be used to define builds. Alternatively, we could allow
adjacent neighborhoods (0 merge (nto villages?) and provide an orderly, compo-
Sition-hased growth path. Al! this supports the use of neighbochood-hased inte-
ration for systems developed by life cycles in which composition dominates,
The biggest drawback to call graph based integration testing is the faule isolation
problem, especially for large neighborhoods. A more subtle but closely related
problem occurs. What happens if (when) a fault is found in a node (un) that
appears in several neighborhoods? (For example, the sercen driver unit appears
in 7 of the 11 neighborhoods ) Obviously, we resolve the fault but this means
changing the units code in some way, which in um means that all he previously
tested neighborhoods that contain the changed node need to be retested,
Finally, a fundamental uncertainty exists in any structural form of testing: the
presumption that units integrated with respect 0 strucural information will exhibit
correct behavior, We know where we are going: we want system-level threads
‘of behavior to be correct. When integration testing based on ell graph information
is complete, we stil have quite a leap to get (0 system evel threads, We resolve
this by changing the basis from call graph Information to special Forms of paths
216 Satan Ten: A Craltoman Apprasch Second Edition
13.4 Path-Based Integration
Much of the progress in the development of mathematics comes from an elegant
Patern: have a clear idea of where you want to go, and then define the concepts
that take you there, We do this here for path-based integration testing, hut frst
‘we need to motivate the definitions.
We already know that the combination of structural and functional testing is
highly desirable at the unit level; it would be nice to have a similar capabiliy For
integration (and system) testing, We also know that we want 10 express system
{esting in terms of behavioral threads. Lastly, we revise our goal for integration
testing: instead of testing interfaces among, separately developed and tested units,
we focus on interactions among these unis, Co-funetioning” might be à good
term) Interfaces are suuctural; interaction is behavioral,
‘When a unit executes, some path of source statements is traversed. Suppose
that a call goes to another unit along such 4 path: a that point, control is passed
from the calling unit tothe called unit, where some other path of source statements
is traversed, We cleverly ignored this situation in Part I, because this à a better
place to address the question. Two possibilities are available: abandon the single-
‘entry, single-ext precept and treat such cali as an ext followed by an entry, or
suppress the call statement because control eventually returns to the caling uni
anyway. The suppression choice works well for unit testing, but it is antithetical
to integration testing.
13.4.1 New and Extended Concepts —
To get where we necd to go, we nced to refine some of the program graph
Concepts. As before, these reer 10 programs writen a an impemve language.
Nic allow statement fagments to be a complete statement, and statement fagients
are nodes in the program graph
Definition
A source node in a Program is a statement fragment at which program execution
begias or resumes.
‘The first executable statement in a unit is clearly a source node. Source nodes
also occur immediately after nodes that transfer control t> other units
Definition
A sink node ina unit isa statement fragment at which program execution terminates,
‘The final executable statement in a program is clearly a sink node; so are
‘statements that transfer control 10 other unit,
Definition
A module execution path is a sequence of slatements that begins with a source
node and ends with a sink node, with no intervening sink nodes.
intra Tsing 27
The effect of the definitions so far is that program graphs now have multiple
source and sink nodes. This would greatly increase the complexiy of uni testing,
Put integration testing presumes unit testing Is complete.
Definition
A message is a programming language mechanism by which one unit transfers
‘contol to another uni,
Depending on the programming language, messages can be interpreted sub-
routine invocations, procedure cals, and function references. We follow the conven
tion that the unit that receives a message (ho message destination) always eventually
returns control to the message source. Messages can pass data to other units, We
can finally make the definitions for patrbased inicgration testing. Our goal is to
have an integration testing analog of DD-Paths.
Definition
An MM-Path is un interleaved sequence of module execution paths and messages.
‘The basic idea of an MM-Path is that we can now describe sequences of
module execution paths that include transfers of control among separate units
‘These transfers are by messages, therefore, MM-Paths always represent feasible
execution paths, and these_paths.ctoss.unit boundaties, We can find. MM-Paths
in an extended program graph in which nodes are module execution paths and
ges are messages. The hypothetical example in Figure 13.8 shows an Math
Ghe dark line) in which module A calls module B, which in turn calls module
©. Notice that, for traditional (procedural) software, MM-Paths will always begin
‘and end) in the main program,
In module A, nodes 1 and 5 are source nodes, und nodes 4 and 6 are sink
nodes, Similarly, in module B, nodes 1 and 3 are source nodes, and nodes 2 and
4 are sink nodes. Module € bas a single source node, 1, and a single sink node,
4. Seven module execution paths are shown in Figure 13,3
MERA =<1, 2.3, 6
MENA à
MEDD = 4,2
MER.) = oh &
MERC.) = 2
MERC 2) = 234,5
‘We can now define an integration testing analog of the PD-Path graph that
serves unit testing so effectively
Definition
Given a set of units, their MM-Path graph is the directed graph in which nodes
are module execution paths and edges correspond to messages and returns from
fone unit to another.
13.4 Path-Based Integration
Much of the progress in the development of mathematics comes from an elegant
Patern: have a clear idea of where you want to go, and then define the concepts
that take you there, We do this here for path-based integration testing, hut frst
‘we need to motivate the definitions.
We already know that the combination of structural and functional testing is
highly desirable at the unit level; it would be nice to have a similar capabiliy For
integration (and system) testing, We also know that we want 10 express system
{esting in terms of behavioral threads. Lastly, we revise our goal for integration
testing: instead of testing interfaces among, separately developed and tested units,
we focus on interactions among these unis, Co-funetioning” might be à good
term) Interfaces are suuctural; interaction is behavioral,
‘When a unit executes, some path of source statements is traversed. Suppose
that a call goes to another unit along such 4 path: a that point, control is passed
from the calling unit tothe called unit, where some other path of source statements
is traversed, We cleverly ignored this situation in Part I, because this à a better
place to address the question. Two possibilities are available: abandon the single-
‘entry, single-ext precept and treat such cali as an ext followed by an entry, or
suppress the call statement because control eventually returns to the caling uni
anyway. The suppression choice works well for unit testing, but it is antithetical
to integration testing.
13.4.1 New and Extended Concepts —
To get where we necd to go, we nced to refine some of the program graph
Concepts. As before, these reer 10 programs writen a an impemve language.
Nic allow statement fagments to be a complete statement, and statement fagients
are nodes in the program graph
Definition
A source node in a Program is a statement fragment at which program execution
begias or resumes.
‘The first executable statement in a unit is clearly a source node. Source nodes
also occur immediately after nodes that transfer control t> other units
Definition
A sink node ina unit isa statement fragment at which program execution terminates,
‘The final executable statement in a program is clearly a sink node; so are
‘statements that transfer control 10 other unit,
Definition
A module execution path is a sequence of slatements that begins with a source
node and ends with a sink node, with no intervening sink nodes.
intra Tsing 27
The effect of the definitions so far is that program graphs now have multiple
source and sink nodes. This would greatly increase the complexiy of uni testing,
Put integration testing presumes unit testing Is complete.
Definition
A message is a programming language mechanism by which one unit transfers
‘contol to another uni,
Depending on the programming language, messages can be interpreted sub-
routine invocations, procedure cals, and function references. We follow the conven
tion that the unit that receives a message (ho message destination) always eventually
returns control to the message source. Messages can pass data to other units, We
can finally make the definitions for patrbased inicgration testing. Our goal is to
have an integration testing analog of DD-Paths.
Definition
An MM-Path is un interleaved sequence of module execution paths and messages.
‘The basic idea of an MM-Path is that we can now describe sequences of
module execution paths that include transfers of control among separate units
‘These transfers are by messages, therefore, MM-Paths always represent feasible
execution paths, and these_paths.ctoss.unit boundaties, We can find. MM-Paths
in an extended program graph in which nodes are module execution paths and
ges are messages. The hypothetical example in Figure 13.8 shows an Math
Ghe dark line) in which module A calls module B, which in turn calls module
©. Notice that, for traditional (procedural) software, MM-Paths will always begin
‘and end) in the main program,
In module A, nodes 1 and 5 are source nodes, und nodes 4 and 6 are sink
nodes, Similarly, in module B, nodes 1 and 3 are source nodes, and nodes 2 and
4 are sink nodes. Module € bas a single source node, 1, and a single sink node,
4. Seven module execution paths are shown in Figure 13,3
MERA =<1, 2.3, 6
MENA à
MEDD = 4,2
MER.) = oh &
MERC.) = 2
MERC 2) = 234,5
‘We can now define an integration testing analog of the PD-Path graph that
serves unit testing so effectively
Definition
Given a set of units, their MM-Path graph is the directed graph in which nodes
are module execution paths and edges correspond to messages and returns from
fone unit to another.
28 Sofware Tesung: A Crafsman Approach, Second Ein
Figure 13.8. MM-Pah across thre units,
‚Notice that MM-Path graphs are defined with respect 10 a set of units. This
direcly supporis composition of uni and compositon-based integration testing
We can even compose down to the level of individual module execution paths,
but that is probably more detailed than necessary.
Figure 139 shows the MM-Patb graph for the example in Figure 138. The
solid arrows indicate messages; the corresponding returas are indicated by dotted
arrows, We should consider the relationships among module execution paths,
program paths, DD-Paths, and MM-Paths. A program path is a sequence of DD.
Paths; and an MI-Päih is a sequence of module execution paths. Unforunaicly,
there is no simple relationship between DD-Paths and module execution pathe
Either might be contained in the other, but more likely. they pastally overlap,
Because MM-Paths implement a function that transcends unit boundaries, we Jo
have one relationship: consider the fntersection of an MM-Path with a unit. The
module execution paths in such an intersection are an analog of à slice with
respect to the (MA-Path) funcion. Stated another way, the module execution
Paths in such an intersection are the restiction of the function to the uni in
‘which they occur
The MM-Path definition needs some practical guidelines, How long (deep"
might be bere is an MM-Path? Two observable behavioral criteria put endpoints
(>
æ =>
ES e
Figure 19.9 Math graph derived from Figure 13.8.
Integration Testing 29
Cava ita ow
oros! data fw
Figure 13.10. Data quiescence.
on MM-Puths: message and data quiescence, Message quiescence occuts when a
‘ont that sends no messages is reached (like module € in Figure 1349.
Data quiescence occurs when a sequence of processing culminates in the
creation of stored dats that is not immediately used. In the ValidateCard uni
account balance is obtained, but i is not used until after a successful PIN ent.
Figure 13.10 shows how data quiescence appears in a rudiional data flow diggiam,
Points of quiescence are natural endpoints for an MM-Path
13.4.2 MM-Paths in the SATM System
‘The pseudocode descriptions developed in Chapter 12 ae repeated for convenient
reference; statement fragments are numbered as we did to construct program
graplo. Also, the messages are numbered as comments, We will use these 10
describe selected MM-Paths, The arguments 10 SercenDriver refer o the sercens
as numbered in Figure 126. Procedure GetPIN is really a stub that is designed
10 respond to a correct digit event sequence for ExpectedPIN = 1234.
1. Ma Program
2 State = AwaitCard
À Case State
4 Case 1 AnsitCard
8 SerenDriver, au) em
$ WaickCardSot(CareStocsiaus) msg?
7 Do While CaS llo
pS WiatchCardSiou(Cadsbaits) mag?
3 End While
ia ConrotCardRolersccee) cod
u VallduecuällardOK. PAN) ag
2 WeaddOK
a Then Ste = AvatPIN
a Ee ConwolCarirolrp) meg 6
15 Boalt
le Ste = AwaitCard
17. Ce 2 AFIN.
3 ‘VallatePIN(PINok, PAN) mg?
Figure 13.8. MM-Pah across thre units,
‚Notice that MM-Path graphs are defined with respect 10 a set of units. This
direcly supporis composition of uni and compositon-based integration testing
We can even compose down to the level of individual module execution paths,
but that is probably more detailed than necessary.
Figure 139 shows the MM-Patb graph for the example in Figure 138. The
solid arrows indicate messages; the corresponding returas are indicated by dotted
arrows, We should consider the relationships among module execution paths,
program paths, DD-Paths, and MM-Paths. A program path is a sequence of DD.
Paths; and an MI-Päih is a sequence of module execution paths. Unforunaicly,
there is no simple relationship between DD-Paths and module execution pathe
Either might be contained in the other, but more likely. they pastally overlap,
Because MM-Paths implement a function that transcends unit boundaries, we Jo
have one relationship: consider the fntersection of an MM-Path with a unit. The
module execution paths in such an intersection are an analog of à slice with
respect to the (MA-Path) funcion. Stated another way, the module execution
Paths in such an intersection are the restiction of the function to the uni in
‘which they occur
The MM-Path definition needs some practical guidelines, How long (deep"
might be bere is an MM-Path? Two observable behavioral criteria put endpoints
(>
æ =>
ES e
Figure 19.9 Math graph derived from Figure 13.8.
Integration Testing 29
Cava ita ow
oros! data fw
Figure 13.10. Data quiescence.
on MM-Puths: message and data quiescence, Message quiescence occuts when a
‘ont that sends no messages is reached (like module € in Figure 1349.
Data quiescence occurs when a sequence of processing culminates in the
creation of stored dats that is not immediately used. In the ValidateCard uni
account balance is obtained, but i is not used until after a successful PIN ent.
Figure 13.10 shows how data quiescence appears in a rudiional data flow diggiam,
Points of quiescence are natural endpoints for an MM-Path
13.4.2 MM-Paths in the SATM System
‘The pseudocode descriptions developed in Chapter 12 ae repeated for convenient
reference; statement fragments are numbered as we did to construct program
graplo. Also, the messages are numbered as comments, We will use these 10
describe selected MM-Paths, The arguments 10 SercenDriver refer o the sercens
as numbered in Figure 126. Procedure GetPIN is really a stub that is designed
10 respond to a correct digit event sequence for ExpectedPIN = 1234.
1. Ma Program
2 State = AwaitCard
À Case State
4 Case 1 AnsitCard
8 SerenDriver, au) em
$ WaickCardSot(CareStocsiaus) msg?
7 Do While CaS llo
pS WiatchCardSiou(Cadsbaits) mag?
3 End While
ia ConrotCardRolersccee) cod
u VallduecuällardOK. PAN) ag
2 WeaddOK
a Then Ste = AvatPIN
a Ee ConwolCarirolrp) meg 6
15 Boalt
le Ste = AwaitCard
17. Ce 2 AFIN.
3 ‘VallatePIN(PINok, PAN) mg?
20 Software Tsing: A Catsman'’s Approach, Second Edin
i, in Solera mi) gs
2 Sie Aairrns
zZ Be Seat mag?
E Boat
E State = AvaitCard
38 Case 3: Ains
2 ‘MonageTransaction mg 10
EA States Coso
28. Case 4 Closeseision
3 Y NenTransaciontoques
a “Then Sate Avian
E he PrinRecept wg
x af
E PouTramseionL ce mue 12
El Cleesession me 13
ES CenolCardRo lee) Ein
E Suse Awaiveard
37. End Cue (Se)
38 End. (Main progam SATA)
39. Prossture ValdaePIN(PINGL, PAN)
40. GAPINIOPANIPAN, ExpeceePIN) mes
Sm 2 at
2. Ce mor
Bowe km
ET SesccaDrver2. at) mg 16
Fo en) mg (7
de ERIN = Expect
ra Ten PINGK =n
x Ba SreenDaver.nal) mg 18
# ‘Ty = Secon
So a
SIL Case 2: Second
32 SeceaDriver2- aut) m
3 GanmEnerrin) sep 20
SK UF EneoaPiN'= ExpectetP
EN Ten PINK = Toe
EA Ele Sabine.) ms
En
8, Tye the
3 Cue: Tha
CT men
6 GaPINEerPIN men
EE BN = BapectedPiN.
@ Tran PINK = Tre
e Be SemenDéver(é ol) msg
6 Pike Flee
“u
1. EnaCase Cry)
SE End, (Prooadur Validate)
9. Procure GaN (Entre, Cancel)
70. Local Dat: Dipikeys = 10,1.2,3.4,5,6.7,8.9)
I. Conca = alse
TE Ent = ul ving
73 DighsReve-0
74 Do While NOT(DighsKevaed OR Cancel)
TS Keysensorkeyhi) ig 25
i
Integration Testing 221
76 KeyALIN Digi
na
Es Ent PIN = EneresPi + Kylin
By CAMEO
a Tac =
EN Then SererDiver@ X=) mag 26
E Ena
A Ido =2
54 en SeeeuDive (2 XI) mug 27
# feat
Se WäigtuRevi=3
Fa Tin ScreenDiver (2, XXX) 9928
se Elf
FA Tiguesd= à
ES "Ten Soccer (2, XXX) mg 29
si Eat saad
2 me
a “Cancel Tre
5% Enr
55. End We
3% End. Procedure GaPIN)
SATM Main contains 16 source nodes, All except node 1 are where a procedures
function call returas control: 1, 6, 7, 9, 11, 12, 15, 19, 21, 23, 27, 30, 32, 34, 35,
and 36, SATM Main contains 16 sink nodes. As with the source nodes, most of
these are at procedure/function calls: 5, 6, 8, 10, 12, 14, 17, 18, 20, 22, 26, 31,
33, 34, 35, and 38, Notice that when two sequential procedure calls are used, a
‘Satement-can-bo-both-a-sink and-a-source node. Most-Of the module execution
paths in SATM Main are very short; this pattern is due to the high density of
messages (0 other unis.
Only one nontrivial module execution path is contained in the fist 17 lines
of SATM Main: <1, 2, 3, >. Procedure call, such as <S>, <b>, <B>, <10>, <11>
and <14> are trivial in the sense that net much happens in SATM Main. Other
very short module execution paths are associated with the contro! structures —
for example, <9, 7>, ©, 10>, and «IS, 16>.
Here is the MM-Path for a correct PIN entry on the first ty. The module
execution paths are described by giving the name of the unit followed by the
sequence of he statement fragment numbers. Figure 13.11 illustrates the sequential
nature of an MM-Path using a UML-style sequence dingrum.
Main (12,3, 17, 18)
me?
ValiduePiN (59,40)
mes
GENLEPAN (o peste coe ven)
ValdaePIN 31,4248, 48)
me 16
‘SezeenDzive (no pseudocode gen)
ValinePIN ces)
IT
SEIN, 10, 21,72,73, 74,75)
25
m
KeySeasr mo pseudo code piven) Tesi
GEPIN (76,77, 78, 79,0, 81)
i, in Solera mi) gs
2 Sie Aairrns
zZ Be Seat mag?
E Boat
E State = AvaitCard
38 Case 3: Ains
2 ‘MonageTransaction mg 10
EA States Coso
28. Case 4 Closeseision
3 Y NenTransaciontoques
a “Then Sate Avian
E he PrinRecept wg
x af
E PouTramseionL ce mue 12
El Cleesession me 13
ES CenolCardRo lee) Ein
E Suse Awaiveard
37. End Cue (Se)
38 End. (Main progam SATA)
39. Prossture ValdaePIN(PINGL, PAN)
40. GAPINIOPANIPAN, ExpeceePIN) mes
Sm 2 at
2. Ce mor
Bowe km
ET SesccaDrver2. at) mg 16
Fo en) mg (7
de ERIN = Expect
ra Ten PINGK =n
x Ba SreenDaver.nal) mg 18
# ‘Ty = Secon
So a
SIL Case 2: Second
32 SeceaDriver2- aut) m
3 GanmEnerrin) sep 20
SK UF EneoaPiN'= ExpectetP
EN Ten PINK = Toe
EA Ele Sabine.) ms
En
8, Tye the
3 Cue: Tha
CT men
6 GaPINEerPIN men
EE BN = BapectedPiN.
@ Tran PINK = Tre
e Be SemenDéver(é ol) msg
6 Pike Flee
“u
1. EnaCase Cry)
SE End, (Prooadur Validate)
9. Procure GaN (Entre, Cancel)
70. Local Dat: Dipikeys = 10,1.2,3.4,5,6.7,8.9)
I. Conca = alse
TE Ent = ul ving
73 DighsReve-0
74 Do While NOT(DighsKevaed OR Cancel)
TS Keysensorkeyhi) ig 25
i
Integration Testing 221
76 KeyALIN Digi
na
Es Ent PIN = EneresPi + Kylin
By CAMEO
a Tac =
EN Then SererDiver@ X=) mag 26
E Ena
A Ido =2
54 en SeeeuDive (2 XI) mug 27
# feat
Se WäigtuRevi=3
Fa Tin ScreenDiver (2, XXX) 9928
se Elf
FA Tiguesd= à
ES "Ten Soccer (2, XXX) mg 29
si Eat saad
2 me
a “Cancel Tre
5% Enr
55. End We
3% End. Procedure GaPIN)
SATM Main contains 16 source nodes, All except node 1 are where a procedures
function call returas control: 1, 6, 7, 9, 11, 12, 15, 19, 21, 23, 27, 30, 32, 34, 35,
and 36, SATM Main contains 16 sink nodes. As with the source nodes, most of
these are at procedure/function calls: 5, 6, 8, 10, 12, 14, 17, 18, 20, 22, 26, 31,
33, 34, 35, and 38, Notice that when two sequential procedure calls are used, a
‘Satement-can-bo-both-a-sink and-a-source node. Most-Of the module execution
paths in SATM Main are very short; this pattern is due to the high density of
messages (0 other unis.
Only one nontrivial module execution path is contained in the fist 17 lines
of SATM Main: <1, 2, 3, >. Procedure call, such as <S>, <b>, <B>, <10>, <11>
and <14> are trivial in the sense that net much happens in SATM Main. Other
very short module execution paths are associated with the contro! structures —
for example, <9, 7>, ©, 10>, and «IS, 16>.
Here is the MM-Path for a correct PIN entry on the first ty. The module
execution paths are described by giving the name of the unit followed by the
sequence of he statement fragment numbers. Figure 13.11 illustrates the sequential
nature of an MM-Path using a UML-style sequence dingrum.
Main (12,3, 17, 18)
me?
ValiduePiN (59,40)
mes
GENLEPAN (o peste coe ven)
ValdaePIN 31,4248, 48)
me 16
‘SezeenDzive (no pseudocode gen)
ValinePIN ces)
IT
SEIN, 10, 21,72,73, 74,75)
25
m
KeySeasr mo pseudo code piven) Tesi
GEPIN (76,77, 78, 79,0, 81)
222 Software Testing A Craisman' Approach, Second Edison
ig 26
SerenDrier (0 pseudo cade sven)
‘GaPIN (82,83, 85,86, 85 85, 91,94, 95, 74,5)
p25
KeySensor (na pseud code given)) ‘second digit
GEPIN (16,77 78, 79,80, 82.93, 88)
‘mig 27
SerenDriver (no pres onde given)
PIN (85, 86,8, 89,91, 94.95, 74, 79)
mas
KeySenor (o peuto-cde given). “hed digit
(GEIPIN (16.97.78. 79,80, 82, 83,85, 86,59)
mie 28,
‘ScrenDriver (no pseudocode ven)
PIN BE, 89,91, 94,95, 14,73)
mus 25
Key eno (o pezao-code given) ) Toon digit
PIN (16.77.78, 7,60. 82, 83,5. 86,88, 89,90)
sg 29
SerenDriver (ao pren cede sven)
GAIN (1, 98, 95,7496)
ValldePIN CA, 43,5, 67 68)
Maint)
13.4.4 MM-Path Complexity
Jf you compare the MM Pals in Figures 138 and 13.12 tis inuitvely clear that
the Iater-i more-complex-ihan- the former. Their directed graphs_are shown
together in Figure 13.12. The muliplicty of edges (ex, between SereenDriver
and GetPIN) preserves the message connections, and the double-headed amos
capture the sending and reium of a message (with less elute. Because these
Figure 13.11. UML sequence diagram ofthe sample MM-Path,
Integration Testing 223
Figure 13.12 MM-Path dircted graphs.
are strongly connected directed graphs, we can “blindly” compute their cyclomatic
‘complexities; recall the Form
MG = en + 2p
where p is the number of strongly connected regions. Tor structured procedural
code, we will always have p= 2, s0 the formula reduces to V(G)=e=0.+2
The result, respectively, are WG) = 3 and V(G) = 20.
‘This seems reasonable, The second graph is clearly more complex than the
First and if we remove some of the message trafic, say between the GeiPIN and
KeySensor nodes, both the intutive and the computed comptesity will e reduced.
the role of 2p" in the formula is annoying. ucts lke an offset because it will
always be exactly 2. This suggests simply dropping à. If we were to do this, the
simples: MM-Path, in which unit A calls unit B and B returns, would have à
complexity of 0. Worse ye, a stand-alone una would have a negative complexity
‘of -1, Some other possihilies are suggested in the exercises
13.4.5 Pros and Cons
MM-Paths are a hybrid of functional and structural testing. They are functional in
the sense that they represent actions with inputs and outputs, As such, all the
functional testing techniques are potentially applicable. The structural side comes
from how they are identified, puricularly the MM-Path graph. The net result is
that the cross-check of the functional and structural approaches is consolidated
into the eonsiructs for path-based integration testing, We therefore avoid the pitfall
Of structural testing: and, a the same time, integration testing gains fairy seamless
Junction with system testing, Pai based integration testing works equally well for
software developed in the traditional waterfall process or with one of the com
Pasition-based altemative life cycle models. We will revisit these concepts again
in Chapter 18; these we will sec tha the concepts are equally applicable to object.
‘oriented software testing. The most important advantage of path-based integration
ig 26
SerenDrier (0 pseudo cade sven)
‘GaPIN (82,83, 85,86, 85 85, 91,94, 95, 74,5)
p25
KeySensor (na pseud code given)) ‘second digit
GEPIN (16,77 78, 79,80, 82.93, 88)
‘mig 27
SerenDriver (no pres onde given)
PIN (85, 86,8, 89,91, 94.95, 74, 79)
mas
KeySenor (o peuto-cde given). “hed digit
(GEIPIN (16.97.78. 79,80, 82, 83,85, 86,59)
mie 28,
‘ScrenDriver (no pseudocode ven)
PIN BE, 89,91, 94,95, 14,73)
mus 25
Key eno (o pezao-code given) ) Toon digit
PIN (16.77.78, 7,60. 82, 83,5. 86,88, 89,90)
sg 29
SerenDriver (ao pren cede sven)
GAIN (1, 98, 95,7496)
ValldePIN CA, 43,5, 67 68)
Maint)
13.4.4 MM-Path Complexity
Jf you compare the MM Pals in Figures 138 and 13.12 tis inuitvely clear that
the Iater-i more-complex-ihan- the former. Their directed graphs_are shown
together in Figure 13.12. The muliplicty of edges (ex, between SereenDriver
and GetPIN) preserves the message connections, and the double-headed amos
capture the sending and reium of a message (with less elute. Because these
Figure 13.11. UML sequence diagram ofthe sample MM-Path,
Integration Testing 223
Figure 13.12 MM-Path dircted graphs.
are strongly connected directed graphs, we can “blindly” compute their cyclomatic
‘complexities; recall the Form
MG = en + 2p
where p is the number of strongly connected regions. Tor structured procedural
code, we will always have p= 2, s0 the formula reduces to V(G)=e=0.+2
The result, respectively, are WG) = 3 and V(G) = 20.
‘This seems reasonable, The second graph is clearly more complex than the
First and if we remove some of the message trafic, say between the GeiPIN and
KeySensor nodes, both the intutive and the computed comptesity will e reduced.
the role of 2p" in the formula is annoying. ucts lke an offset because it will
always be exactly 2. This suggests simply dropping à. If we were to do this, the
simples: MM-Path, in which unit A calls unit B and B returns, would have à
complexity of 0. Worse ye, a stand-alone una would have a negative complexity
‘of -1, Some other possihilies are suggested in the exercises
13.4.5 Pros and Cons
MM-Paths are a hybrid of functional and structural testing. They are functional in
the sense that they represent actions with inputs and outputs, As such, all the
functional testing techniques are potentially applicable. The structural side comes
from how they are identified, puricularly the MM-Path graph. The net result is
that the cross-check of the functional and structural approaches is consolidated
into the eonsiructs for path-based integration testing, We therefore avoid the pitfall
Of structural testing: and, a the same time, integration testing gains fairy seamless
Junction with system testing, Pai based integration testing works equally well for
software developed in the traditional waterfall process or with one of the com
Pasition-based altemative life cycle models. We will revisit these concepts again
in Chapter 18; these we will sec tha the concepts are equally applicable to object.
‘oriented software testing. The most important advantage of path-based integration
224 Sofnsar Tsing: A Crfsman' Approach, Second Elion Integration Testing 225
Ouen Vent
Figure 13.13 Main and frst level unit.
of the
teslog stats closely coupled with actual sytem behavior, nte
amaia! males OF decempaskien and call graakbased cien.
The adratages of path based Integration come ats price: mor eon is needed
to ide the Pas. This son 1 probably offer bythe elimination of sub
and diver development
13.5 Case Study
Figure 13:34. Lower evel unit
Our now friar NexiDate i ¡eviten here as a main programs wih a funcional
decomposition into procedures and funcions. This “iteration version” sa slight
extension: there is (limited) added valid checking for months, days, and years
“The pseudocode grows from 50 statements to 81. Figures 13.13 and 13.14 show
the program graph of us inthe integration version of Neate, The funcional
decomposition is shown in Figure 1315, and the Call Graph is shown in Figure 13.16.
1 Moin negre
Type Dae
Months Integer
Day As Teper
Yes ateger
ExtType
Dim today As Date
Dim tomorow Ax Date
2. GuDsod) ist
3 reducto) m
Figure 13.15. Functional decomposition of integration version.
Ouen Vent
Figure 13.13 Main and frst level unit.
of the
teslog stats closely coupled with actual sytem behavior, nte
amaia! males OF decempaskien and call graakbased cien.
The adratages of path based Integration come ats price: mor eon is needed
to ide the Pas. This son 1 probably offer bythe elimination of sub
and diver development
13.5 Case Study
Figure 13:34. Lower evel unit
Our now friar NexiDate i ¡eviten here as a main programs wih a funcional
decomposition into procedures and funcions. This “iteration version” sa slight
extension: there is (limited) added valid checking for months, days, and years
“The pseudocode grows from 50 statements to 81. Figures 13.13 and 13.14 show
the program graph of us inthe integration version of Neate, The funcional
decomposition is shown in Figure 1315, and the Call Graph is shown in Figure 13.16.
1 Moin negre
Type Dae
Months Integer
Day As Teper
Yes ateger
ExtType
Dim today As Date
Dim tomorow Ax Date
2. GuDsod) ist
3 reducto) m
Figure 13.15. Functional decomposition of integration version.
226
Software Testing: À Crafsman' Approach, Second Eton
Figure 13.16. Call graph of integration version.
4 moon = IermentDaeaoday) tm
2 enduro), ‘set
6. End Main
7. Function Lea) Boolean
A
IF gear aise y 9
8 Then
1. IN (gris NOT disse y 100)
" Then sLeap = Tse
Eke o
Ie (yer ist by 40) ~
“Thea ses Troe
ise isLexp = Fate
16 fe
0 End
1 Eeisteap = Fate
nuit
20. End (Funken Le)
21. Funeioe sDayOMon ont, Jen) Tier
2 Cae minthOF
Bee 1. 3,5,7,8,10,
2 tas OM = 3.
m wen
2. aaO Mont 30
m. Gex?
ES HULE) ‘msgs
2. ‘Tea astDayOMeoth = 29
EN Bs sayo Men
a endif
32 EC
33. Bd (Function syOOMoath)
34. Funcion GetDatefaDate) Di
ES
im aDat As Date
FenetonValdDateaDate) Boolean tin scope of Get
ration Tes ay
im rte As Due
“in dayOR, mombOK, eurOK As Boolean
26 Ie(aDateMoeth > 0) AND aDate Month 12)
y Thea monthOK = Te
ES Eke mamOK = Fale
3 ate
4. mo)
a. Ten
a (te Day > 0) AND
Da Day <= Day OMMont{ Date Mont, ate Yen)
a. ‘Then dyOK = ie
m EisedayOK = Pao
4s Endl
46 End
47. fae Yeu > 1818) AND (aDate Year <= 2012)
ss Thea yen OK = Te
# Eso yeuOK = Flee
so at
St. IF(mORtbOK AND dıyOK AND per)
> ‘Ten ValdDst = Tre
5 Ele Valida = Fale
CS
38 End (Farc la)
“Gear body begins bre
56 "De
7 Opener mont)
SE IopuaDate Moni)
3 Osmanen dy")
0 tre
W Online a year)
2 imputada Yeu)
65. Gale Mach «Date Moot
5 GetDueDay = eDate Day
8 GeiDae Yen = Date eur
66. Uni (VliéDaaDao) mar
9. End Funcion Gao)
@& Funcion MeremenDus Det) Date
5%. {Date Day <lauDayOfMoaINaDat Mont) nse
TO ThenaDateDayaDateDay 1
Ti. Bhe awe Day:
2 (abate Mon = 12)
m Then ite Mon 1
”. De ene «Due Vor 1
1, ise aDate Month = Due oath $1
16 Ent
m nate
7. Ed inrestD as)
Software Testing: À Crafsman' Approach, Second Eton
Figure 13.16. Call graph of integration version.
4 moon = IermentDaeaoday) tm
2 enduro), ‘set
6. End Main
7. Function Lea) Boolean
A
IF gear aise y 9
8 Then
1. IN (gris NOT disse y 100)
" Then sLeap = Tse
Eke o
Ie (yer ist by 40) ~
“Thea ses Troe
ise isLexp = Fate
16 fe
0 End
1 Eeisteap = Fate
nuit
20. End (Funken Le)
21. Funeioe sDayOMon ont, Jen) Tier
2 Cae minthOF
Bee 1. 3,5,7,8,10,
2 tas OM = 3.
m wen
2. aaO Mont 30
m. Gex?
ES HULE) ‘msgs
2. ‘Tea astDayOMeoth = 29
EN Bs sayo Men
a endif
32 EC
33. Bd (Function syOOMoath)
34. Funcion GetDatefaDate) Di
ES
im aDat As Date
FenetonValdDateaDate) Boolean tin scope of Get
ration Tes ay
im rte As Due
“in dayOR, mombOK, eurOK As Boolean
26 Ie(aDateMoeth > 0) AND aDate Month 12)
y Thea monthOK = Te
ES Eke mamOK = Fale
3 ate
4. mo)
a. Ten
a (te Day > 0) AND
Da Day <= Day OMMont{ Date Mont, ate Yen)
a. ‘Then dyOK = ie
m EisedayOK = Pao
4s Endl
46 End
47. fae Yeu > 1818) AND (aDate Year <= 2012)
ss Thea yen OK = Te
# Eso yeuOK = Flee
so at
St. IF(mORtbOK AND dıyOK AND per)
> ‘Ten ValdDst = Tre
5 Ele Valida = Fale
CS
38 End (Farc la)
“Gear body begins bre
56 "De
7 Opener mont)
SE IopuaDate Moni)
3 Osmanen dy")
0 tre
W Online a year)
2 imputada Yeu)
65. Gale Mach «Date Moot
5 GetDueDay = eDate Day
8 GeiDae Yen = Date eur
66. Uni (VliéDaaDao) mar
9. End Funcion Gao)
@& Funcion MeremenDus Det) Date
5%. {Date Day <lauDayOfMoaINaDat Mont) nse
TO ThenaDateDayaDateDay 1
Ti. Bhe awe Day:
2 (abate Mon = 12)
m Then ite Mon 1
”. De ene «Due Vor 1
1, ise aDate Month = Due oath $1
16 Ent
m nate
7. Ed inrestD as)
228 Software Testing: A Crafuman Approach, Second Eon
79. Procedar PiniDsaDale)
30. Ouput("Day i, Date Mani, «Date Day
81, End rina)
13.5.1 Decomposition Based Integration
The isLeap and lastDayOfMonth Functions are in the first level of decompostion
because they must be available to both GetDate and IneremeniDate. (We could
move isleap 10 be contained within the scope of lastPayOfMonih,) Pairwise
integration based on the decomposition in Figure 13.15 is problematic; the slap
and lastDayOfMoath Functions are never direcuy called by the Main program, so.
these integration sessions would be emp. Botom-up pairwise Integration stating
with sLeap, then las'DayOMonth, ValidDate, and GetDate would be useful. The
pairs involving Main and GetDate, IncrementDatc, and PrintDate are all useful (rat
Shord sessions. Building stubs for ValidDate and lastDayOMonth would be easy.
13.5.2 Call Graph Based Integration
Pairwise integration based on the Call Graph in Figure 13.16 is an improvement
‘over that for the decomposition bused pairwise integration. Obviously there are
no empty integration sessions because cages refer to actual unk references. There
is still the problem of stubs. Sandwich integration is appropriate because this
example is so small. In fac, it loads isef 10 a build sequence, Build 1 could
confia Main and PeiniDate. Build 2 could comin Main, IncrementDate, lstDsy=
‘OfMonth, and IncrementDate in addition tothe already present PrintDate, Finally,
Build 3 would add the remaining units, GetDate and ValidDate,
Neighborhood integration based on the Call Graph would likely proceed with
the neighborhoods of ValidDate and lastDayOMMonth. Next, we could integrate
the neighborhoods of GetDate and IncremencDate. Finally, we would integrate
the neighborhood of Main, Notice that these neighborhoods form a build sequence.
13.5.3 MM-Path Based Integration
Because the program is data-driven, all MM-Paths begin in and return to the main
Program. Here is the first MM-Palı for May 27, 2002 Chere are others when the
‘main program calls Print Date and Increment Date)
Main (1,25
ml
Date (34, 6, 57, 8, 9, 60, 61,62, 63,6, 65,66)
mr
tite 85,36, 37, 39, 40, 4,42)
mé
TastDeyOPMon (21,22, 23,24, 32, 9) point of menage uisience
Validate (85,45, 4,4, 4, 50,5, 5, 54,55)
(Game (67
Main)
Integration Tes 29
¡Notice thatthe statement fragment sequences (Figures 13.13 and 13.14) identify
fall paths from source to sink modes, This is dire at the point of message
flesence; athe the unis, the pi of node aeguences must be concnated
(0 get the full source (0 sink path. We are now in a strong position to describe
how many MM-Paths are sufficient: the set of MM-Paths should cover all source
Lo siak paths in the set of units. When loups are present, condens
1. condensation graphs
will reduce result in directed acyclk graphs, thereby resolving the problem of
potential infinie (or excessively large) number uf paths
Reference
Fordabl, Matthew, Elementary Mistake Doomed Mar Probe, Te Associated Pres, Oc I,
1999; ako, wav fs org/mar/991001 mt hm.
Exercises
4. Find the source and sink nodes in ValidatePIN and in GetPIN.
2, Find the module execution paths in VilidatePIN
3. Here are some other possible complexity metrics for MM-Paths
Gea
WG) = 05e = n +2.
sum of the outdejrees of the nodes
sum of the nodes plus the sum of the edyes
Makeup some examples, y those out, and vee they have any explanatory
79. Procedar PiniDsaDale)
30. Ouput("Day i, Date Mani, «Date Day
81, End rina)
13.5.1 Decomposition Based Integration
The isLeap and lastDayOfMonth Functions are in the first level of decompostion
because they must be available to both GetDate and IneremeniDate. (We could
move isleap 10 be contained within the scope of lastPayOfMonih,) Pairwise
integration based on the decomposition in Figure 13.15 is problematic; the slap
and lastDayOfMoath Functions are never direcuy called by the Main program, so.
these integration sessions would be emp. Botom-up pairwise Integration stating
with sLeap, then las'DayOMonth, ValidDate, and GetDate would be useful. The
pairs involving Main and GetDate, IncrementDatc, and PrintDate are all useful (rat
Shord sessions. Building stubs for ValidDate and lastDayOMonth would be easy.
13.5.2 Call Graph Based Integration
Pairwise integration based on the Call Graph in Figure 13.16 is an improvement
‘over that for the decomposition bused pairwise integration. Obviously there are
no empty integration sessions because cages refer to actual unk references. There
is still the problem of stubs. Sandwich integration is appropriate because this
example is so small. In fac, it loads isef 10 a build sequence, Build 1 could
confia Main and PeiniDate. Build 2 could comin Main, IncrementDate, lstDsy=
‘OfMonth, and IncrementDate in addition tothe already present PrintDate, Finally,
Build 3 would add the remaining units, GetDate and ValidDate,
Neighborhood integration based on the Call Graph would likely proceed with
the neighborhoods of ValidDate and lastDayOMMonth. Next, we could integrate
the neighborhoods of GetDate and IncremencDate. Finally, we would integrate
the neighborhood of Main, Notice that these neighborhoods form a build sequence.
13.5.3 MM-Path Based Integration
Because the program is data-driven, all MM-Paths begin in and return to the main
Program. Here is the first MM-Palı for May 27, 2002 Chere are others when the
‘main program calls Print Date and Increment Date)
Main (1,25
ml
Date (34, 6, 57, 8, 9, 60, 61,62, 63,6, 65,66)
mr
tite 85,36, 37, 39, 40, 4,42)
mé
TastDeyOPMon (21,22, 23,24, 32, 9) point of menage uisience
Validate (85,45, 4,4, 4, 50,5, 5, 54,55)
(Game (67
Main)
Integration Tes 29
¡Notice thatthe statement fragment sequences (Figures 13.13 and 13.14) identify
fall paths from source to sink modes, This is dire at the point of message
flesence; athe the unis, the pi of node aeguences must be concnated
(0 get the full source (0 sink path. We are now in a strong position to describe
how many MM-Paths are sufficient: the set of MM-Paths should cover all source
Lo siak paths in the set of units. When loups are present, condens
1. condensation graphs
will reduce result in directed acyclk graphs, thereby resolving the problem of
potential infinie (or excessively large) number uf paths
Reference
Fordabl, Matthew, Elementary Mistake Doomed Mar Probe, Te Associated Pres, Oc I,
1999; ako, wav fs org/mar/991001 mt hm.
Exercises
4. Find the source and sink nodes in ValidatePIN and in GetPIN.
2, Find the module execution paths in VilidatePIN
3. Here are some other possible complexity metrics for MM-Paths
Gea
WG) = 05e = n +2.
sum of the outdejrees of the nodes
sum of the nodes plus the sum of the edyes
Makeup some examples, y those out, and vee they have any explanatory
Chapter 14
o es
System Testing
—— nn
OF the three levels of testing, the system level is closest 10 everyday experience.
We test many chings: a used car before we bay it, an online network service
before we subscribe, and so on. A common pattern in these falar forms i that
we evaluate a product in terms of our expectations — not with respect to à
specification or a standard. Consequently, the goal is not 10 find fauhs, but to
demonstrate performance. Because of his, we tend to approach system testing
from a functional standpoint instead of from a-structural ane. Because it-is-0-
inuitively fami, system testing in practice tends to be less formal than it might
be; and this is compounded by the reduced testing interval that usually remains
before a delivery deadline.
‘The crafisperson metaphor continues to serve us. We need a better under.
standing of the medium; as we said in Chapter 12, we wil view system testing
in terms of threads of systerlevel behavior, We begin with a new construct =>
an atomic system function — and further elaboration on the thread concept,
bightighting some of tie practical problems of thread based system testing, System,
testing is closely coupled with requirements specification, therefore, we will
discuss how to find threads in common notations. All this leads to an orderly
thread-hased system testing strategy that exploits the symbiosis between funcional
and structural testing. We will apply the strategy to our simple automated teller
‘machine (SATM) system.
14.1 Threads
Threads are hard to define; in fact, some published definitions are counterpro-
ductive, misleading, or wrong. 1 is possible to simply treat threads as a prinstive
concept that needs no formal definition. For now, we will use examples to develop
a “shared vision." Here are several views of a thread
A scenario of normal usage
A systemvlevel test case
an
o es
System Testing
—— nn
OF the three levels of testing, the system level is closest 10 everyday experience.
We test many chings: a used car before we bay it, an online network service
before we subscribe, and so on. A common pattern in these falar forms i that
we evaluate a product in terms of our expectations — not with respect to à
specification or a standard. Consequently, the goal is not 10 find fauhs, but to
demonstrate performance. Because of his, we tend to approach system testing
from a functional standpoint instead of from a-structural ane. Because it-is-0-
inuitively fami, system testing in practice tends to be less formal than it might
be; and this is compounded by the reduced testing interval that usually remains
before a delivery deadline.
‘The crafisperson metaphor continues to serve us. We need a better under.
standing of the medium; as we said in Chapter 12, we wil view system testing
in terms of threads of systerlevel behavior, We begin with a new construct =>
an atomic system function — and further elaboration on the thread concept,
bightighting some of tie practical problems of thread based system testing, System,
testing is closely coupled with requirements specification, therefore, we will
discuss how to find threads in common notations. All this leads to an orderly
thread-hased system testing strategy that exploits the symbiosis between funcional
and structural testing. We will apply the strategy to our simple automated teller
‘machine (SATM) system.
14.1 Threads
Threads are hard to define; in fact, some published definitions are counterpro-
ductive, misleading, or wrong. 1 is possible to simply treat threads as a prinstive
concept that needs no formal definition. For now, we will use examples to develop
a “shared vision." Here are several views of a thread
A scenario of normal usage
A systemvlevel test case
an
232 Sofmare Tsung: A Cam Approach, Second Kon
A simulus/response pair
Behavior that results from a sequence of system-level inputs
An interleaved sequence of por input and output events
A sequence of transitions in a state muchine description of the system
An interleaved sequence of object messages and method executions
A sequence of machine instructions
A sequence of source instructions
A sequence of MM-paths
A sequence of atomic system functions
Threads have distinct levels. À vnit-level thread is usefully understood as an
executlon-tme path of source instructions or, alternatively, a5 a path of DD-Paths.
An integration-level thread is an MM-Path — that is, an alternating sequence of
module executions and messages. If we continue this pattern, a systemlevel
read Isa sequence of atomic system functions (0 be defined shorty), Because
atomic system functions have post events as thei ¡opus and Oups, a Sequence
Of atomic system functions implies an interleaved sequence of port input and
‘output events. The end results that threads provide a unifying view of our three
levels of testing. Unit testing tess individual funcions, integration testing examines
interactions among unis; and system testing examines interactions among atomic
system functions. In this chapter, we focus on system-level threads and answer
some fundamental questions, such as, “How big i 3 thread? Where do we find
thea? How do we test them?”
14.1.1 Thread Possibilities
Defining the endpoints of a system level thread is a bit awleward, We motivate a
tidy, graph theory-based definition by working buckward from where we want
10 go with threads. Here are four candidate threads in our SATM system:
Entry of a digit
Fury of a personal identification number (PIN)
A simple transaction: ATM Card Entry, PIN Entry, select transaction type
Geposit, withdraw), present account details (checking or savings,
mound, conduct the operation, and report the results
An ATM session containing two ur more simple transactions
Digit entry isa good example of a minimal atomic system function. le begins
‘ith a port input event (he digit keysroke) and ends with a post ouput event
he sercen digit echo), so it qualifies asa stimulus/tesponse pair. If you go back
10 our example in Chapter 13, you will see that his atomic system function (ASF)
is a subpath of the sample MM-Path that we listed in great detail. This level of
‘granularity I 100 fine for the purposes of system testing, We saw this to be an
appropriate level for integration testing,
‘The second candidate, PIN Entry, is a good example of an upper limit to
integration testing and, at the sume time, a starting point of system testing, PIN
Entry is a good example of an atomic system function. It is also a good example
of a family of stimulus/response pairs Gystem-level behavior tht is inated by
‘Syste Testing 233
2 port input event, traverses some programmed logic, and terminates in one of
several possible responses (pon output events). As we saw in Chapter 13, PIN
Entry entails a sequence of systenvlevel inputs and outputs.
A screen requesting PIN digits
An interleaved sequence of digit keystrokes and screen responses
‚The possibilty of cancellation by the customer before the full PIN is entered
A system disposition: A customer has three chances to enter the correct
PIN, Once a correct PIN has been entered, the user sees a screen requesting,
the transaction type; otherwise, a screen advises the customer that the
ATM card will not be returned, and no access to ATM functions is provided.
This is dearly in the domain of system-level testing, and several stimulus!
response pales are evident, Other examples of ASFs include Card Enuy, Trane-
action Selection, Provision of ‘Transaction Details, Transaction Reporting, and
Session Termination, Hach of these is maximal in an Integration testing sense and
‘minimal in a system testing sense. That is, we would not want to integration test
something larger than an ASP; at the same time, we would not want 10 system
test anything smaller.
‘The third candidate, the simple transaction, has a sense of "endhio-end" com-
pletion. À customer could never execute PIN Entry alone (a Card Entry is needed),
bur the simple transaction is commonly executed, This is a good example of à
systemlevel thread: note that it involves the interaction of several ASFS
- The fast possibilty he session) is actually a sequence ofthresch, Thisisalso—
properly a part of system testing; al this level, we aro interested in the interactions
among threads. Unfortunately, most sytem testing efforts never reach the level
of thread interaction. (More on this in Chapter 15)
14.1.2 Thread Definitions
‘We simply our discussion by defining a new tem that helps us get to our desired!
goal
Definition
‘An atomic system function (ASP) is an action that is observahle at the system
level in terms of port input and output events
In an cventdriven system, ASFS are separated by points uf event quiescence;
these occur when a system is (neatly) idle, waiting for a port input event 10
trigger futher processing. Event quiescence has an interesting Pett net insight.
In 2 traditional Peti net, deadlock ocurs when no wansition is enabled. In an
‘event-liven Petri net, event quiescence is similar to deadlock; hut an input event
can bring new life to the net. The SATM system exhibits event quiescence in
several places. one is the tight loop at the beginning of SATM Main, where the
system has displayed the welcome screen and is waiting for à curd to be entered
imo the card slot. Event quiescence is a system-level propery, there is an analog
at the Integration level — message quiescence,
A simulus/response pair
Behavior that results from a sequence of system-level inputs
An interleaved sequence of por input and output events
A sequence of transitions in a state muchine description of the system
An interleaved sequence of object messages and method executions
A sequence of machine instructions
A sequence of source instructions
A sequence of MM-paths
A sequence of atomic system functions
Threads have distinct levels. À vnit-level thread is usefully understood as an
executlon-tme path of source instructions or, alternatively, a5 a path of DD-Paths.
An integration-level thread is an MM-Path — that is, an alternating sequence of
module executions and messages. If we continue this pattern, a systemlevel
read Isa sequence of atomic system functions (0 be defined shorty), Because
atomic system functions have post events as thei ¡opus and Oups, a Sequence
Of atomic system functions implies an interleaved sequence of port input and
‘output events. The end results that threads provide a unifying view of our three
levels of testing. Unit testing tess individual funcions, integration testing examines
interactions among unis; and system testing examines interactions among atomic
system functions. In this chapter, we focus on system-level threads and answer
some fundamental questions, such as, “How big i 3 thread? Where do we find
thea? How do we test them?”
14.1.1 Thread Possibilities
Defining the endpoints of a system level thread is a bit awleward, We motivate a
tidy, graph theory-based definition by working buckward from where we want
10 go with threads. Here are four candidate threads in our SATM system:
Entry of a digit
Fury of a personal identification number (PIN)
A simple transaction: ATM Card Entry, PIN Entry, select transaction type
Geposit, withdraw), present account details (checking or savings,
mound, conduct the operation, and report the results
An ATM session containing two ur more simple transactions
Digit entry isa good example of a minimal atomic system function. le begins
‘ith a port input event (he digit keysroke) and ends with a post ouput event
he sercen digit echo), so it qualifies asa stimulus/tesponse pair. If you go back
10 our example in Chapter 13, you will see that his atomic system function (ASF)
is a subpath of the sample MM-Path that we listed in great detail. This level of
‘granularity I 100 fine for the purposes of system testing, We saw this to be an
appropriate level for integration testing,
‘The second candidate, PIN Entry, is a good example of an upper limit to
integration testing and, at the sume time, a starting point of system testing, PIN
Entry is a good example of an atomic system function. It is also a good example
of a family of stimulus/response pairs Gystem-level behavior tht is inated by
‘Syste Testing 233
2 port input event, traverses some programmed logic, and terminates in one of
several possible responses (pon output events). As we saw in Chapter 13, PIN
Entry entails a sequence of systenvlevel inputs and outputs.
A screen requesting PIN digits
An interleaved sequence of digit keystrokes and screen responses
‚The possibilty of cancellation by the customer before the full PIN is entered
A system disposition: A customer has three chances to enter the correct
PIN, Once a correct PIN has been entered, the user sees a screen requesting,
the transaction type; otherwise, a screen advises the customer that the
ATM card will not be returned, and no access to ATM functions is provided.
This is dearly in the domain of system-level testing, and several stimulus!
response pales are evident, Other examples of ASFs include Card Enuy, Trane-
action Selection, Provision of ‘Transaction Details, Transaction Reporting, and
Session Termination, Hach of these is maximal in an Integration testing sense and
‘minimal in a system testing sense. That is, we would not want to integration test
something larger than an ASP; at the same time, we would not want 10 system
test anything smaller.
‘The third candidate, the simple transaction, has a sense of "endhio-end" com-
pletion. À customer could never execute PIN Entry alone (a Card Entry is needed),
bur the simple transaction is commonly executed, This is a good example of à
systemlevel thread: note that it involves the interaction of several ASFS
- The fast possibilty he session) is actually a sequence ofthresch, Thisisalso—
properly a part of system testing; al this level, we aro interested in the interactions
among threads. Unfortunately, most sytem testing efforts never reach the level
of thread interaction. (More on this in Chapter 15)
14.1.2 Thread Definitions
‘We simply our discussion by defining a new tem that helps us get to our desired!
goal
Definition
‘An atomic system function (ASP) is an action that is observahle at the system
level in terms of port input and output events
In an cventdriven system, ASFS are separated by points uf event quiescence;
these occur when a system is (neatly) idle, waiting for a port input event 10
trigger futher processing. Event quiescence has an interesting Pett net insight.
In 2 traditional Peti net, deadlock ocurs when no wansition is enabled. In an
‘event-liven Petri net, event quiescence is similar to deadlock; hut an input event
can bring new life to the net. The SATM system exhibits event quiescence in
several places. one is the tight loop at the beginning of SATM Main, where the
system has displayed the welcome screen and is waiting for à curd to be entered
imo the card slot. Event quiescence is a system-level propery, there is an analog
at the Integration level — message quiescence,
234 Software sing: A Cralman's Anprosch. Second in
‘The notion of event quiescence dors for ASIS what message quiescence does.
for MN-Paths: it provides a natural endpoint. An ASF begins with 2 port input
‘event, traverses parts of one or more MM-Paths, and terminates with a port output
sent. When viewed from the system level, no compelling reason exists to
decompose an ASF into lower levels of detail (hence the atomiciy). In the SATM
system, digit enuy is a good example of an ASF — so are card enty, cash
dispensing, and session closing. PIN Entry is probably 100 hig, perhaps we should
call lecular system funcion.
‘Atomic system functions represent the seam between integration and system
testing. They are the largest tem to be teste by integration testing and the smallest
item for system testing, We can test an ASP at both levels. Again, the digit entry
ASF is u good example (see the MM-Path example in Chapter 13). During system
testing, the port input event is a physical key press that is detected by KeySensor
and sent to GetPIN as a string variable. (Novice that KeySensor performs the
physical to-ogical transition) GewPIN determines whether the digit key or the
cancel key was pressed and responds accordingly. (Notice that button presses
are ignored.) The ASF terminates with either screen 2 or 4 displayed, Instead of
requiring system keystrokes and visible screen displays, we could use a driver to
provide these and (est the digit entry ASF via integration testing.
Interesting ASFs are included in ValidatePIN. This unit controls all screen.
displays relevant to the PIN entry process. It begias with the display of screen 2
(wich asks the customer (0 enter the PIN). Next, GeiPIN is called, and the system
is event quiescent until a keystroke occurs. These keystrokes inate the GerDigit
[ASS we just discussed. Here, we find a curious integration fault, Notice that
Green Ts dhplyed in 16 places: by the Then ciuses I Te While-loop in
GetPIN and by the fist statements in each Case clause in ValidatePIN. We could
fix this by removing the screen displays from GetPIN and simply retuening the
string (eg, X —) 10 be displayed.
Referring tthe pseudocode example in Chapter 13, four ASPs are included
in statements 75 through 93: cach begins with KeySensor observing a port input
event (a keystroke) and ends with a closely knit family of port output events (the
calls to ScreenDriver with different PIN echoes). We could name these four ASS
GetDigit, Getigi2, GeiDigls, and GerDigiet. They are sighly different because
the later ones include the earlier If statements. CThis module might be reworked
50 that the while-loop repeated a single ASF)
This ponion of the SATM system also illistrates the difference between unit
and integration testing. When GetPIN is uni tested, ts inputs come from KeySensor
which ‘acts like an input statemend. The input space of GePIN contains the
digits O through 9 and the cancel key. (These would likely be treated as string,
or character data.) We could add inputs for the function keys Bl, BZ, and B3; iF
‘we did, traditional equivalence clas testing would be a good choice. The function
we test is whether GeiDigit reconstructs the Keystrokes into a digit string and
‘whether the Boolean indication for the cancel key is correct.
Definition
Given a system defined in terms of atomic system functions, the ASF Graph of
the system is the directed graph in which nodes are ASFS and edges represent
sequential flow.
AAA
Syste esting 235
Definition
‘A source ASF is an atomic system function that appears us a source node in the
ASF graph of a system; similarly, a sink ASP is an atomic system function that
Appears us a sink node in the ASF graph,
in the SATM system, the Card Entry ASF is a source ASF, and the session.
termination ASF is a sine ASF. Notice that intermediary ASPs could never be tested
at the system level by themselves — they need the predecessor ASPs to “gel there”
Definition
A system thread isa path from a source ASF 16 à sink ASF in the ASE graph of
a system.
Definition
Sisen system defined in terms of sem dead, he rad graph of the system
is the directed graph in which nodes are sysem threads and edges represent
sequential execution of individual threads,
‘This set of definitions provides a coherent set of increasingly broader views
‘of threads, stuning with very short threads (within a uni) and ending ith
interactions among systemlevel hreads, We can use these-viows-much-Hke the
‘ocular on a microscope, switching among them to see diferent levels Of grana.
lar. Having these concepts is only part of the problem; supporting them is
another, We next take a testers view of requirements specification to see how to
identify thecads
14.2 Basis Concepts for Requirements Specification
Recall the notion ofa ba fa vetr space: à vet of independent element from
‘hic al the element inthe space can be gene nerd or amaia all
the watations in scores of sequemens specication method, teat and
techniques, we wll seus system esting th reaper fos pas set a feng,
‚mens specification cond du. ains, devis, event and thea ee
pen, 196. Every system can be express in terms of these Ave furan
Cors ee tunes sico nie some combat
es) We examine these Funcamenal concep here to se how heya
the tester's process of thread identification. id
14.2.1 Data
‘When a system is described in terms of is dat, the focus ls on the information
sed and crested by the system. We describe data in terms of variables, data
structures, fields, records, data stores, and files, Entity/relationship models are dhe
most common choice at the highest level, and somo form of a regular expression
(ea, jackson diagrams or data structure diagrams) is used at a more detaled
‘The notion of event quiescence dors for ASIS what message quiescence does.
for MN-Paths: it provides a natural endpoint. An ASF begins with 2 port input
‘event, traverses parts of one or more MM-Paths, and terminates with a port output
sent. When viewed from the system level, no compelling reason exists to
decompose an ASF into lower levels of detail (hence the atomiciy). In the SATM
system, digit enuy is a good example of an ASF — so are card enty, cash
dispensing, and session closing. PIN Entry is probably 100 hig, perhaps we should
call lecular system funcion.
‘Atomic system functions represent the seam between integration and system
testing. They are the largest tem to be teste by integration testing and the smallest
item for system testing, We can test an ASP at both levels. Again, the digit entry
ASF is u good example (see the MM-Path example in Chapter 13). During system
testing, the port input event is a physical key press that is detected by KeySensor
and sent to GetPIN as a string variable. (Novice that KeySensor performs the
physical to-ogical transition) GewPIN determines whether the digit key or the
cancel key was pressed and responds accordingly. (Notice that button presses
are ignored.) The ASF terminates with either screen 2 or 4 displayed, Instead of
requiring system keystrokes and visible screen displays, we could use a driver to
provide these and (est the digit entry ASF via integration testing.
Interesting ASFs are included in ValidatePIN. This unit controls all screen.
displays relevant to the PIN entry process. It begias with the display of screen 2
(wich asks the customer (0 enter the PIN). Next, GeiPIN is called, and the system
is event quiescent until a keystroke occurs. These keystrokes inate the GerDigit
[ASS we just discussed. Here, we find a curious integration fault, Notice that
Green Ts dhplyed in 16 places: by the Then ciuses I Te While-loop in
GetPIN and by the fist statements in each Case clause in ValidatePIN. We could
fix this by removing the screen displays from GetPIN and simply retuening the
string (eg, X —) 10 be displayed.
Referring tthe pseudocode example in Chapter 13, four ASPs are included
in statements 75 through 93: cach begins with KeySensor observing a port input
event (a keystroke) and ends with a closely knit family of port output events (the
calls to ScreenDriver with different PIN echoes). We could name these four ASS
GetDigit, Getigi2, GeiDigls, and GerDigiet. They are sighly different because
the later ones include the earlier If statements. CThis module might be reworked
50 that the while-loop repeated a single ASF)
This ponion of the SATM system also illistrates the difference between unit
and integration testing. When GetPIN is uni tested, ts inputs come from KeySensor
which ‘acts like an input statemend. The input space of GePIN contains the
digits O through 9 and the cancel key. (These would likely be treated as string,
or character data.) We could add inputs for the function keys Bl, BZ, and B3; iF
‘we did, traditional equivalence clas testing would be a good choice. The function
we test is whether GeiDigit reconstructs the Keystrokes into a digit string and
‘whether the Boolean indication for the cancel key is correct.
Definition
Given a system defined in terms of atomic system functions, the ASF Graph of
the system is the directed graph in which nodes are ASFS and edges represent
sequential flow.
AAA
Syste esting 235
Definition
‘A source ASF is an atomic system function that appears us a source node in the
ASF graph of a system; similarly, a sink ASP is an atomic system function that
Appears us a sink node in the ASF graph,
in the SATM system, the Card Entry ASF is a source ASF, and the session.
termination ASF is a sine ASF. Notice that intermediary ASPs could never be tested
at the system level by themselves — they need the predecessor ASPs to “gel there”
Definition
A system thread isa path from a source ASF 16 à sink ASF in the ASE graph of
a system.
Definition
Sisen system defined in terms of sem dead, he rad graph of the system
is the directed graph in which nodes are sysem threads and edges represent
sequential execution of individual threads,
‘This set of definitions provides a coherent set of increasingly broader views
‘of threads, stuning with very short threads (within a uni) and ending ith
interactions among systemlevel hreads, We can use these-viows-much-Hke the
‘ocular on a microscope, switching among them to see diferent levels Of grana.
lar. Having these concepts is only part of the problem; supporting them is
another, We next take a testers view of requirements specification to see how to
identify thecads
14.2 Basis Concepts for Requirements Specification
Recall the notion ofa ba fa vetr space: à vet of independent element from
‘hic al the element inthe space can be gene nerd or amaia all
the watations in scores of sequemens specication method, teat and
techniques, we wll seus system esting th reaper fos pas set a feng,
‚mens specification cond du. ains, devis, event and thea ee
pen, 196. Every system can be express in terms of these Ave furan
Cors ee tunes sico nie some combat
es) We examine these Funcamenal concep here to se how heya
the tester's process of thread identification. id
14.2.1 Data
‘When a system is described in terms of is dat, the focus ls on the information
sed and crested by the system. We describe data in terms of variables, data
structures, fields, records, data stores, and files, Entity/relationship models are dhe
most common choice at the highest level, and somo form of a regular expression
(ea, jackson diagrams or data structure diagrams) is used at a more detaled
236 Software Tsing: A Crataman’s Approach, Second Edition
level. The data-centered view is also the starting point for several flavors of object:
oriented analysis. Data refers to information that ls ether niilized, stored,
updated, or (possibly) destroyed. In the SATM system, inital data describe the
various accounts (PANS) and their PINs, and each account has a data structure
‘with information such as the account balance. As ATM teansictions occu, the
results are kept as created data and used in the daily posting of terminal data to
the central bank. For many systems, the daa-centered view dominates. These
systems are often developed in terms of CRUD actions (Create, Rewieve, Updat
Delete). We could describe the transaction portion of the SATM system in this
way, but it would not work weil for the user interface portion.
Sometimes threads cun be identified directly from the dats model. Relationships
y, many-lo-one, or manyao-
many; these distinctions all have implications fur threads that process the dat.
Por example, if bank customers can have several accounts, each account needs
a unique PIN. If several people can access the same account, they need ATM
cards with identical PANS. We can also find inital dats (such as PAN, ExpectedPIN
pales) that are read but never written. Such readonly data must be par of the
‘system initialization process. IF not, there must be threads that create such data
only data is therefore an indicator of source ASF.
14.2.2 Actions
Aciion-centered modeling is sil a common requirements specification form, This
Scan historia oungrowth ofthe action-centered nature of imperative programming
languages, Actions have inputs and outputs, and these can be either data or port
events, Here are some methodology-speciic synonyms for actions: transform, data
‘eansform, conto! transform, process, activity, task, method, and service, Actions
can also be decomposed into lower level actions, as we saw with the data flow
diagrams in Chapter 12, The inpui/owpu view of actions is exactly the basis of
functional testing, and the decomposition (and eventual implementation) of actions
is the basis of structural testing
14.2.3 Devices
Every system has port devices; these ure the sources and destinations of system
level inputs and outputs (por events). The slight distincion between ports and
por devices is sometimes helpful ro testers. Technically, a port is the point at which
an VO device is attached 10 a system, as in serial and parallel pors, network port,
and telephone ports. Physical actions (keystokes and light emissions from screen)
‘occur on por devices, and these are translated from physical to logica (or logical
10 physica. In the absence of actual por devices, much of system testing can be
accomplished by “moving the port boundary inward” 10 the logical instances of
por events. Prom now on, we wil just use Ihe term “port 10 refer to port devices.
‘The port in the SATM system include the digit and cancel keys, the function keys,
the display screen, the deposit and wilhdrawal doors, the card and receip: slots,
and several less obvious devices, such as the rollers dhat move cards and deposit
envelopes into the machine, che cash dispenser, the receipt printer, and so on.
System tes 237
Thinking about the parts helps the tester define both the input space that
functional system testing needs; similarly the output devices provide ourput-based
functional test information. (For example, we would like to have enough threads
to generate all 15 SATM screens)
14.2.4 Events
vents are somewhat schizophrenic: they have some characteristics of data and
some of actions. An event is a system-level input (oc outpul) that occurs an a
pon device. Similar 10 data, events can be inputs to or outputs of actions. Events
‘can be discrete (such as SATM keystrokes) or (hey can be continuous (such as
temperature, altitude, or pressure). Discrete events necessarily have a time durar
tion, and this can De a critical factor in real-time systems. We might picture input
‘events as destructive read-out data, but is a stretch to imagine output events
as destructive write operations.
Events are like actions in the sense that they ate the translation point between
realwodd physical events and internal logical manifestations af these, Port input
vents are plysicalo-logical translations; and, symmetrically, port output events
ate logicalto-physical transations. System testers should focus on the physical
side of events, not the logical side (the focus of integration testers). Situations
‘occur where the context of present data values changes the logical meaning of
Physical events. In the SATM system, for example, te por input event of
depressing button BI means “Balance” when sercen 5 is displayed, "checking"
‚when sciven 6 is displayed, and yes” when screens 10, 1 and-14-are displayed: —
We refer to such situations as "context-sensitive por events,” and we would expert
to test such evens in cach context
14.2.5 Threads
Unfortunately For testers, thveads are the least frequently used of the five funda
mental constructs. Because we test threads, i usually falls to the tester to fina!
them in the interactions among the data, events, and actions, About the only
place that threads appear per se in a requirements specification is when rapid
prototyping is used in conjunction with a scenario recorder, I is easy to find
threads in controf models, as we will soon see. The problem with this is that
‘control models are just that — they are models, not the real of a system.
14.2.6 Relationships among Basis Concepts
Figure 14.1 is an entiy/celaionship (ED model of our basis concepts, Notice
‘that all relationships are many-to-many: Data and Events are generalized into an
entity; the two relationships to the Action entity are for Inputs and outputs. The
same event can occur on several pons, and typically many events occur on a
single por. Finally, an action can occur in several threads, and a thread is
composed of several actions. This diagram demonstrtes some of the difficulty of
system testing, Testers must use events and threads to ensure that all the many-
10-many relationships among the five basis concepts are corres.
level. The data-centered view is also the starting point for several flavors of object:
oriented analysis. Data refers to information that ls ether niilized, stored,
updated, or (possibly) destroyed. In the SATM system, inital data describe the
various accounts (PANS) and their PINs, and each account has a data structure
‘with information such as the account balance. As ATM teansictions occu, the
results are kept as created data and used in the daily posting of terminal data to
the central bank. For many systems, the daa-centered view dominates. These
systems are often developed in terms of CRUD actions (Create, Rewieve, Updat
Delete). We could describe the transaction portion of the SATM system in this
way, but it would not work weil for the user interface portion.
Sometimes threads cun be identified directly from the dats model. Relationships
y, many-lo-one, or manyao-
many; these distinctions all have implications fur threads that process the dat.
Por example, if bank customers can have several accounts, each account needs
a unique PIN. If several people can access the same account, they need ATM
cards with identical PANS. We can also find inital dats (such as PAN, ExpectedPIN
pales) that are read but never written. Such readonly data must be par of the
‘system initialization process. IF not, there must be threads that create such data
only data is therefore an indicator of source ASF.
14.2.2 Actions
Aciion-centered modeling is sil a common requirements specification form, This
Scan historia oungrowth ofthe action-centered nature of imperative programming
languages, Actions have inputs and outputs, and these can be either data or port
events, Here are some methodology-speciic synonyms for actions: transform, data
‘eansform, conto! transform, process, activity, task, method, and service, Actions
can also be decomposed into lower level actions, as we saw with the data flow
diagrams in Chapter 12, The inpui/owpu view of actions is exactly the basis of
functional testing, and the decomposition (and eventual implementation) of actions
is the basis of structural testing
14.2.3 Devices
Every system has port devices; these ure the sources and destinations of system
level inputs and outputs (por events). The slight distincion between ports and
por devices is sometimes helpful ro testers. Technically, a port is the point at which
an VO device is attached 10 a system, as in serial and parallel pors, network port,
and telephone ports. Physical actions (keystokes and light emissions from screen)
‘occur on por devices, and these are translated from physical to logica (or logical
10 physica. In the absence of actual por devices, much of system testing can be
accomplished by “moving the port boundary inward” 10 the logical instances of
por events. Prom now on, we wil just use Ihe term “port 10 refer to port devices.
‘The port in the SATM system include the digit and cancel keys, the function keys,
the display screen, the deposit and wilhdrawal doors, the card and receip: slots,
and several less obvious devices, such as the rollers dhat move cards and deposit
envelopes into the machine, che cash dispenser, the receipt printer, and so on.
System tes 237
Thinking about the parts helps the tester define both the input space that
functional system testing needs; similarly the output devices provide ourput-based
functional test information. (For example, we would like to have enough threads
to generate all 15 SATM screens)
14.2.4 Events
vents are somewhat schizophrenic: they have some characteristics of data and
some of actions. An event is a system-level input (oc outpul) that occurs an a
pon device. Similar 10 data, events can be inputs to or outputs of actions. Events
‘can be discrete (such as SATM keystrokes) or (hey can be continuous (such as
temperature, altitude, or pressure). Discrete events necessarily have a time durar
tion, and this can De a critical factor in real-time systems. We might picture input
‘events as destructive read-out data, but is a stretch to imagine output events
as destructive write operations.
Events are like actions in the sense that they ate the translation point between
realwodd physical events and internal logical manifestations af these, Port input
vents are plysicalo-logical translations; and, symmetrically, port output events
ate logicalto-physical transations. System testers should focus on the physical
side of events, not the logical side (the focus of integration testers). Situations
‘occur where the context of present data values changes the logical meaning of
Physical events. In the SATM system, for example, te por input event of
depressing button BI means “Balance” when sercen 5 is displayed, "checking"
‚when sciven 6 is displayed, and yes” when screens 10, 1 and-14-are displayed: —
We refer to such situations as "context-sensitive por events,” and we would expert
to test such evens in cach context
14.2.5 Threads
Unfortunately For testers, thveads are the least frequently used of the five funda
mental constructs. Because we test threads, i usually falls to the tester to fina!
them in the interactions among the data, events, and actions, About the only
place that threads appear per se in a requirements specification is when rapid
prototyping is used in conjunction with a scenario recorder, I is easy to find
threads in controf models, as we will soon see. The problem with this is that
‘control models are just that — they are models, not the real of a system.
14.2.6 Relationships among Basis Concepts
Figure 14.1 is an entiy/celaionship (ED model of our basis concepts, Notice
‘that all relationships are many-to-many: Data and Events are generalized into an
entity; the two relationships to the Action entity are for Inputs and outputs. The
same event can occur on several pons, and typically many events occur on a
single por. Finally, an action can occur in several threads, and a thread is
composed of several actions. This diagram demonstrtes some of the difficulty of
system testing, Testers must use events and threads to ensure that all the many-
10-many relationships among the five basis concepts are corres.
20 Sofware Testing: A Calsman' Approach, Second Edition
Data
1.0 input on
‘Action
Event [fie owe Lo Tom
Ta
sequence!
lin Ln
Device ‘Taread
Figure 14.1. E/R model of bass concepts,
14.2.7 Modeling with the Basis Concepts
All flavors of requirements specification develop models of a system in terms
‘of the basis concepts. Figure 14.2 shows three fundamental forms of requirements.
specification models: structural, contextual, and behavioral, Structural models
are used for development; these express the funcional decomposition, data
decomposition, and the interfaces among components. Contextual models are
often the starting point of structural modeling. They emphasize system devices
andy-to-a lesser extent, actons,-and-threads very- indirectly: The models of
behavior (aso called control models) are where four ofthe five basis constructs
come together, Selection of an appropriate control model is the essence of
requirements specification: models that are too weak cannot express important
system behaviors, while models that are 100 powerful typically obscure inter-
esting behaviors. As a general rule, decision tables are u good choice only far
‘computational systems, finite state machines are good for menu-driven systems;
and Pets nets are the model of choice for concurrent systems. Here, we use
finit state machines for the SATM system, and in Chapter 15, we will use Pe
ets to analyze thread interaction,
System Testing 239
| 4
a
ny?
Figure 14.3 Event partitioning view of function .
‘We must make an important distinction between a system itself (reality) and
models of a system, Consider a system in which some function F cannot occur
ntl 10 prerequisite events El and F2 have occurred, and that they can occur
either order, We coukt use the notion of event Paritioning to model this
itustion. The result would be a diagram like that in Figure 14.3.
In the event partitioning view, events El and E2 occur on their respective
‚external devices. When they occur, they are held in Ihe respective event stores.
(An event store acts ike a destructive read operation.) When both events have
‘occurred, function F gets its prerequisite information from the event stores. Notice
that we cannot ‘ell from the model which event ocur firs; we only know that
Doth must occur
——We conid also model the system as Fr sare Tee CSM i Figo 143
in which sites record which event has occurred, The state machine view explicily
shows the (wo orders of the events,
oth models express the sume prerequisites for the function E, and neither is
the reality of the system. Of these two models, the state machine is more useful
to the tester, because paths are instantly convenible to threads,
a 2
à u
Figure 14.2 Modeling relationships among basic construct.
Figure 184 ESM for function &
Data
1.0 input on
‘Action
Event [fie owe Lo Tom
Ta
sequence!
lin Ln
Device ‘Taread
Figure 14.1. E/R model of bass concepts,
14.2.7 Modeling with the Basis Concepts
All flavors of requirements specification develop models of a system in terms
‘of the basis concepts. Figure 14.2 shows three fundamental forms of requirements.
specification models: structural, contextual, and behavioral, Structural models
are used for development; these express the funcional decomposition, data
decomposition, and the interfaces among components. Contextual models are
often the starting point of structural modeling. They emphasize system devices
andy-to-a lesser extent, actons,-and-threads very- indirectly: The models of
behavior (aso called control models) are where four ofthe five basis constructs
come together, Selection of an appropriate control model is the essence of
requirements specification: models that are too weak cannot express important
system behaviors, while models that are 100 powerful typically obscure inter-
esting behaviors. As a general rule, decision tables are u good choice only far
‘computational systems, finite state machines are good for menu-driven systems;
and Pets nets are the model of choice for concurrent systems. Here, we use
finit state machines for the SATM system, and in Chapter 15, we will use Pe
ets to analyze thread interaction,
System Testing 239
| 4
a
ny?
Figure 14.3 Event partitioning view of function .
‘We must make an important distinction between a system itself (reality) and
models of a system, Consider a system in which some function F cannot occur
ntl 10 prerequisite events El and F2 have occurred, and that they can occur
either order, We coukt use the notion of event Paritioning to model this
itustion. The result would be a diagram like that in Figure 14.3.
In the event partitioning view, events El and E2 occur on their respective
‚external devices. When they occur, they are held in Ihe respective event stores.
(An event store acts ike a destructive read operation.) When both events have
‘occurred, function F gets its prerequisite information from the event stores. Notice
that we cannot ‘ell from the model which event ocur firs; we only know that
Doth must occur
——We conid also model the system as Fr sare Tee CSM i Figo 143
in which sites record which event has occurred, The state machine view explicily
shows the (wo orders of the events,
oth models express the sume prerequisites for the function E, and neither is
the reality of the system. Of these two models, the state machine is more useful
to the tester, because paths are instantly convenible to threads,
a 2
à u
Figure 14.2 Modeling relationships among basic construct.
Figure 184 ESM for function &
240 Software esting: A Caluman' Approach Second Eaton
ee
System Testing zu
Figure 145. Topdevel SATM state machine.
14.3 Finding Threads
‘The fine” Sie michi model of the SATM System are Wie Bes’ PLACE 1H WOOK
for system testing threads, We will san with 2 hieracchy of state machines; the
‘upper level is shown in Figure 14.5. At this level, states correspond to stages of
processing, and transitions are caused by logical Gastead of por) events. The
Card Entry “state” for example, would be decomposed into lower levels that deal
‘with details like jammed cards, cards that are upside down, stuck card rollers,
and checking the card against the list of cards for which service is offered. Once
the details of a macro-sete are tested, we use an easy Ihrend 10 get to the next
The PIN Entry state is decomposed into the more detailed view in Figure 14.6,
which is a slight revision of dhe version in Chapter 12. The adjacent states are
shown because they are sources and destinations of transitions from the PIN Entry
portion. At this level, we focus on the PIN Reury mechanism: all of the output
events are true por events, but the input events are still logical events. The stares
and edges are numbered for reference later when we discuss test coverage.
“To stan the thread identification process, we first lit the port events shown
‘on the state transitions; they appear in Table 14.1. We skipped the eject card
event because it is not really part of the PIN Entry component.
‘Notice that Correct PIN and Incorca PIN are really compound port input
events. We cannot actully enter an entire PIN — we enter digits, and at any
point, we might hit the cancel key. These more detailed possiblities are shown
in Figure 147. A truly paranoid tester might decompose the digit por input event
into the actual choices (O-pressed, 1-presscd,.… 9-pressed), but this should have
been tested at a lower level. The por events in the PIN Try finite state machine
n Table 14.2
Figure 14.6 PIN Entry finite state machine
‘The + in the state names in the PIN Try machine refers to which ty (is
second, or third) is passing through the machine,
In addition to the tue
are three logical output events (Correct PIN, Incorréc: PIN, and Canceled); these
correspond exacıly to the higher level events in Figure 14,
‘The hierarchy of fine state machines multiplies the number of threads. There
are 156 distinct paths from the First PIN Try state to the Await Transaction Choice
or Card Ent states in Figure 14.6. Of these, 31 correspond to eventually correct
PIN entres (1 on the ist ty, 5 on the second y, and 25 on the third tey) the
other 125 paths correspond lo those with incorrect digits or with cancel key.
strokes. This is a fall typical ratio. The input portion of systems, especially
interactive systems, usually has a large number of threads to deal with input
errors and exceptions.
It is good form to reach « sate machine in which transitions are caused by
actual port input events, and the actions on transitions are port output evens. IF
we have such a finite state machine, generating system test cases for these threads
is a mechanical process — simply follow a path of trunsiions and note the por
Inputs and outputs as they occur along the path. This interleaved sequence is
Table 14,1_ Events inthe PIN Entry Finite State Machine
Por Input vents Por Output Events
iegtimate card Display screen 1
Wrong card Display screen 2
Correct PIN. Display screen 3
Incorrect PIN Display screen 4
Canceled Display screen 5
ee
System Testing zu
Figure 145. Topdevel SATM state machine.
14.3 Finding Threads
‘The fine” Sie michi model of the SATM System are Wie Bes’ PLACE 1H WOOK
for system testing threads, We will san with 2 hieracchy of state machines; the
‘upper level is shown in Figure 14.5. At this level, states correspond to stages of
processing, and transitions are caused by logical Gastead of por) events. The
Card Entry “state” for example, would be decomposed into lower levels that deal
‘with details like jammed cards, cards that are upside down, stuck card rollers,
and checking the card against the list of cards for which service is offered. Once
the details of a macro-sete are tested, we use an easy Ihrend 10 get to the next
The PIN Entry state is decomposed into the more detailed view in Figure 14.6,
which is a slight revision of dhe version in Chapter 12. The adjacent states are
shown because they are sources and destinations of transitions from the PIN Entry
portion. At this level, we focus on the PIN Reury mechanism: all of the output
events are true por events, but the input events are still logical events. The stares
and edges are numbered for reference later when we discuss test coverage.
“To stan the thread identification process, we first lit the port events shown
‘on the state transitions; they appear in Table 14.1. We skipped the eject card
event because it is not really part of the PIN Entry component.
‘Notice that Correct PIN and Incorca PIN are really compound port input
events. We cannot actully enter an entire PIN — we enter digits, and at any
point, we might hit the cancel key. These more detailed possiblities are shown
in Figure 147. A truly paranoid tester might decompose the digit por input event
into the actual choices (O-pressed, 1-presscd,.… 9-pressed), but this should have
been tested at a lower level. The por events in the PIN Try finite state machine
n Table 14.2
Figure 14.6 PIN Entry finite state machine
‘The + in the state names in the PIN Try machine refers to which ty (is
second, or third) is passing through the machine,
In addition to the tue
are three logical output events (Correct PIN, Incorréc: PIN, and Canceled); these
correspond exacıly to the higher level events in Figure 14,
‘The hierarchy of fine state machines multiplies the number of threads. There
are 156 distinct paths from the First PIN Try state to the Await Transaction Choice
or Card Ent states in Figure 14.6. Of these, 31 correspond to eventually correct
PIN entres (1 on the ist ty, 5 on the second y, and 25 on the third tey) the
other 125 paths correspond lo those with incorrect digits or with cancel key.
strokes. This is a fall typical ratio. The input portion of systems, especially
interactive systems, usually has a large number of threads to deal with input
errors and exceptions.
It is good form to reach « sate machine in which transitions are caused by
actual port input events, and the actions on transitions are port output evens. IF
we have such a finite state machine, generating system test cases for these threads
is a mechanical process — simply follow a path of trunsiions and note the por
Inputs and outputs as they occur along the path. This interleaved sequence is
Table 14,1_ Events inthe PIN Entry Finite State Machine
Por Input vents Por Output Events
iegtimate card Display screen 1
Wrong card Display screen 2
Correct PIN. Display screen 3
Incorrect PIN Display screen 4
Canceled Display screen 5
242 Software Texting: À rafsman Approach, Second Edin
Figure 14.7. PIN Ty finite state machine,
Table 14.2_ Port Events in the PIN Try Finite State Machine
Por pur Events Port Out Events
Dig echo x 7
Cancel cho ce
echo XX!
che xx
performed by the test executor (person er program). Tables 14.3 and 144 follow
‘wo paths through the hierarchic state machines. Table 143 comesponds 10 a
thread in which a PIN is correctly entered on the fist uy. Table 14.4 comesponds
10 a thread in which a PIN is incorrectly entered on the fist ty, cancels after the
third digit on the second ty, and gets it right on the third ty. To make the test
ase explicit, we assume a precondition thatthe expected PIN is 1234
The event in parentheses in the lat row of Table 14.3 isthe logical event that
chumps up* tothe parent state machine and causes a transition there to the Await
‘Transaction Choice state.
If you look closely at Tables 143 and 144, you will see thatthe bottom third
of Table 144 is exactly Table 14.3; thus, a thread can be a subset of another thread.
14.4 Structural Strategies for Thread Testing
Although generating dircad test cases is easy, deciding which ones to actually
ve is more complex. (If you have an automatic test executor, this ls not a
System Testing
243
Table 14.3 Port Event Sequence for Correct PIN on First Try
ETS Port Output Event
Screen 2 displayed with =="
1 pressed
Screen 2 displayed with X=
2 pressed
Screen 2 displayed with 20%
3 pressed
Screen 2 displayed with XX
A pressed
Screen 2 displayed with 20000
(Correct PIN) Screen 5 displayed
Table 14.4 Port Event Sequence for Correct PIN on Third Try
‘pressed
2 pressed
{incorrect PIN)
(Second try)
1 prossed
2 pressed
3 pressed
Cancel key pressed
{End of second ty)
1 pressed
2 pressed
3 pressed
A pressed
{Correct FIN)
Port Opt Event
Screen 2 displayed with
Screen 2 displayed with à
Screen 2 displayed with XX +
Screen 2 displayed with XxX
Screen 2 displayed with 200"
Sereen 3 displayed
Screen 2 displayed with «
Screen 2 displayed with Xe"
Screen 2 displayed with XX.
Screen 2 displayed with XX
Screen 3 displayed
Screen 2 displayed with
Screen 2 displayed with X-
Screen 2 displayed with 20
Screen 2 displayed with 200%
Screen 2 displayed with OK"
Screen 5 displayed
Figure 14.7. PIN Ty finite state machine,
Table 14.2_ Port Events in the PIN Try Finite State Machine
Por pur Events Port Out Events
Dig echo x 7
Cancel cho ce
echo XX!
che xx
performed by the test executor (person er program). Tables 14.3 and 144 follow
‘wo paths through the hierarchic state machines. Table 143 comesponds 10 a
thread in which a PIN is correctly entered on the fist uy. Table 14.4 comesponds
10 a thread in which a PIN is incorrectly entered on the fist ty, cancels after the
third digit on the second ty, and gets it right on the third ty. To make the test
ase explicit, we assume a precondition thatthe expected PIN is 1234
The event in parentheses in the lat row of Table 14.3 isthe logical event that
chumps up* tothe parent state machine and causes a transition there to the Await
‘Transaction Choice state.
If you look closely at Tables 143 and 144, you will see thatthe bottom third
of Table 144 is exactly Table 14.3; thus, a thread can be a subset of another thread.
14.4 Structural Strategies for Thread Testing
Although generating dircad test cases is easy, deciding which ones to actually
ve is more complex. (If you have an automatic test executor, this ls not a
System Testing
243
Table 14.3 Port Event Sequence for Correct PIN on First Try
ETS Port Output Event
Screen 2 displayed with =="
1 pressed
Screen 2 displayed with X=
2 pressed
Screen 2 displayed with 20%
3 pressed
Screen 2 displayed with XX
A pressed
Screen 2 displayed with 20000
(Correct PIN) Screen 5 displayed
Table 14.4 Port Event Sequence for Correct PIN on Third Try
‘pressed
2 pressed
{incorrect PIN)
(Second try)
1 prossed
2 pressed
3 pressed
Cancel key pressed
{End of second ty)
1 pressed
2 pressed
3 pressed
A pressed
{Correct FIN)
Port Opt Event
Screen 2 displayed with
Screen 2 displayed with à
Screen 2 displayed with XX +
Screen 2 displayed with XxX
Screen 2 displayed with 200"
Sereen 3 displayed
Screen 2 displayed with «
Screen 2 displayed with Xe"
Screen 2 displayed with XX.
Screen 2 displayed with XX
Screen 3 displayed
Screen 2 displayed with
Screen 2 displayed with X-
Screen 2 displayed with 20
Screen 2 displayed with 200%
Screen 2 displayed with OK"
Screen 5 displayed
24 Sofware Testing: A Calma Approach, Second Klon
Table 14.5 Thread Paths inthe PIN Tey FSM.
Input Event Sequence Pa of Tansvons
ua OS
195 20,8,
© oa
1c an
uc 432,39, x
mc 4,22 3, x10, xt
problem.) We have the same path explosion problem at the system level that we
ad at the unit level, Just as we did there, we can use the directed graph insights
to make an intelligent choice of threads to test.
14.4.1 Bottom-up Threads
When we organize sate machines in a hierarchy, we can work from the bottom
up. Six paths are used in the PIN Tiy state machine. If we taverse these six, we
test for three things: correct recognition and echo of entered digits, response to
the cancel keystroke, and matching expected and entered PINS. These paths are
described in Table 143 as sequences of the transitions in Figure 147. A thread
that traverses the path is described in terms of its input keystrokes, thus the input
sequence-1234 corresponds 10 the thread described in more- detail in Table 163
he cancel keystroke is indicated with a O).
Once this portion is tested, we can go up a level to the PIN Entry machine,
‘where four patis are used. These four are concerned wit he three try mechanism
and the sequence of screens presented to the user. In Table 146, the paths in
the PIN Kory state machine (Figure 14.6) are named as transition sequences
‘These threads were identified with the goal of path traversal in mind. Recall
from our discussion of structural testing that these goa's can be misleading, The
assumption is that path traversa uncovers Faults, and traversiog a variety of paths
reduces redundancy. The lat path in Table 146 ulusteates how structural goals
can be counterproductive, Hiting the cancel key three times does indeed cause
the three-ry mechanism 10 fail and returns the system to the Card Entry state:
but it seems like a degenenite thread. A more serious flaw occurs with these
threads: we could not really execute them alone because of the hierarchic state
machines. What really happens with the 1235 input sequence in Table 14.57 It
traverses an interesting path in the PIN Try machine; and then it “returns” 10 the
Table 14.6 Thread Paths in the PIN Entry FSM
input Event Sequence Path of Transitions
124 7
12351234 23
Azsscasa 245
eco 246
System Tsing 245
PIN Entry machine where itis seen as a logical event (incorrect PIN), which
‘causes a transition fo state 22 (Second PIN Try). If no additional keystrokes
¡occur this machine would remain in sate 22, We show how to overcome such
situations next
14.4.2 Node and Edge Coverage Metrics
Because the finite state machines are directed graphs, we can use the same test
‘coverage metrics that we applied at the unit level. The hierarchic relationship
means that the uppertevel machine must teat the lower machine ax a procedure
that is entered and returned. (Actually, we need to do this for one more level to
jet 10 true threads that begin withthe Card Entry state) The two obvious choices
are node coverage and edge coverage. Table 147 is extended from Table 144
10 show the node and edge coverage of the threc-ty thread
Table 14.7_ Node and Edge Traversal of a Thread
Por pt Event Port Output Event ‘Nodes Edges
Screen 2 displayed with™ aa
1 pressed zu
Screen 2 displayed with x x
2 pressed 212
Screen 2 displayed with 0%
S pressed — - as
Screen 2 displayed with XX
5 pressed 214
Screen 2 displayed with X000€ se
(incorrect PIN) Screen 3 displayed 215,3 62
(Second try) Screen 2 displayed with ===> 22
‘pressed 224
Screen 2 displayed a
2 pressed 222
Screen 2 displayed with XX + 2
3 pressed 223
Screen 2 displayed with XX 3
Cancel pressed 224 0
(End of second Screen 3 displayed 26 m
m Screen 2 displayed with.” 23 4
1 pressed 231
Screen 2 displayed with a
2 pressed 232
Screen 2 displayed with X- 2
3 pressed 233
Screen 2 displayed with 20" 3
4 pressed 234
‘Screen 2 displayed with WOK" x
{Correct PIN) Screen 5 displayed 235,3 5,5
AAA
Table 14.5 Thread Paths inthe PIN Tey FSM.
Input Event Sequence Pa of Tansvons
ua OS
195 20,8,
© oa
1c an
uc 432,39, x
mc 4,22 3, x10, xt
problem.) We have the same path explosion problem at the system level that we
ad at the unit level, Just as we did there, we can use the directed graph insights
to make an intelligent choice of threads to test.
14.4.1 Bottom-up Threads
When we organize sate machines in a hierarchy, we can work from the bottom
up. Six paths are used in the PIN Tiy state machine. If we taverse these six, we
test for three things: correct recognition and echo of entered digits, response to
the cancel keystroke, and matching expected and entered PINS. These paths are
described in Table 143 as sequences of the transitions in Figure 147. A thread
that traverses the path is described in terms of its input keystrokes, thus the input
sequence-1234 corresponds 10 the thread described in more- detail in Table 163
he cancel keystroke is indicated with a O).
Once this portion is tested, we can go up a level to the PIN Entry machine,
‘where four patis are used. These four are concerned wit he three try mechanism
and the sequence of screens presented to the user. In Table 146, the paths in
the PIN Kory state machine (Figure 14.6) are named as transition sequences
‘These threads were identified with the goal of path traversal in mind. Recall
from our discussion of structural testing that these goa's can be misleading, The
assumption is that path traversa uncovers Faults, and traversiog a variety of paths
reduces redundancy. The lat path in Table 146 ulusteates how structural goals
can be counterproductive, Hiting the cancel key three times does indeed cause
the three-ry mechanism 10 fail and returns the system to the Card Entry state:
but it seems like a degenenite thread. A more serious flaw occurs with these
threads: we could not really execute them alone because of the hierarchic state
machines. What really happens with the 1235 input sequence in Table 14.57 It
traverses an interesting path in the PIN Try machine; and then it “returns” 10 the
Table 14.6 Thread Paths in the PIN Entry FSM
input Event Sequence Path of Transitions
124 7
12351234 23
Azsscasa 245
eco 246
System Tsing 245
PIN Entry machine where itis seen as a logical event (incorrect PIN), which
‘causes a transition fo state 22 (Second PIN Try). If no additional keystrokes
¡occur this machine would remain in sate 22, We show how to overcome such
situations next
14.4.2 Node and Edge Coverage Metrics
Because the finite state machines are directed graphs, we can use the same test
‘coverage metrics that we applied at the unit level. The hierarchic relationship
means that the uppertevel machine must teat the lower machine ax a procedure
that is entered and returned. (Actually, we need to do this for one more level to
jet 10 true threads that begin withthe Card Entry state) The two obvious choices
are node coverage and edge coverage. Table 147 is extended from Table 144
10 show the node and edge coverage of the threc-ty thread
Table 14.7_ Node and Edge Traversal of a Thread
Por pt Event Port Output Event ‘Nodes Edges
Screen 2 displayed with™ aa
1 pressed zu
Screen 2 displayed with x x
2 pressed 212
Screen 2 displayed with 0%
S pressed — - as
Screen 2 displayed with XX
5 pressed 214
Screen 2 displayed with X000€ se
(incorrect PIN) Screen 3 displayed 215,3 62
(Second try) Screen 2 displayed with ===> 22
‘pressed 224
Screen 2 displayed a
2 pressed 222
Screen 2 displayed with XX + 2
3 pressed 223
Screen 2 displayed with XX 3
Cancel pressed 224 0
(End of second Screen 3 displayed 26 m
m Screen 2 displayed with.” 23 4
1 pressed 231
Screen 2 displayed with a
2 pressed 232
Screen 2 displayed with X- 2
3 pressed 233
Screen 2 displayed with 20" 3
4 pressed 234
‘Screen 2 displayed with WOK" x
{Correct PIN) Screen 5 displayed 235,3 5,5
AAA
246 Solace Testing: A Calsmans Approach, Second tion
Table 14.8 Thread/State Incidence
Input Events 21 2x1 2a2 203 And 2a 226 22 23 31
En u er nr ur: x
msm OX x OX x x x x x
com x x où x x x x x x
ICRA x x x x x x x x
merce x x x x x xox x x
Table 14,9 Thread/Transtion Incidence
Inpun Events x 2 3 18 S16 07 1 9 HO M 1 23 4 58 6
En xe KR ® x
wom x x x kx x x
CRM x x x x x x x xx
IRC x x x x x xx x ox xx
merce x x x Kook x x x x
Node (state) coverage is analogous to statement coverage at the unit level
At is the bare minimum. In the PIN Entry example, we cun attain node coverage
without ever executing a thread with a correct PIN. If you examine Table 148,
you will see that two threads Giitated hy C1234 and 123CICIC) traverse all the
‘sates in both machines. ns
Edge (state transition) coverage is a more acceptable standard, if the sate
machines are “well formed” (tansiions in terms of pon events), edge coverage
also guarantees port event coverage, The threads in Table 149 were picked in a
structural way to guarantee that the less traveled edges (hose caused by cancel
keystrokes) ate traversed.
14.5 Functional Strategies for Thread Testing
‘The fine state machine-based approaches to thread identification are clearly
useful, but what if no behavioral model exists for a system to be tested? Ihe
testing crafisperson has two choices: develop a behavioral model or resort to the
systemdevel analogs of functional testing. Recall that when functional test cases
are identified, we use information from the input and output spaces ax well as
the fonction itself. We describe functional threads here in terms of coverage metrics
that are derived from three of he basis concepts (events, pons, and dat).
14.5.1 Event-Based Thread Testing
Consider the space of port input events. Five port input thread coverage metrics
are of interest. Attaining these levels of system test coverage requires a set of
threads such that
E
F
System Tsing 247
Pit: each port input event occurs
PI2: common sequences of port input events occur
IB: each port input event occurs in every “relevant” data context
Pig: for a given contest, all “inappropriate” input events occur
PIS: for a given context, all passible input events occur
‘The PIL metric is a bare minimum and is inadequate for most systems. PIZ
coverage is the most common, and it correspond to the intuitive view of system
testing because it deals with "normal use." I is dificult to quantify, however
‘What is a common sequence of input events? What is an uncommon one?
‘The last three meti are defined in terms of a "contest The best view of a
‘context is that t is a point of event quiescence. In the SATM system, screen
displays occur at the points of event quiescence. The PI3 metric deals with context.
sensitive port input events, These are physical input events that have logical
‘meanings determined by the context within which they occur In the SATM system,
for example, a keystroke on the B function button occurs in ive separate contexts
(screens displayed) and has thece different meanings, The key to this inetic i
that i is driven by an event in all of its contexts. The PI4 and PIS metrics are
‘converses: they start with a context and seek a variety of events. The PM metric
is often used on an informal basis by testecs who uy to break a system. At a
sven context, they want 10 supply unanticipated input events just to see wi
happens. In the SATM system, for example, what happens ifa function bution is
depressed during the PIN Entry stage? The appropriate events are the digit and
cancel keysirakes. Ihe inappropriate input events are the keystrokes.on-the Bl, ———
12, and DB barons.
This is panlaly a specifcaion problem: we are discussing the différence
between prescribed behavior (hing Pat should happen) and prosrbed behavior
things ihr should not happen). Not requirements specifications have a hard
time only descbing prescribed behavior i sal fers who find proscribed
brhavio. The designer who muintim my local ATM system told me that once
someone insened a Rah sandwich inthe depen envelope slot. Apparent they
thought casa waste recepac) A: any rate, no one atthe bank ever antieipned
insorion of a fish Sandwich as a port input event, The Pid and PIS etic are
nai very elective, but they rise one carious ificuly. How does the tester
Know what the expected response should be to a proscibed inp? Are they
simply ignored Should there he an caput warning message? Usually, this let
to the texters intone permis thi is power! Point of focdback to
requirements specification. I aso a highly desrable focus for ether rapid
prototyping or exceuable specications.
‘We ean also define two covenige metrics based on por output events:
POI: each port output event occurs
PO2: each port output event occurs for euch cause
POI coverage is an acceptable minimum. It is panticulaly effective when a
system has a rich varity of output messages for error condions. (The SATM
system does not) PO2 coversge ix a good goal, but it is hard to quantify; we
will revisi this in Chapter 15 when we examine thread interaction, For now, note
Table 14.8 Thread/State Incidence
Input Events 21 2x1 2a2 203 And 2a 226 22 23 31
En u er nr ur: x
msm OX x OX x x x x x
com x x où x x x x x x
ICRA x x x x x x x x
merce x x x x x xox x x
Table 14,9 Thread/Transtion Incidence
Inpun Events x 2 3 18 S16 07 1 9 HO M 1 23 4 58 6
En xe KR ® x
wom x x x kx x x
CRM x x x x x x x xx
IRC x x x x x xx x ox xx
merce x x x Kook x x x x
Node (state) coverage is analogous to statement coverage at the unit level
At is the bare minimum. In the PIN Entry example, we cun attain node coverage
without ever executing a thread with a correct PIN. If you examine Table 148,
you will see that two threads Giitated hy C1234 and 123CICIC) traverse all the
‘sates in both machines. ns
Edge (state transition) coverage is a more acceptable standard, if the sate
machines are “well formed” (tansiions in terms of pon events), edge coverage
also guarantees port event coverage, The threads in Table 149 were picked in a
structural way to guarantee that the less traveled edges (hose caused by cancel
keystrokes) ate traversed.
14.5 Functional Strategies for Thread Testing
‘The fine state machine-based approaches to thread identification are clearly
useful, but what if no behavioral model exists for a system to be tested? Ihe
testing crafisperson has two choices: develop a behavioral model or resort to the
systemdevel analogs of functional testing. Recall that when functional test cases
are identified, we use information from the input and output spaces ax well as
the fonction itself. We describe functional threads here in terms of coverage metrics
that are derived from three of he basis concepts (events, pons, and dat).
14.5.1 Event-Based Thread Testing
Consider the space of port input events. Five port input thread coverage metrics
are of interest. Attaining these levels of system test coverage requires a set of
threads such that
E
F
System Tsing 247
Pit: each port input event occurs
PI2: common sequences of port input events occur
IB: each port input event occurs in every “relevant” data context
Pig: for a given contest, all “inappropriate” input events occur
PIS: for a given context, all passible input events occur
‘The PIL metric is a bare minimum and is inadequate for most systems. PIZ
coverage is the most common, and it correspond to the intuitive view of system
testing because it deals with "normal use." I is dificult to quantify, however
‘What is a common sequence of input events? What is an uncommon one?
‘The last three meti are defined in terms of a "contest The best view of a
‘context is that t is a point of event quiescence. In the SATM system, screen
displays occur at the points of event quiescence. The PI3 metric deals with context.
sensitive port input events, These are physical input events that have logical
‘meanings determined by the context within which they occur In the SATM system,
for example, a keystroke on the B function button occurs in ive separate contexts
(screens displayed) and has thece different meanings, The key to this inetic i
that i is driven by an event in all of its contexts. The PI4 and PIS metrics are
‘converses: they start with a context and seek a variety of events. The PM metric
is often used on an informal basis by testecs who uy to break a system. At a
sven context, they want 10 supply unanticipated input events just to see wi
happens. In the SATM system, for example, what happens ifa function bution is
depressed during the PIN Entry stage? The appropriate events are the digit and
cancel keysirakes. Ihe inappropriate input events are the keystrokes.on-the Bl, ———
12, and DB barons.
This is panlaly a specifcaion problem: we are discussing the différence
between prescribed behavior (hing Pat should happen) and prosrbed behavior
things ihr should not happen). Not requirements specifications have a hard
time only descbing prescribed behavior i sal fers who find proscribed
brhavio. The designer who muintim my local ATM system told me that once
someone insened a Rah sandwich inthe depen envelope slot. Apparent they
thought casa waste recepac) A: any rate, no one atthe bank ever antieipned
insorion of a fish Sandwich as a port input event, The Pid and PIS etic are
nai very elective, but they rise one carious ificuly. How does the tester
Know what the expected response should be to a proscibed inp? Are they
simply ignored Should there he an caput warning message? Usually, this let
to the texters intone permis thi is power! Point of focdback to
requirements specification. I aso a highly desrable focus for ether rapid
prototyping or exceuable specications.
‘We ean also define two covenige metrics based on por output events:
POI: each port output event occurs
PO2: each port output event occurs for euch cause
POI coverage is an acceptable minimum. It is panticulaly effective when a
system has a rich varity of output messages for error condions. (The SATM
system does not) PO2 coversge ix a good goal, but it is hard to quantify; we
will revisi this in Chapter 15 when we examine thread interaction, For now, note
248 Sofware Tesing A Cratsman’ Approsch Second fin
that POZ coverage refers to threads that interact with respect to a port output
‘event. Usually, a given output event only has a small number of causes, In the
SATM system, screen 10 might be displayed for three reasons: the terminal right
be out of cash, it may be impossible to make a connection with the central bank
(0 get the account balance, or the withdrawal door might be fammed. In practice,
some of the most difficuk faults found in field trouble reports are those in which
an output occurs for an unsuspected cause. Here is one example: My local ATM
system (not the SATM) has a screen that informs me that “Your daily withdrawal
limit has been reached” This screen should occur when 1 attempt to withdraw
more than $300 in one day. When I see this screen, I used 10 assume that my.
wife has made a major withdrawal (hreud interaction), 50 1 request a lesser
amount. 1 found out that the ATM also produces this screen when the amount
of cash in the dispenser is low, Instead of providing a lot of cash to the fist
users, the central bank prefers to provide less cash to more user
14.5.2 Port-Based Thread Testing
Pon-based testing is à uscful complement to event-based testing. With porthased
testing, we ask, for each por, what events can occur at that port. We then seek
threads that exercise input ports and output ports with respect to the event lists
foc each port. (This presumes such event lists have been specified: some require.
ments specification techniques mandate such list.) Port based testing is particu
larly useful for systems in which the port devices come from extemal suppliers
The main reason for port-based testing can be seen in the F/R model of the basis
constructs (Figure 14.1). The many-tomany sclaionship between devices and
events should be exercised in both directions. Fvent-bascd testing covers the anc-
tomany relationship from events 10 ports; and, conversely, portbased testing
covers the one-to-many relationship from ports to events. The SATM system fails
US at this point — ng SATM event ecu at more than one por.
14.5.3 Data-Based Thread Testing
Pon and event-based testing work well or systems that are primarily event driven
Such systems are sometimes called "reactive" systems because they react 10 stimuli
(por input events), and often the reaction is ta the form of por output events
Reactive systems have two important characterises: they are long-running (as
‘opposed 10 the short burst of computation we see in a payroll program) and they
‘maintain a relationship with their environment, Typically, event-deiven, reactive
systems do not have a very interesting data model (as we see with the SATM
system), so data model-based threads are not particularly useful. So, what about
conventional systems that are data driven? These systems, described as “static” in
‘Topper (1993), are transformational Gnstead of reactive) they suppor transactions
on a database. When these systems are specified, the E/R model is dominant and
is therefore a fertile source of system testing threads. To attach our discussion to
something familiar, we use the E/R model of a simple library system. See Figure
148 from Topper (1993)
Sytem Testing 209
K— vies ze Ane
adore
Dok Le
rante
Do
Figure 14.8 E/R model ofa library
Here are some typical transactions in the library system:
‘Add a book to the library.
Delete a book from the library.
Add a bonower ta the library.
Delete a borrower fom the library
Loan a book to a borrower,
Process the return of a book from a borrower,
‘These transactions are all mainline threads; in fact, they represent families of
threads, For example, suppose the book loan transaction is atempted fora honowver
‘whose current number of checked -out books isa the Iending limit (a nice boundary
value example). We might also try 10 return a book that was never owned by the
Library. Here is one more: Suppose we delete a borrower that has some unreturned
books. All are interesting threads o test, and all are at the system level
‘We can identify each of these examples, and many more, by close attention
10 the information in the enty/relationship model. As we did with event-based
resting, we describe sets of threads in terms of data-based coverage metrics, These
refer (© relationships for an important reason. Information in relationships is
generally populated by system-level theads, whereas that in the entities is usually
andled at the unit level, (When E/R modeling is the staring point of object
riented analysis, this is enforced by encapsulation)
DMI: Exercise the cardinality of every relationship
DM2: Exercise the participation of every relationship
DM3: Exercise the functional dependencies among relationships
Cardinalty refers to the four possibles of relationship that we discussed in
Chapter 3: oneYo-one, one-to-many, many-to-one, and many-to-many. la the
library example, both the loan and the writes relationships are many-to-many,
‘meaning that one author can write many books, and one book can have many.
that POZ coverage refers to threads that interact with respect to a port output
‘event. Usually, a given output event only has a small number of causes, In the
SATM system, screen 10 might be displayed for three reasons: the terminal right
be out of cash, it may be impossible to make a connection with the central bank
(0 get the account balance, or the withdrawal door might be fammed. In practice,
some of the most difficuk faults found in field trouble reports are those in which
an output occurs for an unsuspected cause. Here is one example: My local ATM
system (not the SATM) has a screen that informs me that “Your daily withdrawal
limit has been reached” This screen should occur when 1 attempt to withdraw
more than $300 in one day. When I see this screen, I used 10 assume that my.
wife has made a major withdrawal (hreud interaction), 50 1 request a lesser
amount. 1 found out that the ATM also produces this screen when the amount
of cash in the dispenser is low, Instead of providing a lot of cash to the fist
users, the central bank prefers to provide less cash to more user
14.5.2 Port-Based Thread Testing
Pon-based testing is à uscful complement to event-based testing. With porthased
testing, we ask, for each por, what events can occur at that port. We then seek
threads that exercise input ports and output ports with respect to the event lists
foc each port. (This presumes such event lists have been specified: some require.
ments specification techniques mandate such list.) Port based testing is particu
larly useful for systems in which the port devices come from extemal suppliers
The main reason for port-based testing can be seen in the F/R model of the basis
constructs (Figure 14.1). The many-tomany sclaionship between devices and
events should be exercised in both directions. Fvent-bascd testing covers the anc-
tomany relationship from events 10 ports; and, conversely, portbased testing
covers the one-to-many relationship from ports to events. The SATM system fails
US at this point — ng SATM event ecu at more than one por.
14.5.3 Data-Based Thread Testing
Pon and event-based testing work well or systems that are primarily event driven
Such systems are sometimes called "reactive" systems because they react 10 stimuli
(por input events), and often the reaction is ta the form of por output events
Reactive systems have two important characterises: they are long-running (as
‘opposed 10 the short burst of computation we see in a payroll program) and they
‘maintain a relationship with their environment, Typically, event-deiven, reactive
systems do not have a very interesting data model (as we see with the SATM
system), so data model-based threads are not particularly useful. So, what about
conventional systems that are data driven? These systems, described as “static” in
‘Topper (1993), are transformational Gnstead of reactive) they suppor transactions
on a database. When these systems are specified, the E/R model is dominant and
is therefore a fertile source of system testing threads. To attach our discussion to
something familiar, we use the E/R model of a simple library system. See Figure
148 from Topper (1993)
Sytem Testing 209
K— vies ze Ane
adore
Dok Le
rante
Do
Figure 14.8 E/R model ofa library
Here are some typical transactions in the library system:
‘Add a book to the library.
Delete a book from the library.
Add a bonower ta the library.
Delete a borrower fom the library
Loan a book to a borrower,
Process the return of a book from a borrower,
‘These transactions are all mainline threads; in fact, they represent families of
threads, For example, suppose the book loan transaction is atempted fora honowver
‘whose current number of checked -out books isa the Iending limit (a nice boundary
value example). We might also try 10 return a book that was never owned by the
Library. Here is one more: Suppose we delete a borrower that has some unreturned
books. All are interesting threads o test, and all are at the system level
‘We can identify each of these examples, and many more, by close attention
10 the information in the enty/relationship model. As we did with event-based
resting, we describe sets of threads in terms of data-based coverage metrics, These
refer (© relationships for an important reason. Information in relationships is
generally populated by system-level theads, whereas that in the entities is usually
andled at the unit level, (When E/R modeling is the staring point of object
riented analysis, this is enforced by encapsulation)
DMI: Exercise the cardinality of every relationship
DM2: Exercise the participation of every relationship
DM3: Exercise the functional dependencies among relationships
Cardinalty refers to the four possibles of relationship that we discussed in
Chapter 3: oneYo-one, one-to-many, many-to-one, and many-to-many. la the
library example, both the loan and the writes relationships are many-to-many,
‘meaning that one author can write many books, and one book can have many.
250 Solace Tsing: A Crafsmans Approach, Second Lion
Table 14.10 SATM Test Data
PAN Expected PIN Checking Balance Savings Balance
10 14 $1000.00 ET]
20 ar 510000 $90.00
ERE) $25.00 $2.00
authors; and that one book can be loaned to many borrowers Cin sequence), and
one borrower can borrow many books. Each of these possiblities results in a
useful system testing thread.
Participation refers to whether every instance of an entity participates in a
jonship. in the writes relationship, both the Book and the Author entities
have mandatory participation (we cannot have a book with no authors, or an
author of no books). In some modeling techniques, participation is expressed in
terms of numerical limits; he Author entity, for example, might be expressed as
“at least 1 and at most 12” When such information is available, it lends directly
10 obvious boundary value system test threads.
Somictimes, transactions determine explicit logical connections among relation
ships; these are known as functional dependencies. For example, we cannot loan
a book that is not possessed by the library, and we would not delete book that
is out on loan, Also, we would not delete a borrower who sull has some books
checked out. These kinds of dependencies are reduced when the database is
normalized; but they stil exist, and they lead to interesting, system est (reads,
14.6 SATM Test Threads
If we apply the discussion of this chapter to the SATM system, we get a set of
threads that constitutes a thorough systenlevel test. We develop such a set of
threads here in teams of an overall state model in which states correspond to key.
atomic system functions. The macro-evel states are: Card Entry, PIN Entry
‘Transaction Request (and processing) and Session Management. The stated order
is the testing order, because these stages arc in prerequisite order. (We cannot
enter a PIN untl successful card entry, we cannot request a transaction until
successful PIN entry, and so on.) We also need some precondition data that
defines some actual accounts with PANS, Expected PINs, and account balances.
These are given in Table 14.10. Two less obvious preconditions are that the ATM
terminal is intlly displaying screen 1 and the total cash available to the withe
drawal dispenser is $500 (in $10 notes).
‘We will express threads in tables in which pales of rows correspond to port inputs
and expected port outputs at each of the four major tages. We sar with three basic
threads, one for exch transaction type (balance inquiy, deposit, and withdrawal.
Thread} (Balance) Card Ent (PAN) PIN Entry Transaction Request Sesion Management
Porttoputs 70 Te BD El
Port Outputs Screen? Screen 5 Screen 6, screen Screen 15, eject
14, $100000 card, screen 1
System Testing 251
In thread 1, a valid card with PAN = 100 is entered, which causes screen 2 to
be displayed. The PIN digis 1234 are entered; and because they match the
expected PIN for the PAN, screen 5 inviting a transaction selection is displayed.
When bution BH is touched the first tine (requestng a balance inquiry), screen
‘asks which account is displayed. When BI is pressed the second time (checking),
screen 14 is displayed and the checking account balance ($1000.00) is printed on.
the receipt. When B2 is pushed, screen 15 is displayed, the receipt is printed, the
ATM card is ejected, and then screen 1 is displayed,
“Thread 2 is a deposit to checking: same PAN and PIN, but B2 is touched when
screen 5 is displayed and RI is touched when screen 6 is displayed, The amount
25.00 is entered when screen 7 is displayed, and then screen 13 is displayed, The
deposit door opens and the deposit envelope is placed in the deposit sot. Screen.
14 is displayed, and when B2 is pushed, screen 15 is displayed, the receipt
showing the new checking account balance of $1025.00 is printed, the ATM card
ls ejected, and then screen 1 is displayed.
Car tn
Thread 2 Depost) (PAN) PIN Entry Transaction Request Session Management
Fon Input 7 EE 2
insert envelope
Port Outputs Screen? Screens Screen screen?, Screen 15, eject
screen 13 card, screen 1
= dep. door opens >
screen M
$1025.00
Thread 3 ls a withdrawal from savings: again the same PAN and PIN, but BB
is touched when screen 5 is displayed, and B2 is vouched when screen 6 is
displayed. The amount 3000 is entered when screen 7 is displayed, and then
screen 31 is displayed. The withdrawal door opens and three $10 notes are
dispensed. Screen 14 is displayed, and when B2 is pushed, sen 15 is displayed,
ihe receipt showing the new savings account balance of $770.00 Is printed, the
ATM card is ejected, and then screen 1 is displayed,
Tweed 3 Card fry
inca) PAN) PIN Ena Transaction Request Sesion Management
Port inputs 100 12 882,300 3
Port Outputs Screen 2 Screen S Screen 6 screen 7, Screen 15, eject card,
screen, screen 1
withdrawal
¿door opens, 3 $10
notes
$770.00
Table 14.10 SATM Test Data
PAN Expected PIN Checking Balance Savings Balance
10 14 $1000.00 ET]
20 ar 510000 $90.00
ERE) $25.00 $2.00
authors; and that one book can be loaned to many borrowers Cin sequence), and
one borrower can borrow many books. Each of these possiblities results in a
useful system testing thread.
Participation refers to whether every instance of an entity participates in a
jonship. in the writes relationship, both the Book and the Author entities
have mandatory participation (we cannot have a book with no authors, or an
author of no books). In some modeling techniques, participation is expressed in
terms of numerical limits; he Author entity, for example, might be expressed as
“at least 1 and at most 12” When such information is available, it lends directly
10 obvious boundary value system test threads.
Somictimes, transactions determine explicit logical connections among relation
ships; these are known as functional dependencies. For example, we cannot loan
a book that is not possessed by the library, and we would not delete book that
is out on loan, Also, we would not delete a borrower who sull has some books
checked out. These kinds of dependencies are reduced when the database is
normalized; but they stil exist, and they lead to interesting, system est (reads,
14.6 SATM Test Threads
If we apply the discussion of this chapter to the SATM system, we get a set of
threads that constitutes a thorough systenlevel test. We develop such a set of
threads here in teams of an overall state model in which states correspond to key.
atomic system functions. The macro-evel states are: Card Entry, PIN Entry
‘Transaction Request (and processing) and Session Management. The stated order
is the testing order, because these stages arc in prerequisite order. (We cannot
enter a PIN untl successful card entry, we cannot request a transaction until
successful PIN entry, and so on.) We also need some precondition data that
defines some actual accounts with PANS, Expected PINs, and account balances.
These are given in Table 14.10. Two less obvious preconditions are that the ATM
terminal is intlly displaying screen 1 and the total cash available to the withe
drawal dispenser is $500 (in $10 notes).
‘We will express threads in tables in which pales of rows correspond to port inputs
and expected port outputs at each of the four major tages. We sar with three basic
threads, one for exch transaction type (balance inquiy, deposit, and withdrawal.
Thread} (Balance) Card Ent (PAN) PIN Entry Transaction Request Sesion Management
Porttoputs 70 Te BD El
Port Outputs Screen? Screen 5 Screen 6, screen Screen 15, eject
14, $100000 card, screen 1
System Testing 251
In thread 1, a valid card with PAN = 100 is entered, which causes screen 2 to
be displayed. The PIN digis 1234 are entered; and because they match the
expected PIN for the PAN, screen 5 inviting a transaction selection is displayed.
When bution BH is touched the first tine (requestng a balance inquiry), screen
‘asks which account is displayed. When BI is pressed the second time (checking),
screen 14 is displayed and the checking account balance ($1000.00) is printed on.
the receipt. When B2 is pushed, screen 15 is displayed, the receipt is printed, the
ATM card is ejected, and then screen 1 is displayed,
“Thread 2 is a deposit to checking: same PAN and PIN, but B2 is touched when
screen 5 is displayed and RI is touched when screen 6 is displayed, The amount
25.00 is entered when screen 7 is displayed, and then screen 13 is displayed, The
deposit door opens and the deposit envelope is placed in the deposit sot. Screen.
14 is displayed, and when B2 is pushed, screen 15 is displayed, the receipt
showing the new checking account balance of $1025.00 is printed, the ATM card
ls ejected, and then screen 1 is displayed.
Car tn
Thread 2 Depost) (PAN) PIN Entry Transaction Request Session Management
Fon Input 7 EE 2
insert envelope
Port Outputs Screen? Screens Screen screen?, Screen 15, eject
screen 13 card, screen 1
= dep. door opens >
screen M
$1025.00
Thread 3 ls a withdrawal from savings: again the same PAN and PIN, but BB
is touched when screen 5 is displayed, and B2 is vouched when screen 6 is
displayed. The amount 3000 is entered when screen 7 is displayed, and then
screen 31 is displayed. The withdrawal door opens and three $10 notes are
dispensed. Screen 14 is displayed, and when B2 is pushed, sen 15 is displayed,
ihe receipt showing the new savings account balance of $770.00 Is printed, the
ATM card is ejected, and then screen 1 is displayed,
Tweed 3 Card fry
inca) PAN) PIN Ena Transaction Request Sesion Management
Port inputs 100 12 882,300 3
Port Outputs Screen 2 Screen S Screen 6 screen 7, Screen 15, eject card,
screen, screen 1
withdrawal
¿door opens, 3 $10
notes
$770.00
252 Software sing: A Crataman Approach, Second Editon
A few of these detailed descriptions are needed 10 show the pattern; the
remaining threads are described in terms of input and output events that are the
objective of the test thread,
Thread 4 isthe shortest thread in the SATM system: it consists of an invalid
Stem Tsing 253
10 rejects the attempt to withdraw more than the account balance, and theead 11
‘ejects the attempt to withdraw more cash than the dispenser contains
car wich mediately reese Dos Cad ty
itt, Ca” py Hans Req Seo Management
a CI Owe
Hse CA PNRNY inicios Pague: Seno Mesas Port Outputs Screen? Screen5 Screens6,7,9,7 Screen 15, eject
Porn a car screen
Por Outputs ct card
mt pent Ted 1o
conta
Fe me DUR m
Pollo the macros long cd, we net prom varios on PIN En
Hay We ge for new end m Tole 149, whieh O ge average in For Oups Seren? Screens Sucens67,8 Screen 8 eet
the FIN Ey ft sate machines nen
Tan
Tends Caty ana)
ne “ONO” NE Transen Ks Seen Margot
a ad L id air = Port Inputs 100 1234 B3, B2, 510.00 1
Fours 10 Anand Cie
ron Oto Por Outpt Seren Sens 67.10 ren 15, es
ren
per
coo
ving exec e wanton process portion, ve posed ti son
Fri 0 Gia Ane a e
Raps ans a sag, where ets te mul nación ep
Tend
Tad 7 mad Codi
(Baars) tb,” ny tana gus Seen Margene
ee, OU Ai a 2 OU Ex
m rr Outputs Screen? Screen5 ScreenG,screen14 Screen 15, screen 5
2 Stoo Scan eet
ace
rienda
pr]
Fume HO BAG A lis okt, the ic provide coverage ofall pa versens exept or
Port Outs Seren2 Serees 32 see 12, which informs the er that depose cance be proceed, Casing
vier ths condiion i probleme pepa we should place «fh sand in de
Moving to the Transaction Request stage, variations exist with respect 10 the
type of transaction (balance, deposit, or withdraw), the account (checking or
savings), and several that deal with the amount requested, Threads 1, 2, and 3
cover he type and account variations, so we focus on the amount-driven threads
Thread 9 rejets the attempl to withdeaw an amount not in $10 increments, Thread
deposit envelope lo. This is an example of a thread selected hy 2 precondition
hat is u hardware failure. We simply give ita thread name here; itis thread 13,
Next, we develop threads 14 through 22 to exercise context sensitive input events,
‘They are shown in Table 14.13; notice chat some of the firs 13 threads exercise
‘context sensitivity.
These 22 threads comprise a reasonable test of the portion of the SATM system
that we have specified. Of course, certain aspects are untested; one good example
A few of these detailed descriptions are needed 10 show the pattern; the
remaining threads are described in terms of input and output events that are the
objective of the test thread,
Thread 4 isthe shortest thread in the SATM system: it consists of an invalid
Stem Tsing 253
10 rejects the attempt to withdraw more than the account balance, and theead 11
‘ejects the attempt to withdraw more cash than the dispenser contains
car wich mediately reese Dos Cad ty
itt, Ca” py Hans Req Seo Management
a CI Owe
Hse CA PNRNY inicios Pague: Seno Mesas Port Outputs Screen? Screen5 Screens6,7,9,7 Screen 15, eject
Porn a car screen
Por Outputs ct card
mt pent Ted 1o
conta
Fe me DUR m
Pollo the macros long cd, we net prom varios on PIN En
Hay We ge for new end m Tole 149, whieh O ge average in For Oups Seren? Screens Sucens67,8 Screen 8 eet
the FIN Ey ft sate machines nen
Tan
Tends Caty ana)
ne “ONO” NE Transen Ks Seen Margot
a ad L id air = Port Inputs 100 1234 B3, B2, 510.00 1
Fours 10 Anand Cie
ron Oto Por Outpt Seren Sens 67.10 ren 15, es
ren
per
coo
ving exec e wanton process portion, ve posed ti son
Fri 0 Gia Ane a e
Raps ans a sag, where ets te mul nación ep
Tend
Tad 7 mad Codi
(Baars) tb,” ny tana gus Seen Margene
ee, OU Ai a 2 OU Ex
m rr Outputs Screen? Screen5 ScreenG,screen14 Screen 15, screen 5
2 Stoo Scan eet
ace
rienda
pr]
Fume HO BAG A lis okt, the ic provide coverage ofall pa versens exept or
Port Outs Seren2 Serees 32 see 12, which informs the er that depose cance be proceed, Casing
vier ths condiion i probleme pepa we should place «fh sand in de
Moving to the Transaction Request stage, variations exist with respect 10 the
type of transaction (balance, deposit, or withdraw), the account (checking or
savings), and several that deal with the amount requested, Threads 1, 2, and 3
cover he type and account variations, so we focus on the amount-driven threads
Thread 9 rejets the attempl to withdeaw an amount not in $10 increments, Thread
deposit envelope lo. This is an example of a thread selected hy 2 precondition
hat is u hardware failure. We simply give ita thread name here; itis thread 13,
Next, we develop threads 14 through 22 to exercise context sensitive input events,
‘They are shown in Table 14.13; notice chat some of the firs 13 threads exercise
‘context sensitivity.
These 22 threads comprise a reasonable test of the portion of the SATM system
that we have specified. Of course, certain aspects are untested; one good example
254 Software Testing: A Caleman' Approach, Second Ein
Table 14.11 Threads for Context-Sensitve Input Events
Thread Keystroke Screen Toga! Meaning
Cancel 2 PIN Entry error
14 Carcel 5 Transaction selection error
15 Cancel 6 Account selection error
16 Cancel 7 Amount election error
17 Cancel 8 Amount selection error
18 Carcel 13 Deposit envelope not ready
18 5 Balance
18 6 Checking
on 10 Yes a nomwithdrawal transaction)
a m 12 Yestanondepositransacılon)
Ros 14. Yes (another transaction)
2 m 5 Deposit
3 08 6 Savings
a BR 10 No (no addtional transaction)
2 2 12 No (mo additional transaction)
on 14 No {no additional transaction)
involves the balance of an account. Consider two threads — one that deposits
#40 to an account, and a second that withdraws $80 — and suppose that the
balance obtained form the central bank at the Card Entry stage is $50.
possibles exis one is to use the central bank balance, record all wansactions,
and then resolve these when the daily posting uccurs. The other is to maintain
a running local balance, which is what would be shown on a balance inquiry
transaction. If the central hunk balance is used, the withdrawal transaction is
rejected; but if the local balance is used, i is processed, This detail was not
addressed in our specification; we will revisit this when we discuss thread
interaction in Chapter 15
‘Another prominent untested portion of the SATM system is the Amount Entry
process that occurs in screens 7 and 8. The possibilty of a cancel keystroke at
any point during amount eniry produces a mulipliciy greater than that of PIN
Entry. A more subtle (and therefore more interesting) test for Amount Entry can
be used. What actually happens when we enter an amoun? To be specific,
suppose we wish (0 enter $40.00. We expect an echo after each digit keysiroke,
but in which position does the echo occur? Two obvious solutions are available:
always require six digits 10 be entered (so we would enter 04000) or use the
high-order digits first and shift left as successive digits are entered, as shown in
Figure 14.9. Most ATM systems use the shift approach, and this mises the subtle
point: How does the ATM system know when all amount digits have been entered?
‘The ATM system clear cannot predict that the deposit amount is $40.00 instead
of $400.00 or $000,000 because no "enter" key is used to signify when the last
digit has been entered. The reason for this digression is that this is a good example
of the kind of deal discovered by testers that is often missing from a requirements
specification, (Such details would likely be found with ether Rapid Protoryping.
or using an executable specification).
System Testing 255
Figure 14.9 Digit echoes with left shits.
14.7 System Testing Guidelines
If we disallow compound sessions (more than one transaction) and if we disregard
the mulúplicay due to Amount Entry possibilities, there are 435 disinct threads
per valid account in the SATM system. Factor inthe effects of compound sessions
and the Amount Entry possibilities, and tens of thousands of threads are possible
for the SATM system. We end this chapter with three strategies to deal with the
thread explosion problem,
14.7.1 Pseudostructural System Testing
When we studied uni testing, we saw that the combination of functional and
structural testing yields a desirble crosscheck. We have something similar with
system-level threads: we defined ten system-level, functional coverage metrics in
Section 145, and two graph-based metics (node and edge coverage) in Section
144. We can use the griph-based metres as a crosscheck on the functional
threads in much the same way that we used DD-Paths at the unit level to identify
saps and redundancies in functional test cases. We can only claim pscudostructural
testing Jorgensen, 1994) because the node and edge coverage metrics are defined
in terms of a control model of a system and are not derived directly from the
system implementation. (Recall that we stared out with a concern over th
distinction between reality and models of reality.) In general, behavioral models
are only approximations of a system's reality, which is why we could decompose
‘our models down to several levels of deal. If we made a true structural model,
Table 14.11 Threads for Context-Sensitve Input Events
Thread Keystroke Screen Toga! Meaning
Cancel 2 PIN Entry error
14 Carcel 5 Transaction selection error
15 Cancel 6 Account selection error
16 Cancel 7 Amount election error
17 Cancel 8 Amount selection error
18 Carcel 13 Deposit envelope not ready
18 5 Balance
18 6 Checking
on 10 Yes a nomwithdrawal transaction)
a m 12 Yestanondepositransacılon)
Ros 14. Yes (another transaction)
2 m 5 Deposit
3 08 6 Savings
a BR 10 No (no addtional transaction)
2 2 12 No (mo additional transaction)
on 14 No {no additional transaction)
involves the balance of an account. Consider two threads — one that deposits
#40 to an account, and a second that withdraws $80 — and suppose that the
balance obtained form the central bank at the Card Entry stage is $50.
possibles exis one is to use the central bank balance, record all wansactions,
and then resolve these when the daily posting uccurs. The other is to maintain
a running local balance, which is what would be shown on a balance inquiry
transaction. If the central hunk balance is used, the withdrawal transaction is
rejected; but if the local balance is used, i is processed, This detail was not
addressed in our specification; we will revisit this when we discuss thread
interaction in Chapter 15
‘Another prominent untested portion of the SATM system is the Amount Entry
process that occurs in screens 7 and 8. The possibilty of a cancel keystroke at
any point during amount eniry produces a mulipliciy greater than that of PIN
Entry. A more subtle (and therefore more interesting) test for Amount Entry can
be used. What actually happens when we enter an amoun? To be specific,
suppose we wish (0 enter $40.00. We expect an echo after each digit keysiroke,
but in which position does the echo occur? Two obvious solutions are available:
always require six digits 10 be entered (so we would enter 04000) or use the
high-order digits first and shift left as successive digits are entered, as shown in
Figure 14.9. Most ATM systems use the shift approach, and this mises the subtle
point: How does the ATM system know when all amount digits have been entered?
‘The ATM system clear cannot predict that the deposit amount is $40.00 instead
of $400.00 or $000,000 because no "enter" key is used to signify when the last
digit has been entered. The reason for this digression is that this is a good example
of the kind of deal discovered by testers that is often missing from a requirements
specification, (Such details would likely be found with ether Rapid Protoryping.
or using an executable specification).
System Testing 255
Figure 14.9 Digit echoes with left shits.
14.7 System Testing Guidelines
If we disallow compound sessions (more than one transaction) and if we disregard
the mulúplicay due to Amount Entry possibilities, there are 435 disinct threads
per valid account in the SATM system. Factor inthe effects of compound sessions
and the Amount Entry possibilities, and tens of thousands of threads are possible
for the SATM system. We end this chapter with three strategies to deal with the
thread explosion problem,
14.7.1 Pseudostructural System Testing
When we studied uni testing, we saw that the combination of functional and
structural testing yields a desirble crosscheck. We have something similar with
system-level threads: we defined ten system-level, functional coverage metrics in
Section 145, and two graph-based metics (node and edge coverage) in Section
144. We can use the griph-based metres as a crosscheck on the functional
threads in much the same way that we used DD-Paths at the unit level to identify
saps and redundancies in functional test cases. We can only claim pscudostructural
testing Jorgensen, 1994) because the node and edge coverage metrics are defined
in terms of a control model of a system and are not derived directly from the
system implementation. (Recall that we stared out with a concern over th
distinction between reality and models of reality.) In general, behavioral models
are only approximations of a system's reality, which is why we could decompose
‘our models down to several levels of deal. If we made a true structural model,
256 Sofware Tsing: A Cratsmans Approach, Second Ein
its size and complexity would make i 100 cumbersome (0 use. The big weakness
af pseudostructural metrics is dat the underlying model may be a poor choice.
‘The three most common behavioral models (decision tables, finite state machines,
and Petri nets) are appropriate, respectively, 10 transformational, interactive, and
concurrent systeas
Decision tables and finite sate machines are good choices for ASF testing, IF
an ASP is described using a decision table, conditions typically include port input
events, and actions are por output events. We can then devise test cases that
cover every condition, every action, or, most completely, every mule, As we saw
Tor finite state machine models, test cases can cover every sate, every transition,
or every path.
‘Thread testing based on decision tables is cumbersome. We might describe
threads as sequences of rules from different decision tables, but this becomes
very messy 10 track In terms of coverage. We need finite state machines as a
minimum, and if any form of interaction occurs, Peui nets are a better choice.
‘There, we can devise thread tests that cover every place, every tension, and
every sequence of transitions,
14.7.2 Operational Profiles
In its most general form, Zipfs Law holds that 80% of the activities occur in 20%
Of the space. Activities and space can be interpreted in numerous ways: people
‘with messy desks hardly ever use most of their desktop duucr, programmers
seldoï use more {han 20% of the Teatuies of ther favérite programming language;
and Shakespeare (whose writings contain an enormous vocabulary) uses a small
fraction of his vocabulary most ofthe time. Zipfs Law applies to software (and
testing) in several ways. The most useful interpretation for testers is thatthe space
consists ofall possible threads, and activities are thread executions (or traversals)
‘Thas, for a system with many threads, 80% of the execution traverses only 20%
of the threads,
Recall tha a failure occurs when a fault is executed. The whole idea of resting
110 execute test cases such that, when a failure occurs, the presence of a full
is revealed. We can make an important distinction: the distibution of faults in a
system is only indirecly related to the reliability of the system. The simplest view
of system reliability is the probabiliy that no failure occurs during a specific time
interval. (Notice that no mention is even made of faults, the number of faults, or
fault density) IF the only fault are “in the comers’ on threads that are seldom
traversed, the overall reliably is higher than ifthe same number of faults were
‘on “high'taffic’ threads, The ides of operational profiles is to determine the
‘execution frequencies of various threads and 10 use this Information to select
threads for system testing. Panicularly when test time is limited (usually), operi-
tional profiles maximize the probability of finding faults by inducing failures in
the most frequently traversed threads.
One way to determine the operational profile of system is to use a decision
tree, This works particularly well when system behavior is modeled in hierarchic
state machines, as we did with the SATM system, For any state, we find (or
estimate) the probability of each outgoing transition (the sum of dhese must be
1). When a snte is decomposed into a lower level, the probabilies at the lower
System Tsing 257
Figure 14.10 Transition probabilities for the SATM system.
level become "split edges” at Ihe upper level. Figure 14.10 shows the result of
this with hypothetical transition probabilites. Given the transition probabilities,
the overall probability of a thread is simply the product of the transition proba
bilities along the thread. Table 14.12 shows this calculation for the most and least
frequent threads,
Operational profiles provide a feeling for the trafic mix of a delivered system
This is helpful for reasons caber than only optiniring system testing, These profiles
‘can also be used in conjunction with simulators to get an carl indication of
execution time performance and system transaction capacity. Many times, cus-
tomers are a good source of traffic mix information, so this approach to system
testing is often well received simply because it makes an attempt 10 replicate the
reality of a delivered system,
Table 14.12 Thread Probabilities
Common Thread — Probabilics Rare Thread Prababtes
legitimate Card 035 Tegitimate Card 035
PIN OK istry 090 Bad PIN tst ry 010
withdraw 085 Bad PIN 2nd try 010
Normal 085 PIN OK 3rd ty 090
Withdraw 08
Low Cash oor
06177375 00072675
its size and complexity would make i 100 cumbersome (0 use. The big weakness
af pseudostructural metrics is dat the underlying model may be a poor choice.
‘The three most common behavioral models (decision tables, finite state machines,
and Petri nets) are appropriate, respectively, 10 transformational, interactive, and
concurrent systeas
Decision tables and finite sate machines are good choices for ASF testing, IF
an ASP is described using a decision table, conditions typically include port input
events, and actions are por output events. We can then devise test cases that
cover every condition, every action, or, most completely, every mule, As we saw
Tor finite state machine models, test cases can cover every sate, every transition,
or every path.
‘Thread testing based on decision tables is cumbersome. We might describe
threads as sequences of rules from different decision tables, but this becomes
very messy 10 track In terms of coverage. We need finite state machines as a
minimum, and if any form of interaction occurs, Peui nets are a better choice.
‘There, we can devise thread tests that cover every place, every tension, and
every sequence of transitions,
14.7.2 Operational Profiles
In its most general form, Zipfs Law holds that 80% of the activities occur in 20%
Of the space. Activities and space can be interpreted in numerous ways: people
‘with messy desks hardly ever use most of their desktop duucr, programmers
seldoï use more {han 20% of the Teatuies of ther favérite programming language;
and Shakespeare (whose writings contain an enormous vocabulary) uses a small
fraction of his vocabulary most ofthe time. Zipfs Law applies to software (and
testing) in several ways. The most useful interpretation for testers is thatthe space
consists ofall possible threads, and activities are thread executions (or traversals)
‘Thas, for a system with many threads, 80% of the execution traverses only 20%
of the threads,
Recall tha a failure occurs when a fault is executed. The whole idea of resting
110 execute test cases such that, when a failure occurs, the presence of a full
is revealed. We can make an important distinction: the distibution of faults in a
system is only indirecly related to the reliability of the system. The simplest view
of system reliability is the probabiliy that no failure occurs during a specific time
interval. (Notice that no mention is even made of faults, the number of faults, or
fault density) IF the only fault are “in the comers’ on threads that are seldom
traversed, the overall reliably is higher than ifthe same number of faults were
‘on “high'taffic’ threads, The ides of operational profiles is to determine the
‘execution frequencies of various threads and 10 use this Information to select
threads for system testing. Panicularly when test time is limited (usually), operi-
tional profiles maximize the probability of finding faults by inducing failures in
the most frequently traversed threads.
One way to determine the operational profile of system is to use a decision
tree, This works particularly well when system behavior is modeled in hierarchic
state machines, as we did with the SATM system, For any state, we find (or
estimate) the probability of each outgoing transition (the sum of dhese must be
1). When a snte is decomposed into a lower level, the probabilies at the lower
System Tsing 257
Figure 14.10 Transition probabilities for the SATM system.
level become "split edges” at Ihe upper level. Figure 14.10 shows the result of
this with hypothetical transition probabilites. Given the transition probabilities,
the overall probability of a thread is simply the product of the transition proba
bilities along the thread. Table 14.12 shows this calculation for the most and least
frequent threads,
Operational profiles provide a feeling for the trafic mix of a delivered system
This is helpful for reasons caber than only optiniring system testing, These profiles
‘can also be used in conjunction with simulators to get an carl indication of
execution time performance and system transaction capacity. Many times, cus-
tomers are a good source of traffic mix information, so this approach to system
testing is often well received simply because it makes an attempt 10 replicate the
reality of a delivered system,
Table 14.12 Thread Probabilities
Common Thread — Probabilics Rare Thread Prababtes
legitimate Card 035 Tegitimate Card 035
PIN OK istry 090 Bad PIN tst ry 010
withdraw 085 Bad PIN 2nd try 010
Normal 085 PIN OK 3rd ty 090
Withdraw 08
Low Cash oor
06177375 00072675
258 Sofware Testing: A Calsman's Approach, Second tion
14.7.3 Progression vs, Regression Testing
‘When we discussed software development life cycles in Chapter 12, we mentioned
that the use of buids induces the need for regression testing. When build 2 i
added to build 1, we test the new material in build 2, and we also retest build
110 see that he new material has no deleterious effect on build 1 contents. The
industrial average for such “ripple effect” is that 20% of changes 10 an existing
system induce new faults inthe system.) If projet has several builds, regression
testing implics a significant repettion of testing, especially for the early bulk,
‘We can reduce this by concentrating on the difference between progression and
regression testing
‘The most common approach to regression testing is 10 simply repeat the system
tests, We can refine this (and drastically reduce the effor) by choosing test read
wi respect to the goals of regression and progression testing. With progression
testing, we are testing new territory, so we expecta higher failure rate than with
regression testing, Another diference is that because we expect to find more
faults with progression testing, we need to be able to locate the faults. This
requires test cases with a diagnostic cpabiliy — that I, tests that can fal only
in a few ways. For thread-based testing, progression testing should use shorter
heads that can fal only in a few ways. These threads might he ordered as we
did with the SATM thread test set, such that longer thread are bul up from
shorter and previously tested) threads.
‘We have lower expectations of failure with regression esting, and we are less
‘concerned with fal isolation, Taken together, his means regression testing should _
use longer threads, ones that can fil in several ways, If we think in terms of
‘cavecige, both progression and regression testing Will have thorough coverage,
bat the density is diferent, Sate and tension coverage matices like Tables 148
and 149) will be sparse for progression testing threads and dense for regression
testing threads, This is somewhat antithetical 10 Ihe usc of operational profiles. As
a rule, “good?” regression testing threads will have low operational frequencies,
and progression teng threads will have high operational frequencies.
References
‘Topper, Andrew et al, Sructurod Methode Merging Model, Tecbmigues, and CASE
Gran, New York, 1994
Jormensen, Paul C. An operation’) common denominator to the structured realtime
methods, Proceedings ofthe FD Siructured Tscbniques Association (74:9) Con-
rence, Chicago, May 11, 1989.
Jorgensen, Pau C, System testing with pseudostractutes, Amer. Programmer Vol. 7, No.
44 pp. 29-34, Apt 1994
Exercises
1. One of the problems of system testing, particularly interactive systems, is
to anticipate all the strange things the user might do. What happens in the
SATA system if a customer enters three digits of a PIN and then leaves?
|
System Tesung Pen
2. To remain ‘in control” ofabnormal user behavior (the behavior is ahnormal,
not the usen, the SATM system might introduce a timer with a 30-sceond
üme-out. When no por input event occurs for 30 seconds, the SATM
system asks ifthe user needs more time, The user can answer yes or no.
Devise new screen and deny pon event hat would implemen sich
a time-out event.
3. Suppose you add this time-out feature to the SATM system. What regression
testing would you perform?
4. Make an additions refinement to the PIN Try finite state machine (Fi.
re 14.6) to implement your time-out mechanism, then revise the thread
test case in Table 143.
5. The text asser that “the BI function button aces in five separate contexts
screens displayed) and has three different meanings.” Examine the 15
screens (points of event quiescence) and decide whether a BI keystroke
has three or five diferent logical meanings.
6. Does it make sense to use test covernge mecs in conjunction with
operational profiles? Discuss this,
14.7.3 Progression vs, Regression Testing
‘When we discussed software development life cycles in Chapter 12, we mentioned
that the use of buids induces the need for regression testing. When build 2 i
added to build 1, we test the new material in build 2, and we also retest build
110 see that he new material has no deleterious effect on build 1 contents. The
industrial average for such “ripple effect” is that 20% of changes 10 an existing
system induce new faults inthe system.) If projet has several builds, regression
testing implics a significant repettion of testing, especially for the early bulk,
‘We can reduce this by concentrating on the difference between progression and
regression testing
‘The most common approach to regression testing is 10 simply repeat the system
tests, We can refine this (and drastically reduce the effor) by choosing test read
wi respect to the goals of regression and progression testing. With progression
testing, we are testing new territory, so we expecta higher failure rate than with
regression testing, Another diference is that because we expect to find more
faults with progression testing, we need to be able to locate the faults. This
requires test cases with a diagnostic cpabiliy — that I, tests that can fal only
in a few ways. For thread-based testing, progression testing should use shorter
heads that can fal only in a few ways. These threads might he ordered as we
did with the SATM thread test set, such that longer thread are bul up from
shorter and previously tested) threads.
‘We have lower expectations of failure with regression esting, and we are less
‘concerned with fal isolation, Taken together, his means regression testing should _
use longer threads, ones that can fil in several ways, If we think in terms of
‘cavecige, both progression and regression testing Will have thorough coverage,
bat the density is diferent, Sate and tension coverage matices like Tables 148
and 149) will be sparse for progression testing threads and dense for regression
testing threads, This is somewhat antithetical 10 Ihe usc of operational profiles. As
a rule, “good?” regression testing threads will have low operational frequencies,
and progression teng threads will have high operational frequencies.
References
‘Topper, Andrew et al, Sructurod Methode Merging Model, Tecbmigues, and CASE
Gran, New York, 1994
Jormensen, Paul C. An operation’) common denominator to the structured realtime
methods, Proceedings ofthe FD Siructured Tscbniques Association (74:9) Con-
rence, Chicago, May 11, 1989.
Jorgensen, Pau C, System testing with pseudostractutes, Amer. Programmer Vol. 7, No.
44 pp. 29-34, Apt 1994
Exercises
1. One of the problems of system testing, particularly interactive systems, is
to anticipate all the strange things the user might do. What happens in the
SATA system if a customer enters three digits of a PIN and then leaves?
|
System Tesung Pen
2. To remain ‘in control” ofabnormal user behavior (the behavior is ahnormal,
not the usen, the SATM system might introduce a timer with a 30-sceond
üme-out. When no por input event occurs for 30 seconds, the SATM
system asks ifthe user needs more time, The user can answer yes or no.
Devise new screen and deny pon event hat would implemen sich
a time-out event.
3. Suppose you add this time-out feature to the SATM system. What regression
testing would you perform?
4. Make an additions refinement to the PIN Try finite state machine (Fi.
re 14.6) to implement your time-out mechanism, then revise the thread
test case in Table 143.
5. The text asser that “the BI function button aces in five separate contexts
screens displayed) and has three different meanings.” Examine the 15
screens (points of event quiescence) and decide whether a BI keystroke
has three or five diferent logical meanings.
6. Does it make sense to use test covernge mecs in conjunction with
operational profiles? Discuss this,
Chapter 15
o EEE
Interaction Testing
—Z
Paults and failures due to interaction are the bane of testers. Their subtleties make
tem difhcuk o recognize and even more difficult to reveal by testing. These are
deep faults, ones that remain in a system even after extensive thread testing
Unfortunately, favls of interaction most frequently occur as failures in delivered
systems that have been in use for some time, Typically, they have à very low
probability of execution, and they occur only alter a large number of threads
have been.executed. Mostof 1his chapter is devoted to, descabing forms of —
interaction, not to testing them. As such, k is really more concerned with require-
ments specification than with testing, The connection is impurtant. knowing how
10 specify interactions is the fist step in detecting and testing for them. This
chapter is also a somewhat philosophical and mildly mathematical discussion of
faults and falures of interaction; we cannot hope to test something if we do not
understand it. We begin with an important addition 10 our five basic constructs
and use this to develop a taxonomy of types of interaction, Next we develop a
simple extension to conventional Petri nets that reflects the basic consiatets, and
then we illustrate the whole discussion with the simple automated teller machine
(SATM and Saturn windshield wiper systems, and sometimes with examples from
telephone systems. We conclude by applying the taxonomy to an important
application type: client-server systems,
15.1 Context of Interaction
Part of the difficulty of specifying and testing interactions is that they are so
‘common. Think of all the things hat interact in everyday life: people, automobile
drivers, regulations, chemical compounds, and abstractions, 10 name just a few.
‘We are concerned with interactions in software-contolied systems (particularly
the unexpected ones), so we start by restricting our discussion to interactions.
among our basie system constructs: actions, data, events, pons, and threads,
‘One way to establish a contest for interaction is 10 view it as a relationship
among the five constructs. If we did this, we would find that the relation
261
o EEE
Interaction Testing
—Z
Paults and failures due to interaction are the bane of testers. Their subtleties make
tem difhcuk o recognize and even more difficult to reveal by testing. These are
deep faults, ones that remain in a system even after extensive thread testing
Unfortunately, favls of interaction most frequently occur as failures in delivered
systems that have been in use for some time, Typically, they have à very low
probability of execution, and they occur only alter a large number of threads
have been.executed. Mostof 1his chapter is devoted to, descabing forms of —
interaction, not to testing them. As such, k is really more concerned with require-
ments specification than with testing, The connection is impurtant. knowing how
10 specify interactions is the fist step in detecting and testing for them. This
chapter is also a somewhat philosophical and mildly mathematical discussion of
faults and falures of interaction; we cannot hope to test something if we do not
understand it. We begin with an important addition 10 our five basic constructs
and use this to develop a taxonomy of types of interaction, Next we develop a
simple extension to conventional Petri nets that reflects the basic consiatets, and
then we illustrate the whole discussion with the simple automated teller machine
(SATM and Saturn windshield wiper systems, and sometimes with examples from
telephone systems. We conclude by applying the taxonomy to an important
application type: client-server systems,
15.1 Context of Interaction
Part of the difficulty of specifying and testing interactions is that they are so
‘common. Think of all the things hat interact in everyday life: people, automobile
drivers, regulations, chemical compounds, and abstractions, 10 name just a few.
‘We are concerned with interactions in software-contolied systems (particularly
the unexpected ones), so we start by restricting our discussion to interactions.
among our basie system constructs: actions, data, events, pons, and threads,
‘One way to establish a contest for interaction is 10 view it as a relationship
among the five constructs. If we did this, we would find that the relation
261
262 Solar Testing: Cnam Apo, Second Editon
InteracisWab is a reflexive relationship on each entity (data interact with data
actions with other actions, and so on). I also is binary relationship between,
data and events, data and threads, and events and threads The data modeling,
approach is not a dead end. however. Whenever a data model contains such
pervasive relationships, that is a clue that an important entity Es missing. IE we
add some tangible realty to our fairy abstract constructs, we get a more useful
framework for our study of interstion. The missing element is location, and
location has two components: time and position. Data modeling provides another
‘choice: we can treat location a5 a sixth basic entity, or as an atibute of the other
five, We choose the attdbute approach here.
‘What does it mean for location (time and position) to be an atribute of any
of the five basic construct? This is really a shortcoming of neatly al requirements,
specification notations, and techniques. (This is probably also the reason that
interactions are seldom recognized and tested.) Information about location is
usually created when a system is implemented, Sometimes location is mandated
as a requirement — when this happens, the requirement is actully a forced
implementation choice. We frst clafy the meaning of the components of location:
time and position,
‘We can take nwo views of time: as an Instant oras a duration. The instantaneous
view lets us describe when something happens — itis à point when time is an
axis, The duration view is an interval on the time axis. When we think about
duratons, we usually are interested in the length of the time interval, not the
endpoints (he star and finish times). Both views are useful. Because threads
‘execute, they have a duration; they also have points in time when they execute,
Similar observations apply to events, Often, events have very short durations, and
this is problematic if the duration is so shon that the event is mot recognized by
the system.
The position aspect is easier. We could take a very tangible, physical view of
position and describe i in terms of some coordinate system. Position can be a
three-<imensional Cartesian coordinate system with respect 10 some origin, or it
could be a Jongitude-htitude-elevation geographic point. For most systems, itis
more helpful o slightly abstract position into processor residence. Taken together,
‘ime and position tel the tester when and where something happens, and this is
essential 19 understanding interactions.
Before we develop our taxonomy, we need some ground rules about threads
and processors. For now, a processor is something that executes threads of a
device where events occur.
1. Because threads execute, they have a stricly positive time duration. We
usually speak of the execurion time of a thread, but we might also be
interested in when a thread occurs executes). Actions are degenerate cases
of threads, therefore, actions also have durations
2. In a single processor, rwo threads cannot execute simultaneously. This
resembles a fundamental precept of physics: no two bodies may occupy
the same space at the same time. Sometimes threads appear 10 be simul:
taneous, as in time-sharing on a single processor, in fac, time shared
threads are interleaved. Even though threads cannot execute simultaneously
eration Testing 263
> ne
Figure 15.1 Overlapping events,
on single proceso, evens canbe sima. Chis à cel
lematic for testers) o mee
3. vent hive «sly postive tine ain. When we consider event to
beacons hat execute on po devices hs ees tothe at pound ae
4, Two or mor) input event can oc imanes ea car
ecu imac in vo (or moro proceso, Ts Is nua
lear we conker pon des tobe sepa proces
5. Ina singe proceso, wo pa cren cannot begin imalancouy. This
ise cet consequence of output evens beng cused by ead one
We need bea she nata and ration views of nf ally cana
this ground ue Soppoe two ouput evens ae ch Un the aioe ot
one 1 much eter thn the daran ofthe acer, Te duros say
‘reap (cas hy orar on septate deve) bbe sar te ae
be identical. a shown in Pure 15. An example of ths occ ire
SAIM sytem, when teal cues een 1% be daplaye an en
ejes ve ATH ad. The serenely when he ce et ere
occur, (Mismay bea Recto; we could ao ay aT pe Ka
tre separate processor, and tat pon cut evens a alba mu
inrpocesor communion)
À read canon span more than one procesos Th convention help in
the defintion of tends By conning à dead 1 3 sage proces we
«cate a natal endpoi o mado is alo eu In mere cas
intend of ewer complex mes Ina mulirocoira ting le eke
alo real in anche fam of quiescence => pu Auen
‘Taken together, these six ground rules force what we might call “sane behavior”
nto the interactions in the taxonomy we define in Section 15.2
15.2 A Taxonomy of interactions
‘The two aspects of location, time and position, form the slanting point of a useful
taxonomy of interaction, Certain interactions are completely independent of time;
for example, two data items that interact exhibit their interaction regardless of
time, Certain time-dependent interactions also occur, such as when something is
2 prerequisite for something else. We will refer to time-independent interactions
as static and time-dependent interactions as dynamic. We can refine the statie/
dynamic dichotomy with the distinction between single and multiple processors
‘These two considerations yield a two-dimensional plane (as shown in Figure 13.2)
with four basic types of interactions
InteracisWab is a reflexive relationship on each entity (data interact with data
actions with other actions, and so on). I also is binary relationship between,
data and events, data and threads, and events and threads The data modeling,
approach is not a dead end. however. Whenever a data model contains such
pervasive relationships, that is a clue that an important entity Es missing. IE we
add some tangible realty to our fairy abstract constructs, we get a more useful
framework for our study of interstion. The missing element is location, and
location has two components: time and position. Data modeling provides another
‘choice: we can treat location a5 a sixth basic entity, or as an atibute of the other
five, We choose the attdbute approach here.
‘What does it mean for location (time and position) to be an atribute of any
of the five basic construct? This is really a shortcoming of neatly al requirements,
specification notations, and techniques. (This is probably also the reason that
interactions are seldom recognized and tested.) Information about location is
usually created when a system is implemented, Sometimes location is mandated
as a requirement — when this happens, the requirement is actully a forced
implementation choice. We frst clafy the meaning of the components of location:
time and position,
‘We can take nwo views of time: as an Instant oras a duration. The instantaneous
view lets us describe when something happens — itis à point when time is an
axis, The duration view is an interval on the time axis. When we think about
duratons, we usually are interested in the length of the time interval, not the
endpoints (he star and finish times). Both views are useful. Because threads
‘execute, they have a duration; they also have points in time when they execute,
Similar observations apply to events, Often, events have very short durations, and
this is problematic if the duration is so shon that the event is mot recognized by
the system.
The position aspect is easier. We could take a very tangible, physical view of
position and describe i in terms of some coordinate system. Position can be a
three-<imensional Cartesian coordinate system with respect 10 some origin, or it
could be a Jongitude-htitude-elevation geographic point. For most systems, itis
more helpful o slightly abstract position into processor residence. Taken together,
‘ime and position tel the tester when and where something happens, and this is
essential 19 understanding interactions.
Before we develop our taxonomy, we need some ground rules about threads
and processors. For now, a processor is something that executes threads of a
device where events occur.
1. Because threads execute, they have a stricly positive time duration. We
usually speak of the execurion time of a thread, but we might also be
interested in when a thread occurs executes). Actions are degenerate cases
of threads, therefore, actions also have durations
2. In a single processor, rwo threads cannot execute simultaneously. This
resembles a fundamental precept of physics: no two bodies may occupy
the same space at the same time. Sometimes threads appear 10 be simul:
taneous, as in time-sharing on a single processor, in fac, time shared
threads are interleaved. Even though threads cannot execute simultaneously
eration Testing 263
> ne
Figure 15.1 Overlapping events,
on single proceso, evens canbe sima. Chis à cel
lematic for testers) o mee
3. vent hive «sly postive tine ain. When we consider event to
beacons hat execute on po devices hs ees tothe at pound ae
4, Two or mor) input event can oc imanes ea car
ecu imac in vo (or moro proceso, Ts Is nua
lear we conker pon des tobe sepa proces
5. Ina singe proceso, wo pa cren cannot begin imalancouy. This
ise cet consequence of output evens beng cused by ead one
We need bea she nata and ration views of nf ally cana
this ground ue Soppoe two ouput evens ae ch Un the aioe ot
one 1 much eter thn the daran ofthe acer, Te duros say
‘reap (cas hy orar on septate deve) bbe sar te ae
be identical. a shown in Pure 15. An example of ths occ ire
SAIM sytem, when teal cues een 1% be daplaye an en
ejes ve ATH ad. The serenely when he ce et ere
occur, (Mismay bea Recto; we could ao ay aT pe Ka
tre separate processor, and tat pon cut evens a alba mu
inrpocesor communion)
À read canon span more than one procesos Th convention help in
the defintion of tends By conning à dead 1 3 sage proces we
«cate a natal endpoi o mado is alo eu In mere cas
intend of ewer complex mes Ina mulirocoira ting le eke
alo real in anche fam of quiescence => pu Auen
‘Taken together, these six ground rules force what we might call “sane behavior”
nto the interactions in the taxonomy we define in Section 15.2
15.2 A Taxonomy of interactions
‘The two aspects of location, time and position, form the slanting point of a useful
taxonomy of interaction, Certain interactions are completely independent of time;
for example, two data items that interact exhibit their interaction regardless of
time, Certain time-dependent interactions also occur, such as when something is
2 prerequisite for something else. We will refer to time-independent interactions
as static and time-dependent interactions as dynamic. We can refine the statie/
dynamic dichotomy with the distinction between single and multiple processors
‘These two considerations yield a two-dimensional plane (as shown in Figure 13.2)
with four basic types of interactions
264 Software Tsing: A Cralsan' Approach, Second Elton
Figure 15.2 Types of interactions
interactions in a single processor
Static interactions in multiple processors
Dynamic interactions in a single processor
Dynamic interactions in multiple processors
We next refine these basic four types using the notion of duration, Threads
and events have durations (because they execute), thus, they cannot be static
Data. on the other hand, is static, hut we need to be careful here, Consicer two
‘examples, the triangle ype that corresponds o the wiplet (5, 5, 5 of sides, and
the balance of a checking account, The triangle type à always equilateral — time
‘wlll never change geonievy, bur the balance of a bank account is [kei 10 change
in time. IF it does, the change is due to the execution of some thread, and this
Will be a key consideration,
15.2.1 Static Interactions in a Single Processor
Of the five basic construcs, only rw have no duration — ports and daa, Poms
are physical devices, therefore, we can view them as separate processors and
thereby simplily our discussion. Port devices interact in physical ways, such as
space and power consumption, but this is usually not important to testers, Da
items interact in logical ways (as opposed to physical), and these are important
to testers, In an informal way, we often speak of corrupt data and of maintaining
the integrity of a database, We sometimes get a bit more precise and speak of
incompatible or even inconsistent data. We can be very speciic if we borrow
some terms from Aristotle, (We finally have a chance 10 use the propositional
logic discussed in Chapter 3) In the following definitions, let p and q be
propositions about dara items. As examples, we might take p and q to be:
pr AccountBalance = $10.00
4 Sales < $1800.00
Definition
Propositions p and q are:
InteretionTosiog 265
Contrares if they cannot beth be true
Sub-contraries if they cannot both be fase
Contradicories if exactly one is true
45 a subaltern of p ifthe ruth of p guarantees the truth of q
‘These relationships are known o logicians asthe “square of opposition,” which
is shown in Figure 15.3, where p, 4, , and s are all propositions
Aristotelian logic seems arcane for software testers, but here are some situ
that are exactly characterized by data interactions in the square of opposition;
When the precondition for a thread is 2 conjunction of dats propositions,
contrary or contradicory data values will prevent thread execution,
Contextsensitive port input events usually involve contradiciory (or at
least, contrary) dat,
Case statement clauses are contraditores
Rules in a decision table are conteadictoies,
Static interactions in a single processor are exacıly amalogous to combinatorial
circuits; hey are also well represented by decision tables and unmarked event-
driven Pein ness (EDPNS). Features in telephone systems are good examples of
interaction (Zave, 1993). One example isthe logical conflit between calling pany
identification service and unlisted directory numbers. With calling pany identif-
cation, the directory number of the source of a telephone call is provided to the
called pasty. A-canflc occurs. when a party with an unlisted. direciory-nuunber—
makes a call ta party with calling pany identification. Which takes precedence
— the calling pars desire for privacy or the called party's right to know who
is placing an incoming call These two features are contrrics they cannot both
be satisfied, but they could both be waived. Call waiting service and data line
conditioning, comprise another example of contrary features. When à business (or
home computing enthusiast) pays for a specially conditioned deta line, calls on
that line are frequently used for the transmission of formatted binary data, If such
a line also has call waiting service, if call is made to the line chat is already in
use, a call waiting tone is superimposed onto the preexisting connection. If the
connection had been transmitting dat, the transmission would be corrupted by
the call waiting tone. In this case, the resoluion is easier. The customer disables
the call siting service before making data transmission call,
Da |
DS € |
sutatemation | coutadieres | caemaion
N
bci
Figure 15.3 The square of opposit
Figure 15.2 Types of interactions
interactions in a single processor
Static interactions in multiple processors
Dynamic interactions in a single processor
Dynamic interactions in multiple processors
We next refine these basic four types using the notion of duration, Threads
and events have durations (because they execute), thus, they cannot be static
Data. on the other hand, is static, hut we need to be careful here, Consicer two
‘examples, the triangle ype that corresponds o the wiplet (5, 5, 5 of sides, and
the balance of a checking account, The triangle type à always equilateral — time
‘wlll never change geonievy, bur the balance of a bank account is [kei 10 change
in time. IF it does, the change is due to the execution of some thread, and this
Will be a key consideration,
15.2.1 Static Interactions in a Single Processor
Of the five basic construcs, only rw have no duration — ports and daa, Poms
are physical devices, therefore, we can view them as separate processors and
thereby simplily our discussion. Port devices interact in physical ways, such as
space and power consumption, but this is usually not important to testers, Da
items interact in logical ways (as opposed to physical), and these are important
to testers, In an informal way, we often speak of corrupt data and of maintaining
the integrity of a database, We sometimes get a bit more precise and speak of
incompatible or even inconsistent data. We can be very speciic if we borrow
some terms from Aristotle, (We finally have a chance 10 use the propositional
logic discussed in Chapter 3) In the following definitions, let p and q be
propositions about dara items. As examples, we might take p and q to be:
pr AccountBalance = $10.00
4 Sales < $1800.00
Definition
Propositions p and q are:
InteretionTosiog 265
Contrares if they cannot beth be true
Sub-contraries if they cannot both be fase
Contradicories if exactly one is true
45 a subaltern of p ifthe ruth of p guarantees the truth of q
‘These relationships are known o logicians asthe “square of opposition,” which
is shown in Figure 15.3, where p, 4, , and s are all propositions
Aristotelian logic seems arcane for software testers, but here are some situ
that are exactly characterized by data interactions in the square of opposition;
When the precondition for a thread is 2 conjunction of dats propositions,
contrary or contradicory data values will prevent thread execution,
Contextsensitive port input events usually involve contradiciory (or at
least, contrary) dat,
Case statement clauses are contraditores
Rules in a decision table are conteadictoies,
Static interactions in a single processor are exacıly amalogous to combinatorial
circuits; hey are also well represented by decision tables and unmarked event-
driven Pein ness (EDPNS). Features in telephone systems are good examples of
interaction (Zave, 1993). One example isthe logical conflit between calling pany
identification service and unlisted directory numbers. With calling pany identif-
cation, the directory number of the source of a telephone call is provided to the
called pasty. A-canflc occurs. when a party with an unlisted. direciory-nuunber—
makes a call ta party with calling pany identification. Which takes precedence
— the calling pars desire for privacy or the called party's right to know who
is placing an incoming call These two features are contrrics they cannot both
be satisfied, but they could both be waived. Call waiting service and data line
conditioning, comprise another example of contrary features. When à business (or
home computing enthusiast) pays for a specially conditioned deta line, calls on
that line are frequently used for the transmission of formatted binary data, If such
a line also has call waiting service, if call is made to the line chat is already in
use, a call waiting tone is superimposed onto the preexisting connection. If the
connection had been transmitting dat, the transmission would be corrupted by
the call waiting tone. In this case, the resoluion is easier. The customer disables
the call siting service before making data transmission call,
Da |
DS € |
sutatemation | coutadieres | caemaion
N
bci
Figure 15.3 The square of opposit
266 Sofware Tsing: A Crafsman' Approach, Second Edtion
15.2.2 Static Interactions in Multiple Processors
‘The location of data helps resolve the contrarios in the telephone system examples.
‘We would expect that the data for call waiting and data line conditioning are
located in the same processor because both refer to the same subscriber line.
‘Thus, the software that controls calls for that line could check for contrary line
data. This is an unreasonable expectation for the calling party identification
problem, however. Suppose the calling party is a line in an office remote from
the office that serves the line with calling party identification. Because these data
are in separate locations (processor), nether knows about the other, so thelr
contrary nature can only be detected when they are connected hy a thread. To
be very precise, we can say that the contcuy relationship exists as a stale
interaction across multiple processors, and it becomes a failure when executing
reads in the two telephone offices (processors) interact.
Call forwarding provides a beter example of a stati, distributed interaction,
Suppose we have three telephone subscribers in separate cities:
Subscriber A is in Grand Rapids, Michigan
Subscriber B is in Phoenix, Arizona
Subscriber € is in Baltimore, Maryland.
MA”
cath me ande s Kalos: cle À ae made o Ds Cl B are
ferme 0G and els 18 © re Kae 1
Th el orvarding dt ety hy amor al ete CA racing
dau fecal he lps fet pode ba al by Y lead
tens peter defines a new fern dera, Ths means a one of
the offers knows ofa faring ds inthe oer ace weve dab
conte, Tic fa, bu doe ne pane 3 Ar unl someone (ter
thin A B, oF ©) placas eal tomy pone ds all loans oop ch à
cal say o are, gen cl orang texan fal pon
ofc, which scan cata Cs decry mbt Me gente ater en
in phone oft, and soon Yor not, ple note date ent of the
Cnnecing read moves u oto the Sie suns and to nan er
ions Te pent aie sl ext ss flere! pan of oor tenon.
The Dem line that sti irc are eisen the same, wheter
they we came ito 4 ange proce x die among mile pros
sons hey ze handen 1 dete shen ey are doute, However) Ander
Common fc of ae een occas wah we rear tad fon
Gevendencis ins base Comun dt Bath of tee redone
lino dl beens
15.2.3 Dynamic Interactions in a Single Processor
Moving 10 the dynamic quadrants means we consider the implications of time for
imeractions. Among other things, this means we must expand from the date-only
interactions to interactions among data, events, and threads. We also must shift
from the strictly declarative relationships in the square of opposition to a more
imperative view. The notion of n-connectedness in a directed graph (see Chapter
Te AY
> À
Figure 15.4. Forms of n-connectedness.
1 serves perfect. Figure 154 shows the four forms of m-connectedness in a
rected gph
Even the ditidata interactions exhibit forms of n-<omnecteduess. Data that
ase logialy independent are Orconnected, and subalterates are 2connectee,
The other three relationships, contare, contadicones, and sub commen al
pertain to 3-connceted dat, ch ofthese à hidtectional relationship.
SÍ potential pairs of cóncepis can interact: datacdata, dataserons,
dlatatbrends, events-evens, events leads, and treas-threds Bach of these
is father qualified by four degrees of mconnectedness, resul in 24 elemen
lo our taxonony for this quadrant, Take some time to thing through these
interactions, Here are faur examples
connected data with data: occurs hen two or more data items are inputs
to the same action
2-connected data with dats: occurs when a dat Item is used in a compu-
tation (as in datslow testing)
3-connected data with data: occurs when data are deeply related, as in
‘repetition and semaphores
‘connected data with events: contextsenstive port input events
We do not need to analyze al 24 possibilities because faults of interaction only
become failures when threads establish some connection, The fulis are latent,
and when a thread makes a connection, he latent fault becomes a Failure. Thread
«an only interact in two ways, via events or via dita, We will sce this more clearly
using EDPNS after we make another definition.
Definition
In an EDPN, the external inputs are the places with indegice = 0, and the external
‘outputs are dhe places with outdegree = 0.
15.2.2 Static Interactions in Multiple Processors
‘The location of data helps resolve the contrarios in the telephone system examples.
‘We would expect that the data for call waiting and data line conditioning are
located in the same processor because both refer to the same subscriber line.
‘Thus, the software that controls calls for that line could check for contrary line
data. This is an unreasonable expectation for the calling party identification
problem, however. Suppose the calling party is a line in an office remote from
the office that serves the line with calling party identification. Because these data
are in separate locations (processor), nether knows about the other, so thelr
contrary nature can only be detected when they are connected hy a thread. To
be very precise, we can say that the contcuy relationship exists as a stale
interaction across multiple processors, and it becomes a failure when executing
reads in the two telephone offices (processors) interact.
Call forwarding provides a beter example of a stati, distributed interaction,
Suppose we have three telephone subscribers in separate cities:
Subscriber A is in Grand Rapids, Michigan
Subscriber B is in Phoenix, Arizona
Subscriber € is in Baltimore, Maryland.
MA”
cath me ande s Kalos: cle À ae made o Ds Cl B are
ferme 0G and els 18 © re Kae 1
Th el orvarding dt ety hy amor al ete CA racing
dau fecal he lps fet pode ba al by Y lead
tens peter defines a new fern dera, Ths means a one of
the offers knows ofa faring ds inthe oer ace weve dab
conte, Tic fa, bu doe ne pane 3 Ar unl someone (ter
thin A B, oF ©) placas eal tomy pone ds all loans oop ch à
cal say o are, gen cl orang texan fal pon
ofc, which scan cata Cs decry mbt Me gente ater en
in phone oft, and soon Yor not, ple note date ent of the
Cnnecing read moves u oto the Sie suns and to nan er
ions Te pent aie sl ext ss flere! pan of oor tenon.
The Dem line that sti irc are eisen the same, wheter
they we came ito 4 ange proce x die among mile pros
sons hey ze handen 1 dete shen ey are doute, However) Ander
Common fc of ae een occas wah we rear tad fon
Gevendencis ins base Comun dt Bath of tee redone
lino dl beens
15.2.3 Dynamic Interactions in a Single Processor
Moving 10 the dynamic quadrants means we consider the implications of time for
imeractions. Among other things, this means we must expand from the date-only
interactions to interactions among data, events, and threads. We also must shift
from the strictly declarative relationships in the square of opposition to a more
imperative view. The notion of n-connectedness in a directed graph (see Chapter
Te AY
> À
Figure 15.4. Forms of n-connectedness.
1 serves perfect. Figure 154 shows the four forms of m-connectedness in a
rected gph
Even the ditidata interactions exhibit forms of n-<omnecteduess. Data that
ase logialy independent are Orconnected, and subalterates are 2connectee,
The other three relationships, contare, contadicones, and sub commen al
pertain to 3-connceted dat, ch ofthese à hidtectional relationship.
SÍ potential pairs of cóncepis can interact: datacdata, dataserons,
dlatatbrends, events-evens, events leads, and treas-threds Bach of these
is father qualified by four degrees of mconnectedness, resul in 24 elemen
lo our taxonony for this quadrant, Take some time to thing through these
interactions, Here are faur examples
connected data with data: occurs hen two or more data items are inputs
to the same action
2-connected data with dats: occurs when a dat Item is used in a compu-
tation (as in datslow testing)
3-connected data with data: occurs when data are deeply related, as in
‘repetition and semaphores
‘connected data with events: contextsenstive port input events
We do not need to analyze al 24 possibilities because faults of interaction only
become failures when threads establish some connection, The fulis are latent,
and when a thread makes a connection, he latent fault becomes a Failure. Thread
«an only interact in two ways, via events or via dita, We will sce this more clearly
using EDPNS after we make another definition.
Definition
In an EDPN, the external inputs are the places with indegice = 0, and the external
‘outputs are dhe places with outdegree = 0.
268
jure 15.5. External inputs and outputs In an EDPN.
La the EDPN in Figure 155, pl and p2 are the only external inputs, and pS,
9, and pt0 are the only external outputs. Although nor shown here, data places.
that are preconditions and postconditions ate external inputs and outputs, respec
Lively. The best description is that the indegrecs of external inputs and the
outdegrces of external output are always 0.
INOW we are at a key point: we can represent the interaction among threads
hy the composition of their EDPNs. We do this as follows: each thread has its
‘own (unique) FDPN, and within euch FDPN, the places and transitions have
symbolic names. In one sense, these names are local to the dread; but in a larger
sense (when they are composed), local mames must be resolved into global
Synonyms. Suppose, for example, that in one SATM thread, a keystroke on the
Cancel key is named as por ourput event p2, and in another thread the same
(event is called p6, When these wo threads are composed, the synonyms must
be collapsed! onto a single name. (Hares, we noted that was good practice 10
hysical names for port events instead of thei logical names. synonym
is the reason for this recommendation.) Once synonyns are resolved,
idual threads are drawn as EDPNS. Because threads interact only with
respect to their external inputs and outputs, we next ently the sets of external
inputs and ouputs of each thread. The intersection of these sets contains the
events and places at which the composed Ihreads can interact. This process is
illusated in Figures 15.6 and 157.
Thread 1 is the sequence <1, 52>, and thread 2 is the sequence <s3> The
extemal inputs of these threads are the sets Ell = tpl, dll and E12 = tpl, 62,
and the external ourputs of these threads are the sets HOI = {p3, dil and LO2 =
Ip3. ddl intersecting these, we get the sets FI = EIN A BIZ = {pli and EO = FOI
A BO2 = Ip3, dl. The sets El and EO contain the external inputs and outputs a
Interaction Testing 269
Figure 15.6 Two EDPN threads.
‘which threads 1 the 2 may possibly interuct. Notice that the external inputs and
‘outputs ofthe composed threads are the sets PUI U 12 and LOI U EO2. External
events will always be preserved as external, but external data may not. I is
possible for output data of une thread to be input data to another thread. (This.
happens in the cal forwarding loop.) a
‘We can formalize this process into another definition. Let TI and T2 be two
EDEN threads in which synonym places have been resolved, and with external
input and output sets FIL, 12, BOY, and FO2, Furthermore, lt be the compo-
sition of hreads TI and T2, where EI = EIN 1 FIZ and RO = FOI © FO2 are the
external input and output sets of the composed thread 1
Figure 157. Composition of EDPN threads
jure 15.5. External inputs and outputs In an EDPN.
La the EDPN in Figure 155, pl and p2 are the only external inputs, and pS,
9, and pt0 are the only external outputs. Although nor shown here, data places.
that are preconditions and postconditions ate external inputs and outputs, respec
Lively. The best description is that the indegrecs of external inputs and the
outdegrces of external output are always 0.
INOW we are at a key point: we can represent the interaction among threads
hy the composition of their EDPNs. We do this as follows: each thread has its
‘own (unique) FDPN, and within euch FDPN, the places and transitions have
symbolic names. In one sense, these names are local to the dread; but in a larger
sense (when they are composed), local mames must be resolved into global
Synonyms. Suppose, for example, that in one SATM thread, a keystroke on the
Cancel key is named as por ourput event p2, and in another thread the same
(event is called p6, When these wo threads are composed, the synonyms must
be collapsed! onto a single name. (Hares, we noted that was good practice 10
hysical names for port events instead of thei logical names. synonym
is the reason for this recommendation.) Once synonyns are resolved,
idual threads are drawn as EDPNS. Because threads interact only with
respect to their external inputs and outputs, we next ently the sets of external
inputs and ouputs of each thread. The intersection of these sets contains the
events and places at which the composed Ihreads can interact. This process is
illusated in Figures 15.6 and 157.
Thread 1 is the sequence <1, 52>, and thread 2 is the sequence <s3> The
extemal inputs of these threads are the sets Ell = tpl, dll and E12 = tpl, 62,
and the external ourputs of these threads are the sets HOI = {p3, dil and LO2 =
Ip3. ddl intersecting these, we get the sets FI = EIN A BIZ = {pli and EO = FOI
A BO2 = Ip3, dl. The sets El and EO contain the external inputs and outputs a
Interaction Testing 269
Figure 15.6 Two EDPN threads.
‘which threads 1 the 2 may possibly interuct. Notice that the external inputs and
‘outputs ofthe composed threads are the sets PUI U 12 and LOI U EO2. External
events will always be preserved as external, but external data may not. I is
possible for output data of une thread to be input data to another thread. (This.
happens in the cal forwarding loop.) a
‘We can formalize this process into another definition. Let TI and T2 be two
EDEN threads in which synonym places have been resolved, and with external
input and output sets FIL, 12, BOY, and FO2, Furthermore, lt be the compo-
sition of hreads TI and T2, where EI = EIN 1 FIZ and RO = FOI © FO2 are the
external input and output sets of the composed thread 1
Figure 157. Composition of EDPN threads
270 Sotare Testing: A Catomans Approach Second Eten
Figure 15.6. n-connected threads,
Definition
‘The threads TI and T2 are:
O-zonmeered If EN à EL « 2, EO er BO2~B; Ell 202 = and FOL
DE =o
connected if ether EI + © or FO » 2
Zeonnected if either EII À EO2 + 8 or BIZ EOL 4 D
3-connected if both EN EO2 + 2 and E12. 4 EOL # ©
Compare this with the definition of n-connectedness in Chapter 4. There. we
were concerned with relationships among pairs of nodes; here we are concerned
wich a relationship among threads. We can eliminate these somewhat overlapping
definitions hy constructing a rather elaborate directed graph in which nodes arc
threads from which the extemal inputs and outputs have been deleted, and edges
‘connect threads according to the deleted external input and output places. Figure
158 shows such 2 graph for the composition in Figure 15.7. Notice how we sec
directly that the threads are L-connected via the input place pl and via the ousput
places p3 and di
‘The definition of n-connectedness exactly addresses the ways in which threads
can interact in the dynamic, single-processor quadrant. Only one thing is missing
just as with n-connectedness in an ordinary directed yeaph, the connectivity can.
be established by chains of threads instead of hy adjacent threads as we have
done here, (Imagine a cal forwarding loop that involves a dozen phone sub-
sorbers in as many states) In my personal texting experience, many of the
problems described in field trouble reports were unexpected instances of 1: 2
or 3connectedness among theeads that were thought to be O-connected.
‘The two forms of I-connectedness among threads deserve special comment.
When two threads ace connected via a common input (event or data), the
traction Testing an
interaction is the classical form of Petri net confit (In Figure 15.7, threads 3 and
ate 1-connected via the por input event pl; this iv also an example of a contente
sensitive port input event) If we imagine a marking in which the frst transitions
in each thread are both enabled, thea firing one of them consumes the token
From the common input place, and this disables the other. When two threads are
connected by an output place (again either an event or data), no conflict occur
but there isan interesting ambiguity, The common output place is marked when.
‘one of the threads executes, but we usually do not know which one, This can
happen in the SAIM system when screen 10 (temporarily unable to process
withdrawals) is displayed. It could be disphyed hecause the ATM has no more
cash or because there is a malfunction in the withdrawal door. This fore of
‘outputcaused ambiguity is very common in field trouble reports, Many times, it
is due 10 an unexpected interaction among threads, some of which may have
executed long before others,
Port events cannot be both inputs and ousputs, therefore, the only way threads
can be 2-connected is via dsta places, and since 3-connectcdness is bidirectional
2connectedness, threads can only be 3+-comected via data places. In a sense, this
simples interactions because the number of possiblities is reduced. The real
problem with interaction via data is that long intervals of time can be involved.
‘We can add a small refinement here: the difference berween read-only data and
reacewrite data, Read-only data never changes, so we could say it has an infinite
or indefinite) duration. Read-wte data obviously changes upon a wake, so ft has
a duration — the interval betweca successive vete, The real problem with this
is that scidom used (writen) data may be that causes two threads
\o be 2- oF Féüniected, and the lig tiie separation between the first and second
threads can be very difficult to diagnose and dilficut to cause with a test case
Several dynamic interactions occur in the SATM system. Fist, notice tat we
are dealing with one ATM terminal, so we are concerned with dynamic interactions
in a single processor. Technically two processors are used, because the SATM
system gets the account information from the ental bank; Bue we can treat this
interaction as though the bank is a pom device, Severil contextsensitive port
input events take place in the SATM system; because we discussed these ar length
in Chapter 14, we move on to 1+ and 2-connected interactions among threads.
Lis easy to devise threads that make deposits to and wihdruwals from a given
bank account, These are 1> and 2-connected via the data place for the account
balance. Withdrawal threads that atempt to withdraw more than, and less than,
the existing balance are I-connected 10 Ihe balance place. A successful withdrawal
Gin which the requested amount is ess than the balance) clearly changes the
balance, so the betore and after instances of the balance place are 2-connected
via the withdrawal thread. If we add a deposit thread, we obtain even richer
interactions, and the key 10 all of this isthe time order in which they execute.
Another subilety exists in the foregoing discussion that has to do with how we
describe such interacting threads. One possbiliy 8 to rely on specific values, both
for preconltions and for amounts entered in the input portion ofthe threads, With
the explicit approach, we might postulate, as a precondition, a balance of $50.00,
define thread TI as a withdrawal of $40.00; and define thread T2 as a withdrawal
‘of $60.00. Superficial, this is easy — thread T2 will not execute, and thread TI
will. We could be more precise by defining two port inputs for the séthdrawal
amount: pl is withdrawal amount = $4000, and p2 is withdeawal amount = $60.00.
Figure 15.6. n-connected threads,
Definition
‘The threads TI and T2 are:
O-zonmeered If EN à EL « 2, EO er BO2~B; Ell 202 = and FOL
DE =o
connected if ether EI + © or FO » 2
Zeonnected if either EII À EO2 + 8 or BIZ EOL 4 D
3-connected if both EN EO2 + 2 and E12. 4 EOL # ©
Compare this with the definition of n-connectedness in Chapter 4. There. we
were concerned with relationships among pairs of nodes; here we are concerned
wich a relationship among threads. We can eliminate these somewhat overlapping
definitions hy constructing a rather elaborate directed graph in which nodes arc
threads from which the extemal inputs and outputs have been deleted, and edges
‘connect threads according to the deleted external input and output places. Figure
158 shows such 2 graph for the composition in Figure 15.7. Notice how we sec
directly that the threads are L-connected via the input place pl and via the ousput
places p3 and di
‘The definition of n-connectedness exactly addresses the ways in which threads
can interact in the dynamic, single-processor quadrant. Only one thing is missing
just as with n-connectedness in an ordinary directed yeaph, the connectivity can.
be established by chains of threads instead of hy adjacent threads as we have
done here, (Imagine a cal forwarding loop that involves a dozen phone sub-
sorbers in as many states) In my personal texting experience, many of the
problems described in field trouble reports were unexpected instances of 1: 2
or 3connectedness among theeads that were thought to be O-connected.
‘The two forms of I-connectedness among threads deserve special comment.
When two threads ace connected via a common input (event or data), the
traction Testing an
interaction is the classical form of Petri net confit (In Figure 15.7, threads 3 and
ate 1-connected via the por input event pl; this iv also an example of a contente
sensitive port input event) If we imagine a marking in which the frst transitions
in each thread are both enabled, thea firing one of them consumes the token
From the common input place, and this disables the other. When two threads are
connected by an output place (again either an event or data), no conflict occur
but there isan interesting ambiguity, The common output place is marked when.
‘one of the threads executes, but we usually do not know which one, This can
happen in the SAIM system when screen 10 (temporarily unable to process
withdrawals) is displayed. It could be disphyed hecause the ATM has no more
cash or because there is a malfunction in the withdrawal door. This fore of
‘outputcaused ambiguity is very common in field trouble reports, Many times, it
is due 10 an unexpected interaction among threads, some of which may have
executed long before others,
Port events cannot be both inputs and ousputs, therefore, the only way threads
can be 2-connected is via dsta places, and since 3-connectcdness is bidirectional
2connectedness, threads can only be 3+-comected via data places. In a sense, this
simples interactions because the number of possiblities is reduced. The real
problem with interaction via data is that long intervals of time can be involved.
‘We can add a small refinement here: the difference berween read-only data and
reacewrite data, Read-only data never changes, so we could say it has an infinite
or indefinite) duration. Read-wte data obviously changes upon a wake, so ft has
a duration — the interval betweca successive vete, The real problem with this
is that scidom used (writen) data may be that causes two threads
\o be 2- oF Féüniected, and the lig tiie separation between the first and second
threads can be very difficult to diagnose and dilficut to cause with a test case
Several dynamic interactions occur in the SATM system. Fist, notice tat we
are dealing with one ATM terminal, so we are concerned with dynamic interactions
in a single processor. Technically two processors are used, because the SATM
system gets the account information from the ental bank; Bue we can treat this
interaction as though the bank is a pom device, Severil contextsensitive port
input events take place in the SATM system; because we discussed these ar length
in Chapter 14, we move on to 1+ and 2-connected interactions among threads.
Lis easy to devise threads that make deposits to and wihdruwals from a given
bank account, These are 1> and 2-connected via the data place for the account
balance. Withdrawal threads that atempt to withdraw more than, and less than,
the existing balance are I-connected 10 Ihe balance place. A successful withdrawal
Gin which the requested amount is ess than the balance) clearly changes the
balance, so the betore and after instances of the balance place are 2-connected
via the withdrawal thread. If we add a deposit thread, we obtain even richer
interactions, and the key 10 all of this isthe time order in which they execute.
Another subilety exists in the foregoing discussion that has to do with how we
describe such interacting threads. One possbiliy 8 to rely on specific values, both
for preconltions and for amounts entered in the input portion ofthe threads, With
the explicit approach, we might postulate, as a precondition, a balance of $50.00,
define thread TI as a withdrawal of $40.00; and define thread T2 as a withdrawal
‘of $60.00. Superficial, this is easy — thread T2 will not execute, and thread TI
will. We could be more precise by defining two port inputs for the séthdrawal
amount: pl is withdrawal amount = $4000, and p2 is withdeawal amount = $60.00.
m SolareTesing A Ceatsman’s Approach, Second kaon
Wh these, thread TI will contain a port event Lo display screen 11 (ake cash),
and T2 will contain a por event to display screen 8 (insufficient funds). Here is
where the subvley comes into play. Through greater precision, we also remove
the conflict because we have stated in advance which port output will occur. When
do these values exist: when the thread is described (specified) or when it actually
executes? The explicit appreach is commonly used by testers, but ft does not work
well for requirements specificuion because ofthe predisposition. Alternatively, we
‘could he more general and say that head Ti ds a withdrawal of an amount less
tan the balance, thread T2 isa withdrawal of an amount greater than te balance,
and not state the balance until execution time. CThis is a good choice when
fexecitable specifications are used as a rapid prototype.) The problem with the
later choice is that testes cannot provide the expected ouput portions of a test
case, The subi; hinges on when the path of a thread is determined: before i
begins to execute, as in a test case, oras I executes, as in u rapid prototype. We
wall revisa this discussion in Section 153 when we discuss the connections among,
interaction, composition, and determinism. For now, testers might relish the idea
at they can control the destiny of reads, but specifiers cannot.
15.2.4 Dynamic Interactions in Multiple Processors
Dynamic interactions among mule processors are the most complex in our
taxonomy, Because threads and events can execute simultaneously. strictly sequen:
deterministic behavior is replaced by concurrent behavior. In the words of
Robin. Milne, “concurrency inflcis-non-devesmsinisns™ (Milner, 1993). The added
‘complexity is also seen in the models mandated by each quadrant decision tables
‘suffice for static interactions, and Anite state machines express dynamic interactions
‘on a single processor. Dynamic interactions on multiple processors need the
expressive power of communicating finite stare machines or some form of Pets
nes. We will see these interactions in the Saturn windshickl wiper system
The windshield wiper on the Saturn automobile is controled by a lever und
a dal. The lever has four postions, OFF, INT (for intermittent), LOW, and HIGH:
“and the dial has three positions, numbered simply 1, 2, and 3, The dial positions
indicate thee intermitent speeds, and the dial position is relevant only when the
lever is at the INT posklon. The following decision table shows the windshield
viper speeds in wipes per minute) for the lever and dist positions.
Conditions
over O INT INT nm LOW
Dal m 1 2 3 m
wor 0 4 6 2 m
If we think about the Saturn windshield wiper system, we find three natural,
interacting devices: the leves, the dial, and the wiper. Two events occur on the
lever: it can be moved up one position or it can be moved down one position.
Similarly, two events occur on the dial: it can be moved clockwise or counter
‘clockwise, We can decompose these ino the finer granularity shown in Table 15.1
Interaction Testing 273
Table 15.1. Port Input Events in the Saturn
Windshield Wiper
Fo Input Event Description
a Lever from OFF 10 INT
ea Lever from INT 10 LOW
fe Lever from LOW to HIGH
‘et Lever from HIGH to LOW
5 Lover from LOW to INT
le6 Lever from INT 10 OFF
ie? Dilfrom 1102
ies Dielfrom2to3
ie) Dial from 3102
leo al from 2101
Table 15.2. Port Output Events
in the Saturn Windshield Wiper
Po Opa em Desp
sa Dupm
oot ‘opm
oc sup
oot apm
os mp
06 arm
Figure 15.9. Lever and dial finite state machines.
Wh these, thread TI will contain a port event Lo display screen 11 (ake cash),
and T2 will contain a por event to display screen 8 (insufficient funds). Here is
where the subvley comes into play. Through greater precision, we also remove
the conflict because we have stated in advance which port output will occur. When
do these values exist: when the thread is described (specified) or when it actually
executes? The explicit appreach is commonly used by testers, but ft does not work
well for requirements specificuion because ofthe predisposition. Alternatively, we
‘could he more general and say that head Ti ds a withdrawal of an amount less
tan the balance, thread T2 isa withdrawal of an amount greater than te balance,
and not state the balance until execution time. CThis is a good choice when
fexecitable specifications are used as a rapid prototype.) The problem with the
later choice is that testes cannot provide the expected ouput portions of a test
case, The subi; hinges on when the path of a thread is determined: before i
begins to execute, as in a test case, oras I executes, as in u rapid prototype. We
wall revisa this discussion in Section 153 when we discuss the connections among,
interaction, composition, and determinism. For now, testers might relish the idea
at they can control the destiny of reads, but specifiers cannot.
15.2.4 Dynamic Interactions in Multiple Processors
Dynamic interactions among mule processors are the most complex in our
taxonomy, Because threads and events can execute simultaneously. strictly sequen:
deterministic behavior is replaced by concurrent behavior. In the words of
Robin. Milne, “concurrency inflcis-non-devesmsinisns™ (Milner, 1993). The added
‘complexity is also seen in the models mandated by each quadrant decision tables
‘suffice for static interactions, and Anite state machines express dynamic interactions
‘on a single processor. Dynamic interactions on multiple processors need the
expressive power of communicating finite stare machines or some form of Pets
nes. We will see these interactions in the Saturn windshickl wiper system
The windshield wiper on the Saturn automobile is controled by a lever und
a dal. The lever has four postions, OFF, INT (for intermittent), LOW, and HIGH:
“and the dial has three positions, numbered simply 1, 2, and 3, The dial positions
indicate thee intermitent speeds, and the dial position is relevant only when the
lever is at the INT posklon. The following decision table shows the windshield
viper speeds in wipes per minute) for the lever and dist positions.
Conditions
over O INT INT nm LOW
Dal m 1 2 3 m
wor 0 4 6 2 m
If we think about the Saturn windshield wiper system, we find three natural,
interacting devices: the leves, the dial, and the wiper. Two events occur on the
lever: it can be moved up one position or it can be moved down one position.
Similarly, two events occur on the dial: it can be moved clockwise or counter
‘clockwise, We can decompose these ino the finer granularity shown in Table 15.1
Interaction Testing 273
Table 15.1. Port Input Events in the Saturn
Windshield Wiper
Fo Input Event Description
a Lever from OFF 10 INT
ea Lever from INT 10 LOW
fe Lever from LOW to HIGH
‘et Lever from HIGH to LOW
5 Lover from LOW to INT
le6 Lever from INT 10 OFF
ie? Dilfrom 1102
ies Dielfrom2to3
ie) Dial from 3102
leo al from 2101
Table 15.2. Port Output Events
in the Saturn Windshield Wiper
Po Opa em Desp
sa Dupm
oot ‘opm
oc sup
oot apm
os mp
06 arm
Figure 15.9. Lever and dial finite state machines.
ara Sofware Testing: A Calsman's Approach, Second Ken
Figure 15.10. Dia EDP.
The wiper device produces six port output events — the six different wiper
speeds (expressed in wipes per minute). These are shown in Table 15.2.
‘The finite state machines forthe lever and dial are given in Figure 159. Notice
we can casi show the events that cause the state transitions, but some of the
* associated outputs Gndicated By question marks) are indeterminate, For example,
‘when we move the lever from OFF to INT, we cannot assert a specific port output
vent because we do not know the state of the dial machine. We can assert no.
put events in the dial machine, because we do not know if the lever i in the
INT position, The lever und the dial, acing concurrent, drive the wiper,
‘We will use EDENS ta compose the lever and dial finite state machines, and
this composition will exactly describe the interaction. Figure 15.5 is the EDPN
‘equivalent of the lever finite state machine: the EDPN of the dial Finite state
machine is shown in Figure 15.10, The EDPN transitions, places, and events used
fn Figure 15.11 are summarized in Table 154
‘To cast a finite state machine into an EDPN, ses become dita places and
state transitions become EDPN transitions. The events that cause state transitions
become port input events, and the outputs associated with a state transition
become pon output events. In the lever EDEN (Figure 15.5), transitions to the
OFF, LO and H states require no interaction with the dial EDPN to determine
the associated por output events (pS, P9, and pl. Also, transitions to the INT
state (SI and $5) have no port output events. In the dial FDPN (Figure 15.10),
the three dial positions are the three data places, and the dial events (p3 and pa)
are inputs that cause transitions, The dial EDPN has no port output events, because
these are never outputs of tnsitions in the dial EDPN.
Tne interaction between the lever and the dial EDPNS is shown in Figure
15.11 (see Table 15.3), I entails places 42, d5, d6, and d7, transitions $11, 512,
and 513, and port events pó, p7, and pS, which are the INT state, the dial postion
sates, and the three intermittent wiper speeds. The edges with arrowheads at
‘each end (cg, between d2 and s11) indicate that the places are inputs to and
traction Tsang 275
Figure 15.11. EDPN (or the windshield wiper sytem.
‘outputs of the associated transitions, This seems a lite artificial, but makes
the markings work smoothly.
‘We can execute the EDPN in Figure 15.11, but we need to devise (or rationale)
an initial masking. Notice that none of the places is an external input. This means
We must arbitrarly determine some staring position for the dial and lever. The
Physical reality is that both devices can be in exactly one positon ata time (a
form of contraries). thus, exactly one of dl, d2, d3, and d4 is always marked,
similary for d5, dé, and d7. We will assume that the car is shipped with the lever
in the off postion and the dia! at posiion 1. Table 153 shows the marking
sequence for the following scenario
Table 153
oY
User Input Expected Output
Lever up one position 6 strokes per minute
Lever up one position 30 strokes per minute
Dial up one position 30 strokes per minute
Lever down one position 12 strokes per minute
Lever down one position 0 strokes per minute
Figure 15.10. Dia EDP.
The wiper device produces six port output events — the six different wiper
speeds (expressed in wipes per minute). These are shown in Table 15.2.
‘The finite state machines forthe lever and dial are given in Figure 159. Notice
we can casi show the events that cause the state transitions, but some of the
* associated outputs Gndicated By question marks) are indeterminate, For example,
‘when we move the lever from OFF to INT, we cannot assert a specific port output
vent because we do not know the state of the dial machine. We can assert no.
put events in the dial machine, because we do not know if the lever i in the
INT position, The lever und the dial, acing concurrent, drive the wiper,
‘We will use EDENS ta compose the lever and dial finite state machines, and
this composition will exactly describe the interaction. Figure 15.5 is the EDPN
‘equivalent of the lever finite state machine: the EDPN of the dial Finite state
machine is shown in Figure 15.10, The EDPN transitions, places, and events used
fn Figure 15.11 are summarized in Table 154
‘To cast a finite state machine into an EDPN, ses become dita places and
state transitions become EDPN transitions. The events that cause state transitions
become port input events, and the outputs associated with a state transition
become pon output events. In the lever EDEN (Figure 15.5), transitions to the
OFF, LO and H states require no interaction with the dial EDPN to determine
the associated por output events (pS, P9, and pl. Also, transitions to the INT
state (SI and $5) have no port output events. In the dial FDPN (Figure 15.10),
the three dial positions are the three data places, and the dial events (p3 and pa)
are inputs that cause transitions, The dial EDPN has no port output events, because
these are never outputs of tnsitions in the dial EDPN.
Tne interaction between the lever and the dial EDPNS is shown in Figure
15.11 (see Table 15.3), I entails places 42, d5, d6, and d7, transitions $11, 512,
and 513, and port events pó, p7, and pS, which are the INT state, the dial postion
sates, and the three intermittent wiper speeds. The edges with arrowheads at
‘each end (cg, between d2 and s11) indicate that the places are inputs to and
traction Tsang 275
Figure 15.11. EDPN (or the windshield wiper sytem.
‘outputs of the associated transitions, This seems a lite artificial, but makes
the markings work smoothly.
‘We can execute the EDPN in Figure 15.11, but we need to devise (or rationale)
an initial masking. Notice that none of the places is an external input. This means
We must arbitrarly determine some staring position for the dial and lever. The
Physical reality is that both devices can be in exactly one positon ata time (a
form of contraries). thus, exactly one of dl, d2, d3, and d4 is always marked,
similary for d5, dé, and d7. We will assume that the car is shipped with the lever
in the off postion and the dia! at posiion 1. Table 153 shows the marking
sequence for the following scenario
Table 153
oY
User Input Expected Output
Lever up one position 6 strokes per minute
Lever up one position 30 strokes per minute
Dial up one position 30 strokes per minute
Lever down one position 12 strokes per minute
Lever down one position 0 strokes per minute
276 Software Teng: A Craftyman Approach, Second Eaton
Table 15.4 EDPN Windshield Wiper Elements
Gement Description
1 Hansen OFF to INT
52 Transition INT LO
3 Transition LO to Hi
Transition Hf 10 LO
5 Transition LO to INT
56 Transition INT to OFF
7 Transition 1102
8 Trnsition2103
9 Transition 3102
sIO Transition 2001
StI Provide 6 strokes per minute
S12 Provide 12 strokes per minute
513 Provide 20 strokes per minute
dt Lever at Off
Lever at INT
Lever atlo
de Lever at Ht
5 Dilatt
dé Dislat2
& Dias
Pl. Move lever up one position 7
P2 Move lever down one position
P3 Move dial up one position
DA Move dial down one position
PS Provide D strokes per minute
PG Provide 6 strokes per minute
p7 Provide 12 strokes per minute
PB Provide 20 strokes per minute
P9 Provido 30 strokes per minute
IO Provide 60 strokes per minute
‘The EDPN of the ful Saturn windshield wiper system in Figure 15.11 merely
begins to show the complexity of dynamic interactions among multiple processors
For stars, consider cyclomatic complexity: iis 3 for the lever EDPN, 2 for the
dial EDPN, and 20 forthe full EDPN. The jump in complexity is directly atsbutable
10 the interactions. Now, consider the complexities that result from the various
sticly sequential markings of this net, and then the contribution to complexity
‘when concurrent execution is allowed,
Some of this becomes clear if you careful follow the marking sequence in
‘Table 155 for the EDPN in Figure 15.11. ist notice that several points of event
uiescence occur, such as at marking steps m0 und m2, The net at steps m3 and
15 requires some explanation. Basically, we need to clarify the nature of the port
output events (the wiper speeds), When a port output occurs, what i its duration?
Does it continue to provide wiper strokes, or must it be executed periodically?
Both of these styles are shown in Figure 15.11. At step m3, when transition Il
Interaction Tsing
Table 15.5 Marking for Sample Scenario
pt Ph ps
Desrption
EN
a
CREME]
va
3
3 z z
B34: 3 73 8
eff? 9 af 5
Beet o fe €
PEER se sais
BiSsisasisss
só fred
27
Table 15.4 EDPN Windshield Wiper Elements
Gement Description
1 Hansen OFF to INT
52 Transition INT LO
3 Transition LO to Hi
Transition Hf 10 LO
5 Transition LO to INT
56 Transition INT to OFF
7 Transition 1102
8 Trnsition2103
9 Transition 3102
sIO Transition 2001
StI Provide 6 strokes per minute
S12 Provide 12 strokes per minute
513 Provide 20 strokes per minute
dt Lever at Off
Lever at INT
Lever atlo
de Lever at Ht
5 Dilatt
dé Dislat2
& Dias
Pl. Move lever up one position 7
P2 Move lever down one position
P3 Move dial up one position
DA Move dial down one position
PS Provide D strokes per minute
PG Provide 6 strokes per minute
p7 Provide 12 strokes per minute
PB Provide 20 strokes per minute
P9 Provido 30 strokes per minute
IO Provide 60 strokes per minute
‘The EDPN of the ful Saturn windshield wiper system in Figure 15.11 merely
begins to show the complexity of dynamic interactions among multiple processors
For stars, consider cyclomatic complexity: iis 3 for the lever EDPN, 2 for the
dial EDPN, and 20 forthe full EDPN. The jump in complexity is directly atsbutable
10 the interactions. Now, consider the complexities that result from the various
sticly sequential markings of this net, and then the contribution to complexity
‘when concurrent execution is allowed,
Some of this becomes clear if you careful follow the marking sequence in
‘Table 155 for the EDPN in Figure 15.11. ist notice that several points of event
uiescence occur, such as at marking steps m0 und m2, The net at steps m3 and
15 requires some explanation. Basically, we need to clarify the nature of the port
output events (the wiper speeds), When a port output occurs, what i its duration?
Does it continue to provide wiper strokes, or must it be executed periodically?
Both of these styles are shown in Figure 15.11. At step m3, when transition Il
Interaction Tsing
Table 15.5 Marking for Sample Scenario
pt Ph ps
Desrption
EN
a
CREME]
va
3
3 z z
B34: 3 73 8
eff? 9 af 5
Beet o fe €
PEER se sais
BiSsisasisss
só fred
27
28 Sofware Testing: A Calera Approach, Second Edition
fires, it remarks its input places (42 and d5) and executes p6 (six strokes per
minute). Because no other transition is enabled, we can picture 11 as continuously
firing untl something else (an event) happens. This is a strange form of event
quiescence, hut it is clearly a steady state of the system, The other form is shown
at marking step 015, where transition 52 has fied, resulting in port output p> GO
strokes per minute). Here, we picture P9 as having a duration that lasts as long
asd is marked. While p9 is operating, the user can cause dial por events (eps
6 and 7) with no effect on the wiper Strokes. The possible markings, then, give
us additional invight into the complexity of these interactions
The role af time adds the last increment of complexity. Consider the time
interval between port input events and what we might call the reaction time of
the system. The highest wiper speed is 60 strokes per minute, and a driver can
easily move the lever from the OFF position to the HI position in less dan one
second, What happened 10 the intervening port events (p9 and one of pá, 07,
and p8 The model is incomplete on this detail, Markings let us deal wi
simultaneity: if two events (or wansitions) occur at exactly the same point in time,
we can easly show this by marking their corresponding positions in the marking
vector, We can also amend the frig rules to allow simultaneous transition Frings
as long as the transitions are in different processors. We can represent al ofthis
‘with diagrams similar to the timing dlaprams used to describe electronic circuits.
(They are also similar to music notation, in which each voice has its own clef.
Music also has a notion of time steps, the measures of equal time duration. Tables
15.6 and 157 show two possible ways to present the concurrent behavior of the
smacking in ‘able 155,
15.3 Interaction, Composition, and Determinism
‘The question of nondeterminism looms as a backdrop to deep quexions in science:
and philosophy. Einstein did not believe in nondeterminism; he once commented
doubred that God would play dice with the universe. Nondeterminism
ly refers 10 consequences of random events, asking, in effect, if there are
truly random events (inputs), can we ever prediet their consequences? The logical
extreme of this debate ends in the philosophical/heological question of free will
versus predestination. Fortunately for testers, the software version of nondeter-
minis is less severe. You might want to consider this section to be a technical
editorial. I is based on my experience and analysis using the EDPN framework.
ind it yields reasonable answers to the problem of nondeterminism; you may too.
Let us sta with a working definition if determinism; here are two possibilities:
1. A system is deterministic if, given its inputs, we can always predict is outputs
2. A system is deterministic if it always produces the same outputs fora given
set of inputs.
"The second view (repeatable outpuns is less stringent than the first (predictable
outputs), therefore, we will use 11 a5 our working definition. Then a nandeter.
‘ministic system is one in which there is at least one set of inputs that results in.
‘wo distinct sets of outputs. It is easy to devise u nondeterministic finite state
machine; Figure 15,12 Is one example
279
Time 0 1234567689 0 nv
‘When the machine in Figure 15.12 is in state dt, if event el occurs, a transition
takes place cither to state d2 or to dB.
it is so easy to create a nondeterministic finite state machine, why all the
fuss about determinism in the first place? Recall that in Chapter 12 we took great
pains to separate the reality of a system from models of the system's behavior.
Finite state machines are models of reality; they only approximate the behavior
of a real system. This is why i i so important to choose an appropnate model
— we would like (0 use the best approximation. Roughly speaking, decision
tables ace the mode! of choice for static interaction, finite state machines suffice
fires, it remarks its input places (42 and d5) and executes p6 (six strokes per
minute). Because no other transition is enabled, we can picture 11 as continuously
firing untl something else (an event) happens. This is a strange form of event
quiescence, hut it is clearly a steady state of the system, The other form is shown
at marking step 015, where transition 52 has fied, resulting in port output p> GO
strokes per minute). Here, we picture P9 as having a duration that lasts as long
asd is marked. While p9 is operating, the user can cause dial por events (eps
6 and 7) with no effect on the wiper Strokes. The possible markings, then, give
us additional invight into the complexity of these interactions
The role af time adds the last increment of complexity. Consider the time
interval between port input events and what we might call the reaction time of
the system. The highest wiper speed is 60 strokes per minute, and a driver can
easily move the lever from the OFF position to the HI position in less dan one
second, What happened 10 the intervening port events (p9 and one of pá, 07,
and p8 The model is incomplete on this detail, Markings let us deal wi
simultaneity: if two events (or wansitions) occur at exactly the same point in time,
we can easly show this by marking their corresponding positions in the marking
vector, We can also amend the frig rules to allow simultaneous transition Frings
as long as the transitions are in different processors. We can represent al ofthis
‘with diagrams similar to the timing dlaprams used to describe electronic circuits.
(They are also similar to music notation, in which each voice has its own clef.
Music also has a notion of time steps, the measures of equal time duration. Tables
15.6 and 157 show two possible ways to present the concurrent behavior of the
smacking in ‘able 155,
15.3 Interaction, Composition, and Determinism
‘The question of nondeterminism looms as a backdrop to deep quexions in science:
and philosophy. Einstein did not believe in nondeterminism; he once commented
doubred that God would play dice with the universe. Nondeterminism
ly refers 10 consequences of random events, asking, in effect, if there are
truly random events (inputs), can we ever prediet their consequences? The logical
extreme of this debate ends in the philosophical/heological question of free will
versus predestination. Fortunately for testers, the software version of nondeter-
minis is less severe. You might want to consider this section to be a technical
editorial. I is based on my experience and analysis using the EDPN framework.
ind it yields reasonable answers to the problem of nondeterminism; you may too.
Let us sta with a working definition if determinism; here are two possibilities:
1. A system is deterministic if, given its inputs, we can always predict is outputs
2. A system is deterministic if it always produces the same outputs fora given
set of inputs.
"The second view (repeatable outpuns is less stringent than the first (predictable
outputs), therefore, we will use 11 a5 our working definition. Then a nandeter.
‘ministic system is one in which there is at least one set of inputs that results in.
‘wo distinct sets of outputs. It is easy to devise u nondeterministic finite state
machine; Figure 15,12 Is one example
279
Time 0 1234567689 0 nv
‘When the machine in Figure 15.12 is in state dt, if event el occurs, a transition
takes place cither to state d2 or to dB.
it is so easy to create a nondeterministic finite state machine, why all the
fuss about determinism in the first place? Recall that in Chapter 12 we took great
pains to separate the reality of a system from models of the system's behavior.
Finite state machines are models of reality; they only approximate the behavior
of a real system. This is why i i so important to choose an appropnate model
— we would like (0 use the best approximation. Roughly speaking, decision
tables ace the mode! of choice for static interaction, finite state machines suffice
SON: ‚Software Testing: A Criliman's Approach. Second Félion
Table 15.7_ Condensed Version of Table 15.6
Time 0 1 2 3 4 5 6 7 8 9 WH WD
we Data dd Bw
Trans s 2 s
Input pr pr ca
Output Pp po po ps
Dal Data d5 ds 55 ds ds d5 e 0 dé
Trans 7
Input ms
Output
Wiper Data
Trans
input
Output PS pé po : a
el el
8
de
le state machine
for dynamic interactions in a single processor, and some form of Petri ncis is
needed for dynamic interactions in multiple processors. Before going on, we
should indicate invances of nondeterminism in the olher two models. A multiple.
i fe is one in which te inputs (variables in the condition siub) are
such that more than one rule is selected. In Petri nets, nandelerminem occurs
‘when more than one transition is enabled, The choice of which rule executes or
ich trasion fires is made by an external agent. (Notice thatthe chuic te
actually an input)
Our question of nondeterminism reduces to threads in an EDPN, and this is
‘where interactions, composition, and determinism come together. To ground our
discussion in something real, consider the SATM threads we used eurior
TI: withdraw $40.00
T2: withdraw $60.00
T3 deposk $30.00
‚Threads Th, 12, and 13 interact via a data place for the account balance, and
they may be executed in different processors. The initial balance ls 850.00.
Begin with thread TI; if no other thread exer wes, it will execute correctly,
leaving a balance of $1000, Suppose we began with read 72, we should scaly
Interaction Tsing 281
“attempt 10 withdraw’ $60.00." because, if ao other thread executes i will
result in the insufficient funds screen, We should really separate T2 into ve
threads — T21, which is a successful heal that ends withthe display ol
screen 11 (take cash), and T22, which is a failed withdrawal that ends wilt the
display of screen 8 Cinsufficent funds), Now let us add some interaction with
theead T3, Threads T2 and T3 are Zconnected vis the balance dita place If 73
executes before T2 reads the balance data, then 12.1 occurs otlienvise, T22
oceurs, The difference between the Iwo views of determinism is VASE here:
‚when the EDPN of T2 begins to execute, we cannot predict the outcome (12)
‘08 72.2) so by the fist definition, this is nondeteminiui. By the second definition.
however, we can recreate the interaction Cinchading times) between T2 and 13
we do, and we capture the behavior a8 a caring of the composite ED?
will satisfy the repeatable definition of determinisan.
We now have a mild resolution to the question of determinism, at east as it
applies to testing threads that are expressed ar EDPNs. We can go so far as to
say that a thread is locally nondeterminisic, inthe sense that is outcome cannot
be predicted. with information local to the thread. We also saw this in the
‘windshield wiper system, If we are confined to the lever finite state machine. w
cannot determine the output when the lever moves to the Intermitent position
because we do not know the dial postion. In a global sense, however none,
erminism vanishes, because all the inputs are known, The implication for testers
is that, when testing Uareads with extemal inputs (particularly data places), iis
important to text the interaction with al other threads that can be neconnected
via external Inpus,
15.4 Client-Server Testing
Client-server (CS) systems are dificult 10 test because they exhibit the most
Siicult form of interactions, dhe dynamic ones across multiple processors, Here,
we can enjoy the benefits of our strony theoretical development. Client-server
systems always entail at least two Processors — eme where the seiver sollware
sisi and executes, and one (usually several) where the cient software executes
The main components are usually a database management system, application
Programs that use the database, and presentation programs that produce ser
defined output. The position of these components results in che lat server vs. fat
client distinction (Lewis, 1994) (see Figure 15.13).
Client-server systems also include a network to connect the clients with the
server, the network software, and a graphical user interface (GUD for the client
To make matters worse, we can differentiate homogencous and heterogeneous
3 systems in terms of client processors that are identically diverse. The multiple
terminal version of the SATM system would be a fat dit system, since the central
bank does very litle of dhe transaction processing,
In our formulation of EDPN threads, we confined a thread to a processor.
With CS systems, we need a term for a sequence of threads that crosses the CS
boundary al! a CS transaction. A typical CS transaction begins with a request
(or query) at a client processor. The request is transmitted over the network
(where it might be scheduled) 10 the server, where some application program
Processes it using the database management system (DBMS). The resuls are
Table 15.7_ Condensed Version of Table 15.6
Time 0 1 2 3 4 5 6 7 8 9 WH WD
we Data dd Bw
Trans s 2 s
Input pr pr ca
Output Pp po po ps
Dal Data d5 ds 55 ds ds d5 e 0 dé
Trans 7
Input ms
Output
Wiper Data
Trans
input
Output PS pé po : a
el el
8
de
le state machine
for dynamic interactions in a single processor, and some form of Petri ncis is
needed for dynamic interactions in multiple processors. Before going on, we
should indicate invances of nondeterminism in the olher two models. A multiple.
i fe is one in which te inputs (variables in the condition siub) are
such that more than one rule is selected. In Petri nets, nandelerminem occurs
‘when more than one transition is enabled, The choice of which rule executes or
ich trasion fires is made by an external agent. (Notice thatthe chuic te
actually an input)
Our question of nondeterminism reduces to threads in an EDPN, and this is
‘where interactions, composition, and determinism come together. To ground our
discussion in something real, consider the SATM threads we used eurior
TI: withdraw $40.00
T2: withdraw $60.00
T3 deposk $30.00
‚Threads Th, 12, and 13 interact via a data place for the account balance, and
they may be executed in different processors. The initial balance ls 850.00.
Begin with thread TI; if no other thread exer wes, it will execute correctly,
leaving a balance of $1000, Suppose we began with read 72, we should scaly
Interaction Tsing 281
“attempt 10 withdraw’ $60.00." because, if ao other thread executes i will
result in the insufficient funds screen, We should really separate T2 into ve
threads — T21, which is a successful heal that ends withthe display ol
screen 11 (take cash), and T22, which is a failed withdrawal that ends wilt the
display of screen 8 Cinsufficent funds), Now let us add some interaction with
theead T3, Threads T2 and T3 are Zconnected vis the balance dita place If 73
executes before T2 reads the balance data, then 12.1 occurs otlienvise, T22
oceurs, The difference between the Iwo views of determinism is VASE here:
‚when the EDPN of T2 begins to execute, we cannot predict the outcome (12)
‘08 72.2) so by the fist definition, this is nondeteminiui. By the second definition.
however, we can recreate the interaction Cinchading times) between T2 and 13
we do, and we capture the behavior a8 a caring of the composite ED?
will satisfy the repeatable definition of determinisan.
We now have a mild resolution to the question of determinism, at east as it
applies to testing threads that are expressed ar EDPNs. We can go so far as to
say that a thread is locally nondeterminisic, inthe sense that is outcome cannot
be predicted. with information local to the thread. We also saw this in the
‘windshield wiper system, If we are confined to the lever finite state machine. w
cannot determine the output when the lever moves to the Intermitent position
because we do not know the dial postion. In a global sense, however none,
erminism vanishes, because all the inputs are known, The implication for testers
is that, when testing Uareads with extemal inputs (particularly data places), iis
important to text the interaction with al other threads that can be neconnected
via external Inpus,
15.4 Client-Server Testing
Client-server (CS) systems are dificult 10 test because they exhibit the most
Siicult form of interactions, dhe dynamic ones across multiple processors, Here,
we can enjoy the benefits of our strony theoretical development. Client-server
systems always entail at least two Processors — eme where the seiver sollware
sisi and executes, and one (usually several) where the cient software executes
The main components are usually a database management system, application
Programs that use the database, and presentation programs that produce ser
defined output. The position of these components results in che lat server vs. fat
client distinction (Lewis, 1994) (see Figure 15.13).
Client-server systems also include a network to connect the clients with the
server, the network software, and a graphical user interface (GUD for the client
To make matters worse, we can differentiate homogencous and heterogeneous
3 systems in terms of client processors that are identically diverse. The multiple
terminal version of the SATM system would be a fat dit system, since the central
bank does very litle of dhe transaction processing,
In our formulation of EDPN threads, we confined a thread to a processor.
With CS systems, we need a term for a sequence of threads that crosses the CS
boundary al! a CS transaction. A typical CS transaction begins with a request
(or query) at a client processor. The request is transmitted over the network
(where it might be scheduled) 10 the server, where some application program
Processes it using the database management system (DBMS). The resuls are
282 sone
sing: A Cateman' Approach Secund tion
S E
D En
Figure 15.13 Fat clients and servers
wansmitted over the network to the client, where they ure mapped to à user
defined output format, If something goes wrong in a CS transaction, a lot of room
For finger pointing happens: clients blame the server, servers blame the clients,
and both blame the nctwork.
The threads in the DBMS porion of C$ transactions are all of the static
sequential type; and because this i likely 10 be either a commercial product or
a very suble application, testing is not necessarily required. Should testing he
necessary, i is almost always snicly functional, because we rarely have the source
code of commercial applications, Threads in the server applications also exhibe
primarily sic sequental interactions, and these 100 are likely 10 be in stable,
existing applications, The applications will probably be more error prone (iure
[prone is beter) than the DBMS, The network software portion of ES transaction
is also likely to be a commercial application, thus, mest CS transaction testing
will occur en the client processor). The most Interesting Grom a testing stand
point) Is the GU portion. The user ponion of a CS transaction is typically bulk
Within a commercial application program that allows the user to develop in tems
Cf a WIMP interface (windows, icons, menus, and pullslowns), and this is where
the fun begins. Clients are fe 10 move in arbitrary ways across multiple windows,
yet the results must be compatible. In our terminology. we can view a window
as a finite ste machine, and interwindow moves wil correspond to communi
‘cating finite stare machines. All the actions available to a user in a window are
POr input events, and the results of these actions are port output events, The
clicar postions of a CS transaction are therefore dineads that exhibit dynamic
interactions across multiple processors (he windows), which we know to he the
most complex,
his framework at least gives the CS tester a rigorous approach that supports
‘eating strategies that can be measured in terms of coverage metrics. The notion
of operational profiles (see Chapter 14) is very appropriate, because testing all
possible interactions could be very time consuming. Another advantage co this
framework is that, once a client thread is satisfactorily tested, the only further
concem is its interaction with other threads. This should be an important step,
because there ior of potential for n-eonnectivity among the threads of separate
Interaction Testing 28
clients, very much like the SATM scenario in which several people conducted
deposis and withdrawals on the same account
References
Lewis, T. and Rangel M, Pt servers w. a clients: the transition frou cient-server 10
ditibuted computing, American Programmer, Vol. 7 No. 11, Noversher 1994, pp. 2-9
Nine, Robin, Elements of Interción (1993 Turing Award Lecture). Communications of
the AGM, Vol 36, No. 1, January 1993
ave, Feature interactions and formal specieaions in telecomnisitions, MILE Com
ur, Aug 1993
Exercises
1. Figure 15.3 has a nice connection with set cheory. Take the propositions
Pp. q, 1. and 510 be the following set theory statements abour some sets $
and?
ser sR
a SAP = Nosis»
E SoPe® Gome sisi)
© Sah Gome Sis not P)
Convince yourself dat the relaionships inthe Square of Opposition apply
10 these set theory propositions. (The natural language statements are fen
Aristotelian categorical deduction.)
2. Find und discuss examples of reconneciviy for (he events with events
row of Table 15.4
sing: A Cateman' Approach Secund tion
S E
D En
Figure 15.13 Fat clients and servers
wansmitted over the network to the client, where they ure mapped to à user
defined output format, If something goes wrong in a CS transaction, a lot of room
For finger pointing happens: clients blame the server, servers blame the clients,
and both blame the nctwork.
The threads in the DBMS porion of C$ transactions are all of the static
sequential type; and because this i likely 10 be either a commercial product or
a very suble application, testing is not necessarily required. Should testing he
necessary, i is almost always snicly functional, because we rarely have the source
code of commercial applications, Threads in the server applications also exhibe
primarily sic sequental interactions, and these 100 are likely 10 be in stable,
existing applications, The applications will probably be more error prone (iure
[prone is beter) than the DBMS, The network software portion of ES transaction
is also likely to be a commercial application, thus, mest CS transaction testing
will occur en the client processor). The most Interesting Grom a testing stand
point) Is the GU portion. The user ponion of a CS transaction is typically bulk
Within a commercial application program that allows the user to develop in tems
Cf a WIMP interface (windows, icons, menus, and pullslowns), and this is where
the fun begins. Clients are fe 10 move in arbitrary ways across multiple windows,
yet the results must be compatible. In our terminology. we can view a window
as a finite ste machine, and interwindow moves wil correspond to communi
‘cating finite stare machines. All the actions available to a user in a window are
POr input events, and the results of these actions are port output events, The
clicar postions of a CS transaction are therefore dineads that exhibit dynamic
interactions across multiple processors (he windows), which we know to he the
most complex,
his framework at least gives the CS tester a rigorous approach that supports
‘eating strategies that can be measured in terms of coverage metrics. The notion
of operational profiles (see Chapter 14) is very appropriate, because testing all
possible interactions could be very time consuming. Another advantage co this
framework is that, once a client thread is satisfactorily tested, the only further
concem is its interaction with other threads. This should be an important step,
because there ior of potential for n-eonnectivity among the threads of separate
Interaction Testing 28
clients, very much like the SATM scenario in which several people conducted
deposis and withdrawals on the same account
References
Lewis, T. and Rangel M, Pt servers w. a clients: the transition frou cient-server 10
ditibuted computing, American Programmer, Vol. 7 No. 11, Noversher 1994, pp. 2-9
Nine, Robin, Elements of Interción (1993 Turing Award Lecture). Communications of
the AGM, Vol 36, No. 1, January 1993
ave, Feature interactions and formal specieaions in telecomnisitions, MILE Com
ur, Aug 1993
Exercises
1. Figure 15.3 has a nice connection with set cheory. Take the propositions
Pp. q, 1. and 510 be the following set theory statements abour some sets $
and?
ser sR
a SAP = Nosis»
E SoPe® Gome sisi)
© Sah Gome Sis not P)
Convince yourself dat the relaionships inthe Square of Opposition apply
10 these set theory propositions. (The natural language statements are fen
Aristotelian categorical deduction.)
2. Find und discuss examples of reconneciviy for (he events with events
row of Table 15.4
Tags
Categories
TechnologyDownload
Download Slideshow Get the original presentation fileQuick Actions
Statistics
- Views 10
- Slides 40
- Age 480 days
Related Slideshows
11
8-top-ai-courses-for-customer-support-representatives-in-2025.pptx
JeroenErne2
48 views
10
7-essential-ai-courses-for-call-center-supervisors-in-2025.pptx
JeroenErne2
47 views
13
25-essential-ai-courses-for-user-support-specialists-in-2025.pptx
JeroenErne2
37 views
11
8-essential-ai-courses-for-insurance-customer-service-representatives-in-2025.pptx
JeroenErne2
35 views
21
Know for Certain
DaveSinNM
23 views
17
PPT OPD LES 3ertt4t4tqqqe23e3e3rq2qq232.pptx
novasedanayoga46
26 views