Natural Language Processing to Automate System Testing based on Requirements
briand_lionel
513 views
92 slides
Feb 19, 2019
Slide 1 of 106
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
About This Presentation
IFIP RE meeting 2019
Slide Content
.lu
software verification & validation
VVS
Natural Language Processing to
Automate System Testing based
on Requirements
Lionel Briand
SnTCentre / University of Luxembourg
IFIP 2019
software verification & validation
VVS
Natural Language Processing to
Automate System Testing based
on Requirements
Lionel Briand
SnTCentre / University of Luxembourg
IFIP 2019
•Fabrizio Pastore
•Chunhui Wang
•ArdaGoknil
•PhuMai
Acknowledgments
•Chunhui Wang
•ArdaGoknil
•PhuMai
Acknowledgments
•Requirements in natural language (one form or another)
•are prominent throughout industry sectors, even safety-
critical ones,
•are not fading away any time soon.
•Usually requirements have many stakeholders, as they
serve as contract, are a basis for independent
acceptance testing, are shared by many organizations,
etc.
Practice
•are prominent throughout industry sectors, even safety-
critical ones,
•are not fading away any time soon.
•Usually requirements have many stakeholders, as they
serve as contract, are a basis for independent
acceptance testing, are shared by many organizations,
etc.
Practice
•In many sectors, traceability between requirements and test
cases is required by standards, customers, certifiers ...
•Requirements change, and therefore test cases as well.
•Huge traceability matrices are built and maintained manually.
•Academic work on automatically matching requirements and
test cases is not close to be sufficiently accurate or practical.
Practice
cases is required by standards, customers, certifiers ...
•Requirements change, and therefore test cases as well.
•Huge traceability matrices are built and maintained manually.
•Academic work on automatically matching requirements and
test cases is not close to be sufficiently accurate or practical.
Practice
Objective:
Support the Generation of
System Test Cases from
Requirements in Natural Language
Traceability is a by-product
Support the Generation of
System Test Cases from
Requirements in Natural Language
Traceability is a by-product
Textual descriptions are often ambiguous,
Incomplete, and not analyzable automatically
Problem
Incomplete, and not analyzable automatically
Problem
Stick to natural language but …
Compromise?
Restrict its usage so as to make it amenable
to NLPfor system testing purposes
Find the right balance
Compromise?
Restrict its usage so as to make it amenable
to NLPfor system testing purposes
Find the right balance
Restricted Use Case Modeling (RUCM)
Experiments shows it yields better use cases
[Yue et al. 2013]Restricted Use Case Specifications
More analyzable with NLP
Use Case Modeling is widely used
Compliance is tool-supported (NLP)
Experiments shows it yields better use cases
[Yue et al. 2013]Restricted Use Case Specifications
More analyzable with NLP
Use Case Modeling is widely used
Compliance is tool-supported (NLP)
Use Case Specifications Example
BodySense: embedded system that determines the occupancy
status of seats in a car
BodySense: embedded system that determines the occupancy
status of seats in a car
Use Case Specifications Example
Precondition: The system has been initialized
Basic Flow
1. The SeatSensorSENDSthe weightTOthe system.
2. INCLUDE USE CASE Self Diagnosis.
3. The system VALIDATES THATno error has been detected.
4. The system VALIDATES THATthe weight is above 20 Kg.
5. The system sets the occupancy status to adult.
6. The system SENDSthe occupancy statusTOAirbagControlUnit.
--written according to RUCM template--
Precondition: The system has been initialized
Basic Flow
1. The SeatSensorSENDSthe weightTOthe system.
2. INCLUDE USE CASE Self Diagnosis.
3. The system VALIDATES THATno error has been detected.
4. The system VALIDATES THATthe weight is above 20 Kg.
5. The system sets the occupancy status to adult.
6. The system SENDSthe occupancy statusTOAirbagControlUnit.
--written according to RUCM template--
Use Case Specifications Example
Precondition: The system has been initialized
Basic Flow
1. The SeatSensorSENDSthe weightTOthe system.
2. INCLUDE USE CASE Self Diagnosis.
3. The system VALIDATES THATno error has been detected.
4. The system VALIDATES THATthe weight is above 20 Kg.
5. The system sets the occupancy status to adult.
6. The system SENDSthe occupancy statusTOAirbagControlUnit.
--written according to RUCM template--
INPUT STEP
Precondition: The system has been initialized
Basic Flow
1. The SeatSensorSENDSthe weightTOthe system.
2. INCLUDE USE CASE Self Diagnosis.
3. The system VALIDATES THATno error has been detected.
4. The system VALIDATES THATthe weight is above 20 Kg.
5. The system sets the occupancy status to adult.
6. The system SENDSthe occupancy statusTOAirbagControlUnit.
--written according to RUCM template--
INPUT STEP
Use Case Specifications Example
Precondition: The system has been initialized
Basic Flow
1. The SeatSensorSENDSthe weightTOthe system.
2. INCLUDE USE CASE Self Diagnosis.
3. The system VALIDATES THATno error has been detected.
4. The system VALIDATES THATthe weight is above 20 Kg.
5. The system sets the occupancy status to adult.
6. The system SENDSthe occupancy statusTOAirbagControlUnit.
--written according to RUCM template--
INPUT STEP
INCLUDE STEP
Precondition: The system has been initialized
Basic Flow
1. The SeatSensorSENDSthe weightTOthe system.
2. INCLUDE USE CASE Self Diagnosis.
3. The system VALIDATES THATno error has been detected.
4. The system VALIDATES THATthe weight is above 20 Kg.
5. The system sets the occupancy status to adult.
6. The system SENDSthe occupancy statusTOAirbagControlUnit.
--written according to RUCM template--
INPUT STEP
INCLUDE STEP
Use Case Specifications Example
Precondition: The system has been initialized
Basic Flow
1. The SeatSensorSENDSthe weightTOthe system.
2. INCLUDE USE CASE Self Diagnosis.
3. The system VALIDATES THATno error has been detected.
4. The system VALIDATES THATthe weight is above 20 Kg.
5. The system sets the occupancy status to adult.
6. The system SENDSthe occupancy statusTOAirbagControlUnit.
--written according to RUCM template--
INPUT STEP
CONDITIONAL STEP
INCLUDE STEP
Precondition: The system has been initialized
Basic Flow
1. The SeatSensorSENDSthe weightTOthe system.
2. INCLUDE USE CASE Self Diagnosis.
3. The system VALIDATES THATno error has been detected.
4. The system VALIDATES THATthe weight is above 20 Kg.
5. The system sets the occupancy status to adult.
6. The system SENDSthe occupancy statusTOAirbagControlUnit.
--written according to RUCM template--
INPUT STEP
CONDITIONAL STEP
INCLUDE STEP
Use Case Specifications Example
Precondition: The system has been initialized
Basic Flow
1. The SeatSensorSENDSthe weightTOthe system.
2. INCLUDE USE CASE Self Diagnosis.
3. The system VALIDATES THATno error has been detected.
4. The system VALIDATES THATthe weight is above 20 Kg.
5. The system sets the occupancy status to adult.
6. The system SENDSthe occupancy statusTOAirbagControlUnit.
--written according to RUCM template--
INPUT STEP
CONDITIONAL STEP
INTERNAL STEP
INCLUDE STEP
Precondition: The system has been initialized
Basic Flow
1. The SeatSensorSENDSthe weightTOthe system.
2. INCLUDE USE CASE Self Diagnosis.
3. The system VALIDATES THATno error has been detected.
4. The system VALIDATES THATthe weight is above 20 Kg.
5. The system sets the occupancy status to adult.
6. The system SENDSthe occupancy statusTOAirbagControlUnit.
--written according to RUCM template--
INPUT STEP
CONDITIONAL STEP
INTERNAL STEP
INCLUDE STEP
Use Case Specifications Example
Precondition: The system has been initialized
Basic Flow
1. The SeatSensorSENDSthe weightTOthe system.
2. INCLUDE USE CASE Self Diagnosis.
3. The system VALIDATES THATno error has been detected.
4. The system VALIDATES THATthe weight is above 20 Kg.
5. The system sets the occupancy status to adult.
6. The system SENDSthe occupancy statusTOAirbagControlUnit.
--written according to RUCM template--
INPUT STEP
CONDITIONAL STEP
INTERNAL STEP
INCLUDE STEP
OUTPUT STEP
Precondition: The system has been initialized
Basic Flow
1. The SeatSensorSENDSthe weightTOthe system.
2. INCLUDE USE CASE Self Diagnosis.
3. The system VALIDATES THATno error has been detected.
4. The system VALIDATES THATthe weight is above 20 Kg.
5. The system sets the occupancy status to adult.
6. The system SENDSthe occupancy statusTOAirbagControlUnit.
--written according to RUCM template--
INPUT STEP
CONDITIONAL STEP
INTERNAL STEP
INCLUDE STEP
OUTPUT STEP
Use Case Specifications Example
Precondition: The system has been initialized
Basic Flow
1. The SeatSensorSENDSthe weightTOthe system.
2. INCLUDE USE CASE Self Diagnosis.
3. The system VALIDATES THATno error has been detected.
4. The system VALIDATES THATthe weight is above 20 Kg.
5. The system sets the occupancy status to adult.
6. The system SENDSthe occupancy statusTOAirbagControlUnit.
AlternativeFlow
RFS4.
1. IF the weight is above 1 Kg THEN
2. The system sets the occupancy status to child.
3. …
4. RESUME STEP 6.
--written according to RUCM template--
INPUT STEP
CONDITIONAL STEP
INTERNAL STEP
INCLUDE STEP
OUTPUT STEP
Precondition: The system has been initialized
Basic Flow
1. The SeatSensorSENDSthe weightTOthe system.
2. INCLUDE USE CASE Self Diagnosis.
3. The system VALIDATES THATno error has been detected.
4. The system VALIDATES THATthe weight is above 20 Kg.
5. The system sets the occupancy status to adult.
6. The system SENDSthe occupancy statusTOAirbagControlUnit.
AlternativeFlow
RFS4.
1. IF the weight is above 1 Kg THEN
2. The system sets the occupancy status to child.
3. …
4. RESUME STEP 6.
--written according to RUCM template--
INPUT STEP
CONDITIONAL STEP
INTERNAL STEP
INCLUDE STEP
OUTPUT STEP
Precondition: The system has been initialized
Basic Flow
1. The SeatSensorSENDSthe weightTOthe system.
2. INCLUDE USE CASE Self Diagnosis.
3. The system VALIDATES THATno error has been detected.
4. The system VALIDATES THATthe weight is above 20 Kg.
5. The system sets the occupancy status to adult.
6. The system SENDSthe occupancy statusTOAirbagControlUnit.
AlternativeFlow
RFS4.
1. IF the weight is above 1 Kg THEN
2. The system sets the occupancy status to child.
3. …
4. RESUME STEP 6.
NLP enables the extraction of additional information
DOMAIN ENTITY
Basic Flow
1. The SeatSensorSENDSthe weightTOthe system.
2. INCLUDE USE CASE Self Diagnosis.
3. The system VALIDATES THATno error has been detected.
4. The system VALIDATES THATthe weight is above 20 Kg.
5. The system sets the occupancy status to adult.
6. The system SENDSthe occupancy statusTOAirbagControlUnit.
AlternativeFlow
RFS4.
1. IF the weight is above 1 Kg THEN
2. The system sets the occupancy status to child.
3. …
4. RESUME STEP 6.
NLP enables the extraction of additional information
DOMAIN ENTITY
Precondition: The system has been initialized
Basic Flow
1. The SeatSensorSENDSthe weightTOthe system.
2. INCLUDE USE CASE Self Diagnosis.
3. The system VALIDATES THATno error has been detected.
4. The system VALIDATES THATthe weight is above 20 Kg.
5. The system sets the occupancy status to adult.
6. The system SENDSthe occupancy statusTOAirbagControlUnit.
AlternativeFlow
RFS4.
1. IF the weight is above 1 Kg THEN
2. The system sets the occupancy status to child.
3. …
4. RESUME STEP 6.
NLP enables the extraction of additional information
CONSTRAINT
Basic Flow
1. The SeatSensorSENDSthe weightTOthe system.
2. INCLUDE USE CASE Self Diagnosis.
3. The system VALIDATES THATno error has been detected.
4. The system VALIDATES THATthe weight is above 20 Kg.
5. The system sets the occupancy status to adult.
6. The system SENDSthe occupancy statusTOAirbagControlUnit.
AlternativeFlow
RFS4.
1. IF the weight is above 1 Kg THEN
2. The system sets the occupancy status to child.
3. …
4. RESUME STEP 6.
NLP enables the extraction of additional information
CONSTRAINT
UseCaseStart
Input
Condition
Condition
Output
Exit
Condition
Internal
Internal
Include"no error has been detected"
"the weight is above 20 Kg"
"the weight is above 1 Kg"
Weight
DomainEntityTest Model
Input
Condition
Condition
Output
Exit
Condition
Internal
Internal
Include"no error has been detected"
"the weight is above 20 Kg"
"the weight is above 1 Kg"
Weight
DomainEntityTest Model
Use Case
Specifications
Executable Test Cases
Constraints capturing
the meaning of
conditionsDomain
Model
UMTG
Error.allInstances()->forAll(
i| i.isDetected= false)
Automated Generation of System Test Cases
for Embedded Systems from Requirements in NL
https://sntsvv.github.io/UMTG/
Specifications
Executable Test Cases
Constraints capturing
the meaning of
conditionsDomain
Model
UMTG
Error.allInstances()->forAll(
i| i.isDetected= false)
Automated Generation of System Test Cases
for Embedded Systems from Requirements in NL
https://sntsvv.github.io/UMTG/
Use Case Specifications Example
BodySense: embedded system that determines the occupancy status
of seats in a car
BodySense: embedded system that determines the occupancy status
of seats in a car
UseCaseStart
Input
Condition
Condition
Output
Exit
Condition
Internal
Internal
IncludeINCLUDE USE CASE Self Diagnosis.
IF the weight is above 1 Kg THEN
The SeatSensorSENDSthe weight TOthe system.
The system sets the occupancy status to adult.
The system SENDSthe occupant class TOAirbagControlUnit.
The system VALIDATES THATno error has been detected.
The system sets the occupancy status to child.
The system VALIDATES THATthe weight is above 20 Kg.
Precondition:The system has been initialized.
OCL
OCL
OCL
OCL
Input
Condition
Condition
Output
Exit
Condition
Internal
Internal
IncludeINCLUDE USE CASE Self Diagnosis.
IF the weight is above 1 Kg THEN
The SeatSensorSENDSthe weight TOthe system.
The system sets the occupancy status to adult.
The system SENDSthe occupant class TOAirbagControlUnit.
The system VALIDATES THATno error has been detected.
The system sets the occupancy status to child.
The system VALIDATES THATthe weight is above 20 Kg.
Precondition:The system has been initialized.
OCL
OCL
OCL
OCL
Domain Model:
Formalizing Conditions
Manually written OCL constraint:
“The system VALIDATES THATno error has been detected.”
Error.allInstances()->forAll( i| i.isDetected= false)
Formalizing Conditions
Manually written OCL constraint:
“The system VALIDATES THATno error has been detected.”
Error.allInstances()->forAll( i| i.isDetected= false)
UseCaseStart
Input
Condition
Condition
Output
Exit
Condition
Internal
Internal
IncludeINCLUDE USE CASE Self Diagnosis.
IF the weight is above 1 Kg THEN
The SeatSensorSENDSthe weight TOthe system.
The system sets the occupancy status to adult.
The system SENDSthe occupant class TOAirbagControlUnit.
The system VALIDATES THATno error has been detected.
The system sets the occupancy status to child.
The system VALIDATES THATthe weight is above 20 Kg.
Precondition:The system has been initialized.
OCL
OCL
OCL
OCL
System.allInstances()->forAll( s | s.initialized= true)
ANDSystem.allInstances()->forAll( s | s.initialized= true )
AND Error.allInstances()->forAll( e | e.isDetected= false)
AND System.allInstances()
->forAll( s | s.occupancyStatus= Occupancy::Adult )
Path condition:
Input
Condition
Condition
Output
Exit
Condition
Internal
Internal
IncludeINCLUDE USE CASE Self Diagnosis.
IF the weight is above 1 Kg THEN
The SeatSensorSENDSthe weight TOthe system.
The system sets the occupancy status to adult.
The system SENDSthe occupant class TOAirbagControlUnit.
The system VALIDATES THATno error has been detected.
The system sets the occupancy status to child.
The system VALIDATES THATthe weight is above 20 Kg.
Precondition:The system has been initialized.
OCL
OCL
OCL
OCL
System.allInstances()->forAll( s | s.initialized= true)
ANDSystem.allInstances()->forAll( s | s.initialized= true )
AND Error.allInstances()->forAll( e | e.isDetected= false)
AND System.allInstances()
->forAll( s | s.occupancyStatus= Occupancy::Adult )
Path condition:
UseCaseStart
Input
Condition
Condition
Output
Exit
Condition
Internal
Internal
IncludeINCLUDE USE CASE Self Diagnosis.
IF the weight is above 1 Kg THEN
The SeatSensorSENDSthe weight TOthe system.
The system sets the occupancy status to adult.
The system SENDSthe occupant class TOAirbagControlUnit.
The system VALIDATES THATno error has been detected.
The system sets the occupancy status to child.
The system VALIDATES THATthe weight is above 20 Kg.
Precondition:The system has been initialized.
OCL
OCL
OCL
OCL
System.allInstances()->forAll( s | s.initialized= true)
ANDSystem.allInstances()->forAll( s | s.initialized= true )
AND Error.allInstances()->forAll( e | e.isDetected= false)
AND System.allInstances()
->forAll( s | s.occupancyStatus= Occupancy::Adult )
Path condition:
Constraint
Solving
(PLEDGE)
Test inputs:
https://sites.google.com/view/hybridoclsolver/
Input
Condition
Condition
Output
Exit
Condition
Internal
Internal
IncludeINCLUDE USE CASE Self Diagnosis.
IF the weight is above 1 Kg THEN
The SeatSensorSENDSthe weight TOthe system.
The system sets the occupancy status to adult.
The system SENDSthe occupant class TOAirbagControlUnit.
The system VALIDATES THATno error has been detected.
The system sets the occupancy status to child.
The system VALIDATES THATthe weight is above 20 Kg.
Precondition:The system has been initialized.
OCL
OCL
OCL
OCL
System.allInstances()->forAll( s | s.initialized= true)
ANDSystem.allInstances()->forAll( s | s.initialized= true )
AND Error.allInstances()->forAll( e | e.isDetected= false)
AND System.allInstances()
->forAll( s | s.occupancyStatus= Occupancy::Adult )
Path condition:
Constraint
Solving
(PLEDGE)
Test inputs:
https://sites.google.com/view/hybridoclsolver/
Automatically derive
formal constraints
“The system VALIDATES THAT
no error has been detected.”
Error.allInstances()->forAll( i| i.isDetected= false)
OCLgenBased on NLP
formal constraints
“The system VALIDATES THAT
no error has been detected.”
Error.allInstances()->forAll( i| i.isDetected= false)
OCLgenBased on NLP
Four assumptions
(hold in our context)
(hold in our context)
Assumption-1: the concepts appearing in requirements
specifications are modeled as entities, associations or
attributes in the domain model
“no error has been detected”
BodySense
Occupancy
Adult
Child
Empty
VoltageError
Error
isDetected: Bool
TemperatureError
initialized: Boolerrors
occupancyStatus: Occupancy
«enumeration»
weight: int
specifications are modeled as entities, associations or
attributes in the domain model
“no error has been detected”
BodySense
Occupancy
Adult
Child
Empty
VoltageError
Error
isDetected: Bool
TemperatureError
initialized: Boolerrors
occupancyStatus: Occupancy
«enumeration»
weight: int
Assumption-2: use case sentences are simple,
no compound sentences, no complex sentences
•Ensured by methodologies like RUCM
•The UMTG tool supports compliance
no compound sentences, no complex sentences
•Ensured by methodologies like RUCM
•The UMTG tool supports compliance
Error.allInstances()->forAll(i| i.isDetected= false)
EntityNameleft-hand side
(variable)
right-hand side
(variable/value)
operator
Assumption-3 : the OCL constraints required to support
automated testing of embedded systems follow a pattern
EntityNameleft-hand side
(variable)
right-hand side
(variable/value)
operator
Assumption-3 : the OCL constraints required to support
automated testing of embedded systems follow a pattern
“The system sets the occupancy status toadult.”
BodySense.allInstances()->forAll(i| i.occupancyStatus= Occupancy::Adult)
Assumption-4: OCL constraints capture information about
the state of the “actors”
typically the state reached after the activity described in the sentence
“actor” affected by the verbfinal state
BodySense.allInstances()->forAll(i| i.occupancyStatus= Occupancy::Adult)
Assumption-4: OCL constraints capture information about
the state of the “actors”
typically the state reached after the activity described in the sentence
“actor” affected by the verbfinal state
OCLgensolution
OCLgensolution
“The system sets the occupancy status toadult.”
actor affected by the verbfinal state
1. determine the role of words in a sentence
“The system sets the occupancy status toadult.”
actor affected by the verbfinal state
1. determine the role of words in a sentence
OCLgensolution
“The system sets the occupancy status toadult.”
actor affected by the verbfinal state
2. match words in the sentence with concepts in the domain model
1. determine the role of words in a sentence
“The system sets the occupancy status toadult.”
actor affected by the verbfinal state
2. match words in the sentence with concepts in the domain model
1. determine the role of words in a sentence
OCLgensolution
“The system sets the occupancy status toadult.”
BodySense.allInstances()
->forAll(i| i.occupancyStatus= Occupancy::Adult)
actor affected by the verbfinal state
2. match words in the sentence with concepts in the domain model
3. generate the OCL constraint using a verb-specific transformation rule
1. determine the role of words in a sentence
“The system sets the occupancy status toadult.”
BodySense.allInstances()
->forAll(i| i.occupancyStatus= Occupancy::Adult)
actor affected by the verbfinal state
2. match words in the sentence with concepts in the domain model
3. generate the OCL constraint using a verb-specific transformation rule
1. determine the role of words in a sentence
OCLgensolution
“The system sets the occupancy status toadult.”
BodySense.allInstances()
->forAll(i| i.occupancyStatus= Occupancy::Adult)
actor affected by the verbfinal state
2. match words in the sentence with concepts in the domain model
1. determine the role of words in a sentence
Based on Semantic Role Labeling
3. generate the OCL constraint using a verb-specific transformation rule
“The system sets the occupancy status toadult.”
BodySense.allInstances()
->forAll(i| i.occupancyStatus= Occupancy::Adult)
actor affected by the verbfinal state
2. match words in the sentence with concepts in the domain model
1. determine the role of words in a sentence
Based on Semantic Role Labeling
3. generate the OCL constraint using a verb-specific transformation rule
Semantic Role Labeling (SRL)
A0: actor that performs
an activity
“The system sets the occupancy status to adult.”
A0
A0: actor that performs
an activity
“The system sets the occupancy status to adult.”
A0
Semantic Role Labeling (SRL)
A0: actor that performs
an activity
“The system sets the occupancy status to adult.”
A0verb
A0: actor that performs
an activity
“The system sets the occupancy status to adult.”
A0verb
Semantic Role Labeling (SRL)
A1: actor that is affected
by the activitydescribed
in a sentence
A0: actor that performs
an activity
“The system sets the occupancy status to adult.”
A0A1verb
A1: actor that is affected
by the activitydescribed
in a sentence
A0: actor that performs
an activity
“The system sets the occupancy status to adult.”
A0A1verb
Semantic Role Labeling (SRL)
A1: actor that is affected
by the activitydescribed
in a sentence
A0: actor that performs
an activity
“The system sets the occupancy status to adult.”
A0A1A2
A2: final state
verb
A1: actor that is affected
by the activitydescribed
in a sentence
A0: actor that performs
an activity
“The system sets the occupancy status to adult.”
A0A1A2
A2: final state
verb
Semantic Role Labeling (SRL)
“The system resets the counter.”
A0A1verb
“The counter was reset by the system.”
“The counter has been reset.”
A1 A0verb
determine the role of words independently from passive voices,
transitive/intransitive forms
A1verb
“The system resets the counter.”
A0A1verb
“The counter was reset by the system.”
“The counter has been reset.”
A1 A0verb
determine the role of words independently from passive voices,
transitive/intransitive forms
A1verb
Semantic Role Labeling (SRL)
“The system resets the counter.”
A0A1verb
“The counter was reset by the system.”
“The counter has been reset.”
A1 A0verb
determine the role of words independently from passive voices,
transitive/intransitive forms
A1verb
“The system resets the counter.”
A0A1verb
“The counter was reset by the system.”
“The counter has been reset.”
A1 A0verb
determine the role of words independently from passive voices,
transitive/intransitive forms
A1verb
Semantic Role Labeling (SRL)
“The system resets the counter.”
A0A1verb
“The counter was reset by the system.”
“The counter has been reset.”
A1 A0verb
determine the role of words independently from passive voices,
transitive/intransitive forms
A1verb
enable pattern-based text transformations
System.allInstances()->forAll(i| i.counter= 0 )
<A1> = 0
“The system resets the counter.”
A0A1verb
“The counter was reset by the system.”
“The counter has been reset.”
A1 A0verb
determine the role of words independently from passive voices,
transitive/intransitive forms
A1verb
enable pattern-based text transformations
System.allInstances()->forAll(i| i.counter= 0 )
<A1> = 0
OCLgensolution
“The system sets the occupancy status toadult.”
BodySense.allInstances()
->forAll(i| i.occupancyStatus= Occupancy::Adult)
actor affected by the verbfinal state
2. match words in the sentence with concepts in the domain model
1. determine the role of words in a sentence
Based on Semantic Role Labeling
3. generate the OCL constraint using a verb-specific transformation rule
“The system sets the occupancy status toadult.”
BodySense.allInstances()
->forAll(i| i.occupancyStatus= Occupancy::Adult)
actor affected by the verbfinal state
2. match words in the sentence with concepts in the domain model
1. determine the role of words in a sentence
Based on Semantic Role Labeling
3. generate the OCL constraint using a verb-specific transformation rule
OCLgensolution
“The system sets the occupancy status toadult.”
BodySense.allInstances()
->forAll(i| i.occupancyStatus= Occupancy::Adult)
actor affected by the verbfinal state
2. match words in the sentence with concepts in the domain model
3. generate OCL constraint using a verb-specific transformation rule
1. determine the role of words in a sentence
Based on Semantic Role Labeling
Based on String similarity
“The system sets the occupancy status toadult.”
BodySense.allInstances()
->forAll(i| i.occupancyStatus= Occupancy::Adult)
actor affected by the verbfinal state
2. match words in the sentence with concepts in the domain model
3. generate OCL constraint using a verb-specific transformation rule
1. determine the role of words in a sentence
Based on Semantic Role Labeling
Based on String similarity
OCLgensolution
“The system sets the occupancy status toadult.”
BodySense.allInstances()
->forAll(i| i.occupancyStatus= Occupancy::Adult)
actor affected by the verbfinal state
2. match words in the sentence with concepts in the domain model
3. generate OCL constraint using a verb-specific transformation rule
1. determine the role of words in a sentence
Based on Semantic Role Labeling
How to ensure that the terms appearing
in the sentence are properly used?
Based on String similarity
“The system sets the occupancy status toadult.”
BodySense.allInstances()
->forAll(i| i.occupancyStatus= Occupancy::Adult)
actor affected by the verbfinal state
2. match words in the sentence with concepts in the domain model
3. generate OCL constraint using a verb-specific transformation rule
1. determine the role of words in a sentence
Based on Semantic Role Labeling
How to ensure that the terms appearing
in the sentence are properly used?
Based on String similarity
OCLgensolution
“The system sets the occupancy status toadult.”
BodySense.allInstances()
->forAll(i| i.occupancyStatus= Occupancy::Adult)
actor affected by the verbfinal state
2. match words in the sentence with concepts in the domain model
1. determine the role of words in a sentence
Based on Semantic Role Labeling
Based on String similarity
How to ensure that the terms appearing
in the sentence are properly used?
“The system sets the occupancy status toadult.”
BodySense.allInstances()
->forAll(i| i.occupancyStatus= Occupancy::Adult)
Adult.allInstances()
->forAll(i| i.system= occupancyStatus)
OK
NO
3. generate the OCL constraint using a verb-specific transformation rule
“The system sets the occupancy status toadult.”
BodySense.allInstances()
->forAll(i| i.occupancyStatus= Occupancy::Adult)
actor affected by the verbfinal state
2. match words in the sentence with concepts in the domain model
1. determine the role of words in a sentence
Based on Semantic Role Labeling
Based on String similarity
How to ensure that the terms appearing
in the sentence are properly used?
“The system sets the occupancy status toadult.”
BodySense.allInstances()
->forAll(i| i.occupancyStatus= Occupancy::Adult)
Adult.allInstances()
->forAll(i| i.system= occupancyStatus)
OK
NO
3. generate the OCL constraint using a verb-specific transformation rule
Ensure the Correctness of
Transformation Rules
NLP research: lexicons that describe, for each English verb,
the sets of roles typically occurring with the verb. [Kipper et al. 2000]
VerbNet(http://verbs.colorado.edu)
Transformation Rules
NLP research: lexicons that describe, for each English verb,
the sets of roles typically occurring with the verb. [Kipper et al. 2000]
VerbNet(http://verbs.colorado.edu)
Ensure the Correctness of
Transformation Rules
NLP research: lexicons that describe, for each English verb,
the sets of roles typically occurring with the verb. [Kipper et al. 2000]
A1: actor that is affected
by the activitydescribed
in a sentence
A0: actor that performs
an activity
A2: final state
“The system sets the occupancy status to adult.”
A0A1A2verb
Role sets for ‘set’ { <A0><A1><A2> , <A0><A1>, … }
Transformation Rules
NLP research: lexicons that describe, for each English verb,
the sets of roles typically occurring with the verb. [Kipper et al. 2000]
A1: actor that is affected
by the activitydescribed
in a sentence
A0: actor that performs
an activity
A2: final state
“The system sets the occupancy status to adult.”
A0A1A2verb
Role sets for ‘set’ { <A0><A1><A2> , <A0><A1>, … }
Ensure the Correctness of
Transformation Rules
NLP research: lexicons that describe, for each English verb,
the sets of roles typically occurring with the verb. [Kipper et al. 2000]
“The system sets the occupancy status to adult.”
A0A1A2verb
Role sets for ‘set’ { <A0><A1><A2> , <A0><A1>, … }
Since we know which are the roles that can appear with a verb,
we can identify patterns for generating expressions
Transformation Rules
NLP research: lexicons that describe, for each English verb,
the sets of roles typically occurring with the verb. [Kipper et al. 2000]
“The system sets the occupancy status to adult.”
A0A1A2verb
Role sets for ‘set’ { <A0><A1><A2> , <A0><A1>, … }
Since we know which are the roles that can appear with a verb,
we can identify patterns for generating expressions
Transformation Rule Pattern
‘set’
A1A2
“The system sets the occupancy status to adult.”
A0A1A2verb
BodySense.allInstances()->forAll(i| i.occupancyStatus= Occupancy::Adult)
A1: actor that is affected
by the activitydescribed
in a sentence
A0: actor that performs
an activity
A2: final state
<A1> = <A2>
‘set’
A1A2
“The system sets the occupancy status to adult.”
A0A1A2verb
BodySense.allInstances()->forAll(i| i.occupancyStatus= Occupancy::Adult)
A1: actor that is affected
by the activitydescribed
in a sentence
A0: actor that performs
an activity
A2: final state
<A1> = <A2>
Transformation Rule Pattern
‘be’
A1AM-MDR
“The weight is above twenty.”
A1AM-MDRverb
A1: actor that is affected
by the activitydescribed
in a sentence
A0: actor that performs
an activity
AM-MNR:
Adverbial modifier
<A1> operator<AM-MDR>
BodySense.allInstances()->forAll(i| i.weight> 20)
‘be’
A1AM-MDR
“The weight is above twenty.”
A1AM-MDRverb
A1: actor that is affected
by the activitydescribed
in a sentence
A0: actor that performs
an activity
AM-MNR:
Adverbial modifier
<A1> operator<AM-MDR>
BodySense.allInstances()->forAll(i| i.weight> 20)
Transformation Rule Pattern
‘be’
A1AM-MDR
“The weight is above twenty.”
A1AM-MDRverb
BodySense.allInstances()->forAll(i| i.weight> 20)
A1: actor that is affected
by the activitydescribed
in a sentence
A0: actor that performs
an activity
AM-MNR:
Adverbial modifier
[Roy et . al., 2015]
<A1> operator<AM-MDR>
‘be’
A1AM-MDR
“The weight is above twenty.”
A1AM-MDRverb
BodySense.allInstances()->forAll(i| i.weight> 20)
A1: actor that is affected
by the activitydescribed
in a sentence
A0: actor that performs
an activity
AM-MNR:
Adverbial modifier
[Roy et . al., 2015]
<A1> operator<AM-MDR>
Constraints Generation Process
Execute SRL
use case sentence
text with SRL labels
Select and apply verb-specific transformation rule
•All rules share a common algorithmic structure
•Rules differ wrtthe SRL roles labels considered
EntityName.allInstances()->forAll(i| i.LHS<Operator> RHS)
Execute SRL
use case sentence
text with SRL labels
Select and apply verb-specific transformation rule
•All rules share a common algorithmic structure
•Rules differ wrtthe SRL roles labels considered
EntityName.allInstances()->forAll(i| i.LHS<Operator> RHS)
Transformation Rules Algorithm
lhsVariables←identify a set of pairs <EntityName,LHS>
for each LHSin lhsVariablesdo
RHS←identify the term to put on the right-hand side
OP←identify the operator to use in the OCL expression
ocl←build the OCL constraint using LHS, OP and RHS
score←calculate the score of the OCL constraint
end for
bestOcl←select the OCL constraint with the best score
“The system sets the occupancy status to adult.”
A0A1A2verb
Output:EntityName.allInstances()->forAll(i| i.LHS<Operator> RHS)
lhsVariables←identify a set of pairs <EntityName,LHS>
for each LHSin lhsVariablesdo
RHS←identify the term to put on the right-hand side
OP←identify the operator to use in the OCL expression
ocl←build the OCL constraint using LHS, OP and RHS
score←calculate the score of the OCL constraint
end for
bestOcl←select the OCL constraint with the best score
“The system sets the occupancy status to adult.”
A0A1A2verb
Output:EntityName.allInstances()->forAll(i| i.LHS<Operator> RHS)
Transformation Rules Algorithm
lhsVariables←identify a set of pairs <EntityName,LHS>
for each LHSin lhsVariablesdo
RHS←identify the term to put on the right-hand side
OP←identify the operator to use in the OCL expression
ocl←build the OCL constraint using LHS, OP and RHS
score←calculate the score of the OCL constraint
end for
bestOcl←select the OCL constraint with the best score
“The system sets the occupancy status to adult.”
A0A1A2verb
BodySense, occupancyStatus;
BodySense, status;
…
Output:EntityName.allInstances()->forAll(i| i.LHS<Operator> RHS)
lhsVariables←identify a set of pairs <EntityName,LHS>
for each LHSin lhsVariablesdo
RHS←identify the term to put on the right-hand side
OP←identify the operator to use in the OCL expression
ocl←build the OCL constraint using LHS, OP and RHS
score←calculate the score of the OCL constraint
end for
bestOcl←select the OCL constraint with the best score
“The system sets the occupancy status to adult.”
A0A1A2verb
BodySense, occupancyStatus;
BodySense, status;
…
Output:EntityName.allInstances()->forAll(i| i.LHS<Operator> RHS)
Transformation Rules Algorithm
lhsVariables←identify a set of pairs <EntityName,LHS>
for each LHSin lhsVariablesdo
RHS←identify the term to put on the right-hand side
OP←identify the operator to use in the OCL expression
ocl←build the OCL constraint using LHS, OP and RHS
score←calculate the score of the OCL constraint
end for
bestOcl←select the OCL constraint with the best score
“The system sets the occupancy status to adult.”
A0A1A2verb
BodySense, occupancyStatus;
BodySense, status;
…
BodySense.allInstances()->forAll(i| i.occupancyStatus= Occupancy::Adult )
lhsVariables←identify a set of pairs <EntityName,LHS>
for each LHSin lhsVariablesdo
RHS←identify the term to put on the right-hand side
OP←identify the operator to use in the OCL expression
ocl←build the OCL constraint using LHS, OP and RHS
score←calculate the score of the OCL constraint
end for
bestOcl←select the OCL constraint with the best score
“The system sets the occupancy status to adult.”
A0A1A2verb
BodySense, occupancyStatus;
BodySense, status;
…
BodySense.allInstances()->forAll(i| i.occupancyStatus= Occupancy::Adult )
Transformation Rules Algorithm
lhsVariables←identify a set of pairs <EntityName,LHS>
for each LHSin lhsVariablesdo
RHS←identify the term to put on the right-hand side
OP←identify the operator to use in the OCL expression
ocl←build the OCL constraint using LHS, OP and RHS
score←calculate the score of the OCL constraint
end for
bestOcl←select the OCL constraint with the best score
“The system sets the occupancy status to adult.”
A0A1A2verb
BodySense, occupancyStatus;
BodySense, status;
…
BodySense.allInstances()->forAll(i| i.occupancyStatus= Occupancy::Adult )
lhsVariables←identify a set of pairs <EntityName,LHS>
for each LHSin lhsVariablesdo
RHS←identify the term to put on the right-hand side
OP←identify the operator to use in the OCL expression
ocl←build the OCL constraint using LHS, OP and RHS
score←calculate the score of the OCL constraint
end for
bestOcl←select the OCL constraint with the best score
“The system sets the occupancy status to adult.”
A0A1A2verb
BodySense, occupancyStatus;
BodySense, status;
…
BodySense.allInstances()->forAll(i| i.occupancyStatus= Occupancy::Adult )
Transformation Rules Algorithm
lhsVariables←identify a set of pairs <EntityName,LHS>
for each LHSin lhsVariablesdo
RHS←identify the term to put on the right-hand side
OP←identify the operator to use in the OCL expression
ocl←build the OCL constraint using LHS, OP and RHS
score←calculate the score of the OCL constraint
end for
bestOcl←select the OCL constraint with the best score
“The system sets the occupancy status to adult.”
A0A1A2verb
BodySense, occupancyStatus;
BodySense, status;
…
BodySense.allInstances()->forAll(i| i.occupancyStatus= Occupancy::Adult )
lhsVariables←identify a set of pairs <EntityName,LHS>
for each LHSin lhsVariablesdo
RHS←identify the term to put on the right-hand side
OP←identify the operator to use in the OCL expression
ocl←build the OCL constraint using LHS, OP and RHS
score←calculate the score of the OCL constraint
end for
bestOcl←select the OCL constraint with the best score
“The system sets the occupancy status to adult.”
A0A1A2verb
BodySense, occupancyStatus;
BodySense, status;
…
BodySense.allInstances()->forAll(i| i.occupancyStatus= Occupancy::Adult )
Transformation Rules Algorithm
lhsVariables←identify a set of pairs <EntityName,LHS>
for each LHSin lhsVariablesdo
RHS←identify the term to put on the right-hand side
OP←identify the operator to use in the OCL expression
ocl←build the OCL constraint using LHS, OP and RHS
score←calculate the score of the OCL constraint
end for
bestOcl←select the OCL constraint with the best score
“The system sets the occupancy status to adult.”
A0A1A2verb
BodySense, occupancyStatus;
BodySense, status;
…
BodySense.allInstances()->forAll(i| i.occupancyStatus= Occupancy::Adult )
lhsVariables←identify a set of pairs <EntityName,LHS>
for each LHSin lhsVariablesdo
RHS←identify the term to put on the right-hand side
OP←identify the operator to use in the OCL expression
ocl←build the OCL constraint using LHS, OP and RHS
score←calculate the score of the OCL constraint
end for
bestOcl←select the OCL constraint with the best score
“The system sets the occupancy status to adult.”
A0A1A2verb
BodySense, occupancyStatus;
BodySense, status;
…
BodySense.allInstances()->forAll(i| i.occupancyStatus= Occupancy::Adult )
Transformation Rules Algorithm
lhsVariables←identify a set of pairs <EntityName,LHS>
for each LHSin lhsVariablesdo
RHS←identify the term to put on the right-hand side
OP←identify the operator to use in the OCL expression
ocl←build the OCL constraint using LHS, OP and RHS
score←calculate the score of the OCL constraint
end for
bestOcl←select the OCL constraint with the best score
“The system sets the occupancy status to adult.”
A0A1A2verb
BodySense, occupancyStatus;
BodySense, status;
…
BodySense.allInstances()->forAll(i| i.occupancyStatus= Occupancy::Adult )
BodySense.allInstances()->forAll(i| i.status= Status::Halt )
score: 1.0
score: 0.4
lhsVariables←identify a set of pairs <EntityName,LHS>
for each LHSin lhsVariablesdo
RHS←identify the term to put on the right-hand side
OP←identify the operator to use in the OCL expression
ocl←build the OCL constraint using LHS, OP and RHS
score←calculate the score of the OCL constraint
end for
bestOcl←select the OCL constraint with the best score
“The system sets the occupancy status to adult.”
A0A1A2verb
BodySense, occupancyStatus;
BodySense, status;
…
BodySense.allInstances()->forAll(i| i.occupancyStatus= Occupancy::Adult )
BodySense.allInstances()->forAll(i| i.status= Status::Halt )
score: 1.0
score: 0.4
Transformation Rules Algorithm
lhsVariables←identify a set of pairs <EntityName,LHS>
for each LHSin lhsVariablesdo
RHS←identify the term to put on the right-hand side
OP←identify the operator to use in the OCL expression
ocl←build the OCL constraint using LHS, OP and RHS
score←calculate the score of the OCL constraint
end for
bestOcl←select the OCL constraint with the best score
“The system sets the occupancy status to adult.”
A0A1A2verb
BodySense, occupancyStatus;
BodySense, status;
…
BodySense.allInstances()->forAll(i| i.occupancyStatus= Occupancy::Adult )
BodySense.allInstances()->forAll(i| i.status= Status::Halt )
score: 1.0
score: 0.4
lhsVariables←identify a set of pairs <EntityName,LHS>
for each LHSin lhsVariablesdo
RHS←identify the term to put on the right-hand side
OP←identify the operator to use in the OCL expression
ocl←build the OCL constraint using LHS, OP and RHS
score←calculate the score of the OCL constraint
end for
bestOcl←select the OCL constraint with the best score
“The system sets the occupancy status to adult.”
A0A1A2verb
BodySense, occupancyStatus;
BodySense, status;
…
BodySense.allInstances()->forAll(i| i.occupancyStatus= Occupancy::Adult )
BodySense.allInstances()->forAll(i| i.status= Status::Halt )
score: 1.0
score: 0.4
•OCL score measures completenessand correctnessof the OCL constraint
•The generated OCL might be incomplete, in fact not all the the concepts appearing in the sentence might be captured by a term in the constraint
•LHS/RHS are based on terms that best-match the nouns/verbs in the use case steps, thus some of the selected terms might be wrong/inappropriate
•OCLscore= ( completenessScore+ correctnessScore) / 2
•completenessScore= % of SRL labels used by the transformation rule
•correctnessScore= average of similarity of LHS and RHS terms (from domain model) with the names appearing in the use case steps (percentage of matching chars)
Computing the OCL score
•The generated OCL might be incomplete, in fact not all the the concepts appearing in the sentence might be captured by a term in the constraint
•LHS/RHS are based on terms that best-match the nouns/verbs in the use case steps, thus some of the selected terms might be wrong/inappropriate
•OCLscore= ( completenessScore+ correctnessScore) / 2
•completenessScore= % of SRL labels used by the transformation rule
•correctnessScore= average of similarity of LHS and RHS terms (from domain model) with the names appearing in the use case steps (percentage of matching chars)
Computing the OCL score
Constraints Generation Algorithm
lhsVariables←identify a set of pairs <EntityName,LHS>
for each LHSin lhsVariablesdo
RHS←identify the term to put on the right-hand side
OP←identify the operator to use in the OCL expression
ocl←build the OCL constraint using LHS, OP and RHS
score←calculate the score of the OCL constraint
end for
bestOcl←select the OCL constraint with the best score
text annotated with SRL ←executeSRL on the text of a use case specification step
apply verb-specific transformation rule
transformation
rules differ
for the role labels
considered
Output:EntityName.allInstances()->forAll(i| i.LHS<Operator> RHS)
lhsVariables←identify a set of pairs <EntityName,LHS>
for each LHSin lhsVariablesdo
RHS←identify the term to put on the right-hand side
OP←identify the operator to use in the OCL expression
ocl←build the OCL constraint using LHS, OP and RHS
score←calculate the score of the OCL constraint
end for
bestOcl←select the OCL constraint with the best score
text annotated with SRL ←executeSRL on the text of a use case specification step
apply verb-specific transformation rule
transformation
rules differ
for the role labels
considered
Output:EntityName.allInstances()->forAll(i| i.LHS<Operator> RHS)
Constraints Generation Algorithm
lhsVariables←identify a set of pairs <EntityName,LHS>
for each LHSin lhsVariablesdo
RHS←identify the term to put on the right-hand side
OP←identify the operator to use in the OCL expression
ocl←build the OCL constraint using LHS, OP and RHS
score←calculate the score of the OCL constraint
end for
bestOcl←select the OCL constraint with the best score
text annotated with SRL ←executeSRL on the text of a use case specification step
apply verb-specific transformation rule
and for the
identification of
the operator
transformation
rules differ
for the role labels
considered
Output:EntityName.allInstances()->forAll(i| i.LHS<Operator> RHS)
lhsVariables←identify a set of pairs <EntityName,LHS>
for each LHSin lhsVariablesdo
RHS←identify the term to put on the right-hand side
OP←identify the operator to use in the OCL expression
ocl←build the OCL constraint using LHS, OP and RHS
score←calculate the score of the OCL constraint
end for
bestOcl←select the OCL constraint with the best score
text annotated with SRL ←executeSRL on the text of a use case specification step
apply verb-specific transformation rule
and for the
identification of
the operator
transformation
rules differ
for the role labels
considered
Output:EntityName.allInstances()->forAll(i| i.LHS<Operator> RHS)
Does the approach generalize and scale?
A same transformation rule can be reused for many verbs.
Does the approach generalize and scale?
NLP research: lexicon (VerbNet) which clusters together
verbs that are synonyms and work with the same role labels.
Does the approach generalize and scale?
NLP research: lexicon (VerbNet) which clusters together
verbs that are synonyms and work with the same role labels.
Manually inspected VerbNet: 80% of the verbs clusters are
unlikely to be used in software specifications.
A same transformation rule can be reused for many verbs.
Does the approach generalize and scale?
NLP research: lexicon (VerbNet) which clusters together
verbs that are synonyms and work with the same role labels.
unlikely to be used in software specifications.
A same transformation rule can be reused for many verbs.
Does the approach generalize and scale?
NLP research: lexicon (VerbNet) which clusters together
verbs that are synonyms and work with the same role labels.
Manually inspected VerbNet: 80% of the verbs clusters are
unlikely to be used in software specifications.
A same transformation rule can be reused for many verbs.
Many (~400) verbs can be processed with the same rule.
TemperatureError.allInstances()->forAll( i| i.isDetected= true)
A1verb
Does the approach generalize and scale?
NLP research: lexicon (VerbNet) which clusters together
verbs that are synonyms and work with the same role labels.
“The system detectsa temperature error.”
unlikely to be used in software specifications.
A same transformation rule can be reused for many verbs.
Many (~400) verbs can be processed with the same rule.
TemperatureError.allInstances()->forAll( i| i.isDetected= true)
A1verb
Does the approach generalize and scale?
NLP research: lexicon (VerbNet) which clusters together
verbs that are synonyms and work with the same role labels.
“The system detectsa temperature error.”
Empirical Evaluation
•Evaluate the effectiveness of the approach
•Case study: BodySense, embedded system for detecting
occupancy status in a car
•Evaluation:
•Automatically generate the OCL constraints required
to automatically derive system test cases
•Evaluate the effectiveness of the approach
•Case study: BodySense, embedded system for detecting
occupancy status in a car
•Evaluation:
•Automatically generate the OCL constraints required
to automatically derive system test cases
Results: Precision and Recall
•88 OCL constraints to be generated
from 88 use case specification sentences
•OCLGengenerates 69 constraints
•66 correct, only 3 incorrect
•Very high precision
•High Recall
# Correctly generated constraints
# Generated constraints=precision =
# Correctly generated constraints
# Constraints to be generated=recall =
= 0.9766
69
= 0.7566
88
•88 OCL constraints to be generated
from 88 use case specification sentences
•OCLGengenerates 69 constraints
•66 correct, only 3 incorrect
•Very high precision
•High Recall
# Correctly generated constraints
# Generated constraints=precision =
# Correctly generated constraints
# Constraints to be generated=recall =
= 0.9766
69
= 0.7566
88
Results: Limiting Factors
•Imprecise specifications
•“The system VALIDATES THAT the temperature is valid“
•Inconsistent terminology
•“The system VALIDATES THAT the occupancy status is valid“
BodySense.allInstances()->forAll(i| i.temperature< 200)
BodySense.allInstances()->forAll(i| i.occupancyStatus<> Empty)
•Imprecise specifications
•“The system VALIDATES THAT the temperature is valid“
•Inconsistent terminology
•“The system VALIDATES THAT the occupancy status is valid“
BodySense.allInstances()->forAll(i| i.temperature< 200)
BodySense.allInstances()->forAll(i| i.occupancyStatus<> Empty)
https://oclgen.github.io
https://sntsvv.github.io/UMTG/
https://sntsvv.github.io/UMTG/
Security Requirements
•Security requirements should be “usable” by all
stakeholders
•Technical and, to some extent, non-technical roles
•Technical roles: IT people who cannot handle formal
representations
•Security requirements with integratedwith
functional requirements (Use Cases)
•Security requirements should be automatically
analyzablefor multiple purposes, e.g.,
automated security testing
•Security requirements should be “usable” by all
stakeholders
•Technical and, to some extent, non-technical roles
•Technical roles: IT people who cannot handle formal
representations
•Security requirements with integratedwith
functional requirements (Use Cases)
•Security requirements should be automatically
analyzablefor multiple purposes, e.g.,
automated security testing
Restricted Misuse Case Modeling
RMCM
Misuse
Case
SpecificationsUse Case
Specifications
Security
Use Case
SpecificationsMisuse
Case
Diagram
Mitigation
Schemes
Based on templates with keywords
RMCM
Misuse
Case
SpecificationsUse Case
Specifications
Security
Use Case
SpecificationsMisuse
Case
Diagram
Mitigation
Schemes
Based on templates with keywords
Patient
Access
Discussion Boards
Provide
Health Data
Get Fitter
by Walking
Monitor
Patient Progress
Configure
Patient Tablet
Login
View
Patient Information
Malicious
User
Malicious
App
Bypass
Authorization
Schema
Collect
Credentials
Extract Data
From Insecure
Data Storage
Get Access
with SQLi
«threaten
»
«threaten
»
«threaten
»
Validate
Data Access
Rights
«include»
«mitigate»
Family
Member
EDLAH2
Access
Discussion Boards
Provide
Health Data
Get Fitter
by Walking
Monitor
Patient Progress
Configure
Patient Tablet
Login
View
Patient Information
Malicious
User
Malicious
App
Bypass
Authorization
Schema
Collect
Credentials
Extract Data
From Insecure
Data Storage
Get Access
with SQLi
«threaten
»
«threaten
»
«threaten
»
Validate
Data Access
Rights
«include»
«mitigate»
Family
Member
EDLAH2
Example Misuse Case Specification Flows
Basic Threat Flow:
1. FOREACH role
2. The MALICIOUS user SENDS username and password TO the system
through the login page
3. FOREACH resource
4. The MALICIOUS user REQUESTS the resource FROM the system.
5. The system SENDS a response page TO the MALICIOUS user.
6. The MALICIOUS user EXPLOITS the system using the response page.
7. ENDFOR
8. ENDFOR
Postcondition: The MALICIOUS user has accessed a resourcededicatedto
another user with different role
Specific Alternative Threat Flow (SATF1)
RFS 4
1. IF the URL includes a role parameter THEN
2. The MALICIOUS user updates the role value with his role
3. RESUME STEP 5.
…
Nominal scenario for a malicious actor to
successfully harm the system
Basic Threat Flow:
1. FOREACH role
2. The MALICIOUS user SENDS username and password TO the system
through the login page
3. FOREACH resource
4. The MALICIOUS user REQUESTS the resource FROM the system.
5. The system SENDS a response page TO the MALICIOUS user.
6. The MALICIOUS user EXPLOITS the system using the response page.
7. ENDFOR
8. ENDFOR
Postcondition: The MALICIOUS user has accessed a resourcededicatedto
another user with different role
Specific Alternative Threat Flow (SATF1)
RFS 4
1. IF the URL includes a role parameter THEN
2. The MALICIOUS user updates the role value with his role
3. RESUME STEP 5.
…
Nominal scenario for a malicious actor to
successfully harm the system
Example Misuse Case Specification Flows
Basic Threat Flow:
1. FOREACH role
2. The MALICIOUS user SENDS username and password TO the system
through the login page
3. FOREACH resource
4. The MALICIOUS user REQUESTS the resource FROM the system.
5. The system SENDS a response page TO the MALICIOUS user.
6. The MALICIOUS user EXPLOITS the system using the response page.
7. ENDFOR
8. ENDFOR
Postcondition: The MALICIOUS user has accessed a resourcededicatedto
another user with different role
Specific Alternative Threat Flow (SATF1)
RFS 4
1. IF the URL includes a role parameter THEN
2. The MALICIOUS user updates the role value with his role
3. RESUME STEP 5.
…
Control flow
Input and output steps
Exploit information exposed in
error or exception messages
Unwanted impact on asset
Basic Threat Flow:
1. FOREACH role
2. The MALICIOUS user SENDS username and password TO the system
through the login page
3. FOREACH resource
4. The MALICIOUS user REQUESTS the resource FROM the system.
5. The system SENDS a response page TO the MALICIOUS user.
6. The MALICIOUS user EXPLOITS the system using the response page.
7. ENDFOR
8. ENDFOR
Postcondition: The MALICIOUS user has accessed a resourcededicatedto
another user with different role
Specific Alternative Threat Flow (SATF1)
RFS 4
1. IF the URL includes a role parameter THEN
2. The MALICIOUS user updates the role value with his role
3. RESUME STEP 5.
…
Control flow
Input and output steps
Exploit information exposed in
error or exception messages
Unwanted impact on asset
Basic Threat Flow:
1. FOREACH role
2. The MALICIOUS user SENDS username and password TO the system
through the login page
3. FOREACH resource
4. The MALICIOUS user REQUESTS the resource FROM the system.
5. The system SENDS a response page TO the MALICIOUS user.
6. The MALICIOUS user EXPLOITS the system using the response page.
7. ENDFOR
8. ENDFOR
Postcondition: The MALICIOUS user has accessed a resourcededicatedto
another user with different role
Specific Alternative Threat Flow (SATF1)
RFS 4
1. IF the URL includes a role parameter THEN
2. The MALICIOUS user updates the role value with his role
3. RESUME STEP 5.
...
Alternative attack scenario:
includes the keyword ‘Threat’
Example Misuse Case Specification Flows
Specific alternative threat flow branching from step 4
in basic threat flow
1. FOREACH role
2. The MALICIOUS user SENDS username and password TO the system
through the login page
3. FOREACH resource
4. The MALICIOUS user REQUESTS the resource FROM the system.
5. The system SENDS a response page TO the MALICIOUS user.
6. The MALICIOUS user EXPLOITS the system using the response page.
7. ENDFOR
8. ENDFOR
Postcondition: The MALICIOUS user has accessed a resourcededicatedto
another user with different role
Specific Alternative Threat Flow (SATF1)
RFS 4
1. IF the URL includes a role parameter THEN
2. The MALICIOUS user updates the role value with his role
3. RESUME STEP 5.
...
Alternative attack scenario:
includes the keyword ‘Threat’
Example Misuse Case Specification Flows
Specific alternative threat flow branching from step 4
in basic threat flow
Basic Threat Flow:
1. FOREACH role
2. The MALICIOUS user SENDS username and password TO the system
through the login page
3. FOREACH resource
4. The MALICIOUS user REQUESTS the resource FROM the system.
5. The system SENDS a response page TO the MALICIOUS user.
6. The MALICIOUS user EXPLOITS the system using the response page.
7. ENDFOR
8. ENDFOR
Postcondition: The MALICIOUS user has accessed a resourcededicatedto
another user with different role
Specific Alternative Flow (SAF1)
RFS 5
1. IF the response page contains an error message THEN
2. RESUME STEP 7.
3. ENDIF.
Postcondition:The MALICIOUSuser cannot access the resource
Failed attack scenario:
‘Alternative Flow’ used instead of
‘Alternative Threat Flow’.
Alternative flows always begin with a conditional
statement: simplify the identification of the
conditions that trigger the alternative behavior.
Example Misuse Case Specification Flows
1. FOREACH role
2. The MALICIOUS user SENDS username and password TO the system
through the login page
3. FOREACH resource
4. The MALICIOUS user REQUESTS the resource FROM the system.
5. The system SENDS a response page TO the MALICIOUS user.
6. The MALICIOUS user EXPLOITS the system using the response page.
7. ENDFOR
8. ENDFOR
Postcondition: The MALICIOUS user has accessed a resourcededicatedto
another user with different role
Specific Alternative Flow (SAF1)
RFS 5
1. IF the response page contains an error message THEN
2. RESUME STEP 7.
3. ENDIF.
Postcondition:The MALICIOUSuser cannot access the resource
Failed attack scenario:
‘Alternative Flow’ used instead of
‘Alternative Threat Flow’.
Alternative flows always begin with a conditional
statement: simplify the identification of the
conditions that trigger the alternative behavior.
Example Misuse Case Specification Flows
Security Use Case Specification
81
SECURITY USE CASE: Validate Data Access Rights
Description: The system verifies if the access to the data of a certain patient
can be granted to a given user.
Precondition:A user has requested patient DATA
Compliance: ISO/IEC 27001:2013 clause A.9.4
Mitigate: Bypass Authorization Schema
Basic Flow
1. The system VALIDATES THAT the user is a carer.
2. The system VALIDATES THAT the user belongs to the
group of carersof the patient.
3. The system SENDS the requested data TO the user.
Postcondition:The DATA access is granted.
Specific Alternative Flow (SAF2):
RFS 1
1. ABORT.
Postcondition:Patient DATA access is denied.
Counter-measure for misuse case(s)
Steps performed to
mitigate attacks in
misuse cases
Traceability to provisions in
standards
81
SECURITY USE CASE: Validate Data Access Rights
Description: The system verifies if the access to the data of a certain patient
can be granted to a given user.
Precondition:A user has requested patient DATA
Compliance: ISO/IEC 27001:2013 clause A.9.4
Mitigate: Bypass Authorization Schema
Basic Flow
1. The system VALIDATES THAT the user is a carer.
2. The system VALIDATES THAT the user belongs to the
group of carersof the patient.
3. The system SENDS the requested data TO the user.
Postcondition:The DATA access is granted.
Specific Alternative Flow (SAF2):
RFS 1
1. ABORT.
Postcondition:Patient DATA access is denied.
Counter-measure for misuse case(s)
Steps performed to
mitigate attacks in
misuse cases
Traceability to provisions in
standards
Security Use Case Specification
82
SECURITY USE CASE: Validate Data Access Rights
Description: The system verifies if the access to the data of a certain patient
can be granted to a given user.
Precondition:A user has requested patient DATA
Compliance: ISO/IEC 27001:2013 clause A.9.4
Mitigate: Bypass Authorization Schema
Basic Flow
1. The system VALIDATES THAT the user is a carer.
2. The system VALIDATES THAT the user belongs to the
group of carersof the patient.
3. The system SENDS the requested data TO the user.
Postcondition:The DATA access is granted.
Specific Alternative Flow (SAF2):
RFS 1
1. ABORT.
Postcondition:Patient DATA access is denied
User not a carer
Check condition
Condition violated
82
SECURITY USE CASE: Validate Data Access Rights
Description: The system verifies if the access to the data of a certain patient
can be granted to a given user.
Precondition:A user has requested patient DATA
Compliance: ISO/IEC 27001:2013 clause A.9.4
Mitigate: Bypass Authorization Schema
Basic Flow
1. The system VALIDATES THAT the user is a carer.
2. The system VALIDATES THAT the user belongs to the
group of carersof the patient.
3. The system SENDS the requested data TO the user.
Postcondition:The DATA access is granted.
Specific Alternative Flow (SAF2):
RFS 1
1. ABORT.
Postcondition:Patient DATA access is denied
User not a carer
Check condition
Condition violated
Misuse
Case
Specifications
Use Case
Specifications
Security
Use Case
Specifications
Misuse
Case
Diagram Mitigation
Schemes
Toolset
Checks conformance of specifications with RMCM syntax
Checks consistency of diagrams and specifications
•Use cases should appear both in diagram and specifications
•Dependencies should appear both in diagram and specifications
•https://sntsvv.github.io/MCP/
+ Conformance & Consistency Checker
based on NLP+ UML profiles
Case
Specifications
Use Case
Specifications
Security
Use Case
Specifications
Misuse
Case
Diagram Mitigation
Schemes
Toolset
Checks conformance of specifications with RMCM syntax
Checks consistency of diagrams and specifications
•Use cases should appear both in diagram and specifications
•Dependencies should appear both in diagram and specifications
•https://sntsvv.github.io/MCP/
+ Conformance & Consistency Checker
based on NLP+ UML profiles
Summary
•Natural Language Requirements prevail and will keep doing so in
practice –it has not changed for 50 years
•Traceability with system test cases is required –no scalable and
practical solutions
•We propose a way to generate test cases from requirements using
Natural Language Processing
•Trade-off: We compromise by adding structure and constraints
(Restricted Use Case Modeling) to NL requirements and requiring
minimum modeling (Domain model)
•Many highly powerful NLP techniques are now available:
Lexicons, Semantic Role Labeling, etc.
•We have demonstrated feasibility in industrial context
•Natural Language Requirements prevail and will keep doing so in
practice –it has not changed for 50 years
•Traceability with system test cases is required –no scalable and
practical solutions
•We propose a way to generate test cases from requirements using
Natural Language Processing
•Trade-off: We compromise by adding structure and constraints
(Restricted Use Case Modeling) to NL requirements and requiring
minimum modeling (Domain model)
•Many highly powerful NLP techniques are now available:
Lexicons, Semantic Role Labeling, etc.
•We have demonstrated feasibility in industrial context
Future Work?
Automated acceptance testing in the presence of a GUI: Map use
case specification steps to sequences of GUI widget operations
Transform non-functional requirements to models amenable to
verification, e.g., timing requirements to timed automata (UPPAAL)
Extract temporal properties from requirements to support
verification and testing
Derive test cases from bug and vulnerability reports
Automated acceptance testing in the presence of a GUI: Map use
case specification steps to sequences of GUI widget operations
Transform non-functional requirements to models amenable to
verification, e.g., timing requirements to timed automata (UPPAAL)
Extract temporal properties from requirements to support
verification and testing
Derive test cases from bug and vulnerability reports
References
•UMTG
•Chunhui Wang et al., Automatic Generation of System Test Cases from Use Case Specifications. ISSTA 2015.
•Chunhui Wang, et al., UMTG: A Toolset to Automatically Generate System Test Cases from Use Case Specifications. ESEC/FSE’15, Tool demo track.
•C. Wang et al., Automated generation of constraints from use case specifications to support system testing. ICST 2018.
•RMCM
•Mai et al., “Modeling Security and Privacy Requirements: a Use Case-Driven Approach”, IST journal (Elsevier), 2018
•Mai et al., “A Natural Language Programming Approach for Requirements-based Security Testing”, ISSRE 2018
•RUCM
•T. Yue et al., “Facilitating the transition from use case models to analysis models: Approach and experiments,” ACM Transactions on Software Engineering and Methodology, vol. 22, no. 1, 2013.
•UMTG
•Chunhui Wang et al., Automatic Generation of System Test Cases from Use Case Specifications. ISSTA 2015.
•Chunhui Wang, et al., UMTG: A Toolset to Automatically Generate System Test Cases from Use Case Specifications. ESEC/FSE’15, Tool demo track.
•C. Wang et al., Automated generation of constraints from use case specifications to support system testing. ICST 2018.
•RMCM
•Mai et al., “Modeling Security and Privacy Requirements: a Use Case-Driven Approach”, IST journal (Elsevier), 2018
•Mai et al., “A Natural Language Programming Approach for Requirements-based Security Testing”, ISSRE 2018
•RUCM
•T. Yue et al., “Facilitating the transition from use case models to analysis models: Approach and experiments,” ACM Transactions on Software Engineering and Methodology, vol. 22, no. 1, 2013.
NLP References
•V. Punyakanok, D. Roth, and W. Yih, “The importance of syntactic
parsing and inference in semantic role labeling,” Computational
Linguistics, vol. 34, no. 2, 2008. [Online]. Available:
http://cogcomp.cs.illinois.edu/papers/PunyakanokRoYi07.pdf
•K. Kipper, H. T. Dang, and M. Palmer, “Class-based construction
of a verb lexicon,” in Proceedings of National Conference on
Artificial Intelligence and Twelfth Conference on Innovative
Applications of Artificial Intelligence. AAAI Press, 2000, pp. 691–
696. [Online]. Available:
http://dl.acm.org/citation.cfm?id=647288.721573
•S. Roy, T. Vieira, and D. Roth, “Reasoning about quantities in
natural language,” Transactions of the Association for
Computational Linguistics (TACL), vol. 3, 2015. [Online].
Available: http://cogcomp.org/papers/RoyViRo15.pdf
•V. Punyakanok, D. Roth, and W. Yih, “The importance of syntactic
parsing and inference in semantic role labeling,” Computational
Linguistics, vol. 34, no. 2, 2008. [Online]. Available:
http://cogcomp.cs.illinois.edu/papers/PunyakanokRoYi07.pdf
•K. Kipper, H. T. Dang, and M. Palmer, “Class-based construction
of a verb lexicon,” in Proceedings of National Conference on
Artificial Intelligence and Twelfth Conference on Innovative
Applications of Artificial Intelligence. AAAI Press, 2000, pp. 691–
696. [Online]. Available:
http://dl.acm.org/citation.cfm?id=647288.721573
•S. Roy, T. Vieira, and D. Roth, “Reasoning about quantities in
natural language,” Transactions of the Association for
Computational Linguistics (TACL), vol. 3, 2015. [Online].
Available: http://cogcomp.org/papers/RoyViRo15.pdf
.lu
software verification & validation
VVS
Natural Language Processing to
Automate System Testing based
on Requirements
Lionel Briand
SnTCentre / University of Luxembourg
software verification & validation
VVS
Natural Language Processing to
Automate System Testing based
on Requirements
Lionel Briand
SnTCentre / University of Luxembourg
Additional Notes
Identifying the LHS variable
identify an attribute of the main class of the system (e.g., BodySense)
that best-matches the chunk labeled as A1
A0A1
“The system sets the occupancy status to empty.”
BodySense.allInstances()->forAll(i| i.occupancyStatus= Occupancy::Empty)
A2verb
identify an attribute of the main class of the system (e.g., BodySense)
that best-matches the chunk labeled as A1
A0A1
“The system sets the occupancy status to empty.”
BodySense.allInstances()->forAll(i| i.occupancyStatus= Occupancy::Empty)
A2verb
Identifying the LHS variable
identify an attribute of the main class of the system (e.g.,
BodySense) that best-matches the chunk labeled as A1
best-matching:
labeled chunk is a prefix/suffix of the class/attribute name (or viceversa)
the class/attribute name more similar to the labeled chunk (string alignment)
identify an attribute of the main class of the system (e.g.,
BodySense) that best-matches the chunk labeled as A1
best-matching:
labeled chunk is a prefix/suffix of the class/attribute name (or viceversa)
the class/attribute name more similar to the labeled chunk (string alignment)
“no error has been detected”
Error.allInstances()->forAll( i| i.isDetected= false)A1verb
if LHS is booleanor integer
use the boolean/integer representation of the value tagged by a support role
(e.g., A2)
if LHS is booleanand support roles were already used to identify LHS
use true/false depending on the positivenessof the sentence
neg
“The system sets the temperature to zero”
if LHS is of enumeration type
use enumeration literal that best-matches a support role (e.g. verb)
“the build check has been passed”
A1verb
BodySense.allInstances()
→forAll( i| i.buildCheckStatus= BuildCheckStatus::Passed)
A1A2verbA0BodySense.allInstances()->forAll( i| i.temperature= 0 )
Identifying RHS
Error.allInstances()->forAll( i| i.isDetected= false)A1verb
if LHS is booleanor integer
use the boolean/integer representation of the value tagged by a support role
(e.g., A2)
if LHS is booleanand support roles were already used to identify LHS
use true/false depending on the positivenessof the sentence
neg
“The system sets the temperature to zero”
if LHS is of enumeration type
use enumeration literal that best-matches a support role (e.g. verb)
“the build check has been passed”
A1verb
BodySense.allInstances()
→forAll( i| i.buildCheckStatus= BuildCheckStatus::Passed)
A1A2verbA0BodySense.allInstances()->forAll( i| i.temperature= 0 )
Identifying RHS
Identifying the LHS variable
Identifying the LHS variable
identify the entity name that best-matches the chunk labeled as A1
identify the entity name that best-matches the chunk labeled as A1
Identifying the LHS variable
“The system sets temperature errors to detected”
A1A0verbA2
TemperatureError.allInstances()->forAll( i| i.isDetected= true)
identify the entity name that best-matches the chunk labeled as A1
“The system sets temperature errors to detected”
A1A0verbA2
TemperatureError.allInstances()->forAll( i| i.isDetected= true)
identify the entity name that best-matches the chunk labeled as A1
Identifying the LHS variable
“The system sets temperature errors to detected”
A1A0verbA2
TemperatureError.allInstances()->forAll( i| i.isDetected= true)
identify the entity name that best-matches the chunk labeled as A1
select an attribute of the identified entity that best-matches a
support role (e.g., A2)
“The system sets temperature errors to detected”
A1A0verbA2
TemperatureError.allInstances()->forAll( i| i.isDetected= true)
identify the entity name that best-matches the chunk labeled as A1
select an attribute of the identified entity that best-matches a
support role (e.g., A2)
Identifying the RHS term
if LHS is of enumeration type
use enumeration literal that best-matches a support role (e.g., A2)
A0A1
Identifying the RHS term
“The system sets the occupancy status to adult.”
BodySense.allInstances()->forAll(i| i.occupancyStatus= Occupancy::Adult)
A2verb
use enumeration literal that best-matches a support role (e.g., A2)
A0A1
Identifying the RHS term
“The system sets the occupancy status to adult.”
BodySense.allInstances()->forAll(i| i.occupancyStatus= Occupancy::Adult)
A2verb
Identifying the OCL operator
in general the operator equalis the one selected by most of the
verb-specific transformation rules
in general the operator equalis the one selected by most of the
verb-specific transformation rules
Identifying the OCL operator
in general the operator equalis the one selected by most of the
verb-specific transformation rules
for the verb “to be” we rely on existing NLP solution
[Roy et . al., 2015]
“the weight is above 10 kg”
BodySense.allInstances()->forAll( i| i.weight> 10 )
in general the operator equalis the one selected by most of the
verb-specific transformation rules
for the verb “to be” we rely on existing NLP solution
[Roy et . al., 2015]
“the weight is above 10 kg”
BodySense.allInstances()->forAll( i| i.weight> 10 )
CogCompSRL toolset
CogCompSRL toolset
(S (NP (DT The)
(NN system))
(VP (VBZ sets)
(NP (NP (DT the)
(NN occupancy)
(NN status))
(PP (TO to)
(ADJP (JJ empty)
Chunks Identification
(based on parse tree)
(S (NP (DT The)
(NN system))
(VP (VBZ sets)
(NP (NP (DT the)
(NN occupancy)
(NN status))
(PP (TO to)
(ADJP (JJ empty)
Chunks Identification
(based on parse tree)
CogCompSRL toolset
Chunks Identification
(based on parse tree)
(S (NP (DT The)
(NN system))
(VP (VBZ sets)
(NP (NP (DT the)
(NN occupancy)
(NN status))
(PP (TO to)
(ADJP (JJ empty)
Chunks Identification
(based on parse tree)
(S (NP (DT The)
(NN system))
(VP (VBZ sets)
(NP (NP (DT the)
(NN occupancy)
(NN status))
(PP (TO to)
(ADJP (JJ empty)
CogCompSRL toolset
Chunks Identification
(based on parse tree)
Chunks Refinement
(binary classifiers)
The system
sets
the occupancy status
to empty
the occupancy status to empty
(S (NP (DT The)
(NN system))
(VP (VBZ sets)
(NP (NP (DT the)
(NN occupancy)
(NN status))
(PP (TO to)
(ADJP (JJ empty)
Chunks Identification
(based on parse tree)
Chunks Refinement
(binary classifiers)
The system
sets
the occupancy status
to empty
the occupancy status to empty
(S (NP (DT The)
(NN system))
(VP (VBZ sets)
(NP (NP (DT the)
(NN occupancy)
(NN status))
(PP (TO to)
(ADJP (JJ empty)
CogCompSRL toolset
Chunks Identification
(based on parse tree)
Chunks Refinement
(binary classifiers)
RolesClassification
(network of linear
functions)
The system
sets
the occupancy status
to empty
the occupancy status to empty
A0=0.78
A1=0.25
A2=0.12
…
A0=0.14
A1=0.68
A2=0.20
…
A0=0.04
A1=0.28
A2=0.40
…
A0=0.06
A1=0.28
A2=0.47
…
(S (NP (DT The)
(NN system))
(VP (VBZ sets)
(NP (NP (DT the)
(NN occupancy)
(NN status))
(PP (TO to)
(ADJP (JJ empty)
Chunks Identification
(based on parse tree)
Chunks Refinement
(binary classifiers)
RolesClassification
(network of linear
functions)
The system
sets
the occupancy status
to empty
the occupancy status to empty
A0=0.78
A1=0.25
A2=0.12
…
A0=0.14
A1=0.68
A2=0.20
…
A0=0.04
A1=0.28
A2=0.40
…
A0=0.06
A1=0.28
A2=0.47
…
(S (NP (DT The)
(NN system))
(VP (VBZ sets)
(NP (NP (DT the)
(NN occupancy)
(NN status))
(PP (TO to)
(ADJP (JJ empty)
CogCompSRL toolset
Chunks Identification
(based on parse tree)
Chunks Refinement
(binary classifiers)
RolesClassification
(network of linear
functions)
Final Labeling
(integer linear
programming)
The system
sets
the occupancy status
to empty
the occupancy status to empty
A0=0.78
A1=0.25
A2=0.12
…
A0=0.14
A1=0.68
A2=0.20
…
A0=0.04
A1=0.28
A2=0.40
…
A0=0.78
A2=0.48
A1=0.78
satisfy grammar
rules
A0=0.06
A1=0.28
A2=0.47
…<ignore>
(S (NP (DT The)
(NN system))
(VP (VBZ sets)
(NP (NP (DT the)
(NN occupancy)
(NN status))
(PP (TO to)
(ADJP (JJ empty)
Chunks Identification
(based on parse tree)
Chunks Refinement
(binary classifiers)
RolesClassification
(network of linear
functions)
Final Labeling
(integer linear
programming)
The system
sets
the occupancy status
to empty
the occupancy status to empty
A0=0.78
A1=0.25
A2=0.12
…
A0=0.14
A1=0.68
A2=0.20
…
A0=0.04
A1=0.28
A2=0.40
…
A0=0.78
A2=0.48
A1=0.78
satisfy grammar
rules
A0=0.06
A1=0.28
A2=0.47
…<ignore>
(S (NP (DT The)
(NN system))
(VP (VBZ sets)
(NP (NP (DT the)
(NN occupancy)
(NN status))
(PP (TO to)
(ADJP (JJ empty)
Tags
Categories
BusinessDownload
Download Slideshow Get the original presentation fileQuick Actions
Statistics
- Views 513
- Slides 92
- Favorites 1
- Age 2478 days
Related Slideshows
1
DTI BPI Pivot Small Business - BUSINESS START UP PLAN
MeljunCortes
30 views
1
CATHOLIC EDUCATIONAL Corporate Responsibilities
MeljunCortes
31 views
11
Karin Schaupp – Evocation; lançamento: 2000
alfeuRIO
31 views
10
Pillars of Biblical Oneness in the Book of Acts
JanParon
27 views
31
7-10. STP + Branding and Product & Services Strategies.pptx
itsyash298
29 views
44
Business Legislation PPT - UNIT 1 jimllpkggg
slogeshk98
32 views