Modest-Pharo: Unit Test Generation Based on Traces and Metamodels

Talk from IWST 2024: "Modest-Pharo: Unit Test Generation Based on Traces and Metamodels"

PDF: http://archive.esug.org/ESUG2024/iwst-day2/06-Modest-Pharo,%20Unit%20Test%20Generation%20Based%20on%20Traces%20and%20Metamodels.pdf

1. 111
Evref
fervE
Modest-Pharo:
Unit Test Generation
Basedon Traces and Metamodels
Gabriel Darbord¹
Fabio Vandewaeter¹
Anne Etien¹
Nicolas Anquetil¹
Benoit Verhaeghe²
¹Univ. Lille, CNRS, Inria, Centrale Lille, UMR 9189 CRIStAL, F-59000 Lille, France
²Berger-Levrault, France
IWST 2024
EvreffervE

2. 222
Evref
fervE
The Importance of Testing
§Nowadays, whendeveloppingnew software systems:
§20-50% time spenton testing
§Wetest becausewewant:
§Bug detectionand prevention
§Qualityassurance
§User satisfaction
§Non-regression
§Confidence
§Etc.

3. 333
Evref
fervE
Our Test GenerationApproach
§Usingsoftware modelsand executiontraces
§staticand dynamicanalysis
§Our objective isto generatetests thatare:
§Relevant
§Readable
§Maintainable
§Not requiringexistingtests
§Not contaminating

4. 444
Evref
fervE
Test Oracles
§How can we verify that a program returns the correct answer?
§Mechanism that determines whether a test has passed or failed
§Oracles hold the “truth”
§In our case:
§Consider legacy to be correct
§Capture behavior using traces
§Verify updated behavior matches traces
→Non-Regression Testing

5. 555
Evref
fervE
Example of a Generated Test

6. 666
Evref
fervE
ApproachSteps
Codebase
Traces
Value
Model
Code
Model
Unit Test
Model
Test Suite
modeling parsing exporting
building
tracing
1
2
3
4
5

7. 777
Evref
fervE
Modeling with the Moose Platform
§Moose is a platform for software analysis
§It allows to:
§Represent a software system in a model
§Query, manipulate, transform, and visualize models
1.Modeling
2.Tracing
3.Parsing
4.Building
5.Exporting

8. 888
Evref
fervE
Tracing by Instrumentation
§MetaLink, MethodProxies...
§Before and after method
§Requisite payload:
§Identity of target method
§Serialized arguments
§Serialized return value
§Serialized receiver
1.Modeling
2.Tracing
3.Parsing
4.Building
5.Exporting

9. 999
Evref
fervE
Parsing Trace Data
§JSON based on Jackson
§Type metadata
→Dynamic typing
§Object identity metadata
→Circular dependencies
§Easy to
§write
§parse
1.Modeling
2.Tracing
3.Parsing
4.Building
5.Exporting
github.com/Modest-Project/PharoJackson

10. 101010
Evref
fervE
Building Test Model
Code
Model
Method
Parameter
Class
Unit Test
Model
TestCase
TestMethod
Arrange
Act
Assert
Value
Model
Argument
Result
1.Modeling
2.Tracing
3.Parsing
4.Building
5.Exporting

11. 111111
Evref
fervE
Exporting Test Model
§Use reflectivity to create packages, classes and methods
§Write the code using Pharo’sAST
Test
Exporter
Value
Exporter
Value Model
Test Model
Value Reconstruction AST
Test Structure AST
1.Modeling
2.Tracing
3.Parsing
4.Building
5.Exporting

12. 121212
Evref
fervE
Reconstructing Values

13. 131313
Evref
fervE
Example of a Generated Test (bis)
Existing test
Generated test

14. 141414
Evref
fervE
Results
About target projects
About generated tests

15. 151515
Evref
fervE
555
Evref
fervE
ApproachSteps
Codebase
Traces
Value
Model
Code
Model
Unit Test
Model
Test Suite
modeling parsing exporting
building
tracing
1
2
3
4
5
131313
Evref
fervE
Example of a Generated Test (bis)Existing test
Generated test
333
Evref
fervE
Our Test GenerationApproach§Usingsoftware modelsand executiontraces §staticand dynamicanalysis§Our objective isto generatetests thatare:§Relevant§Readable§Maintainable§Not requiringexistingtests§Not contaminating
999
Evref
fervE
Building Test Model
Code
Model
Method
Parameter
Class
Unit Test
Model
TestCase
TestMethod
Arrange
Act
Assert
Value
Model
Argument
Result
Conclusion