Test Smells Learning by a Gamification Approach

The presence of test smells related to low-quality test cases is a known factor contributing to problems in maintaining both test suites and production code. The need to avoid and �x test smells is becoming more and more popular in the scientific community, as well as the importance of knowing how to detect and refactor existing test cases. However, these practices are very little considered in academic testing courses, due also to the difficulty of making them
attractive to students. This position paper presents an
approach for teaching test smells based on gamification. It exploits a tool, TSGame, that provides a serious game where students can familiarize with test smells by practicing with their detection and removal from JUnit test code. TSGame has been implemented as
a Web-based application that allows a teacher to assign students test smell detection and refactoring tasks that they have to accomplish in game sessions. Upon completion of the tasks they have the
possibility to gain rewards. A preliminary version of the tool has been validated in the context of a Software Testing course at Master degree level, with 37 students that showed the tool effectiveness and usefulness for test smell learning.

1. Test Smells Learning by a Gamification Approach Anna Rita Fasolino , Porfirio Tramontana , University of Naples “Federico II” Third edition of the international workshop on Gamification in Software Development, Verification, and Validation. GAMIFY 2024 @ ISSTA 2024 Vienna, Austria, September 17th, 2024.

2. Motivations GAMIFY 2024, Vienna, September 17th, 2024

3. Test Smells Test smells negatively affect the quality of test suites and production code Their presence negatively impacts the maintainability of test suites and production code or their functionality The concept of test smells was formalized for Java test cases implemented in JUnit in 2001 by van Deursen et al. Many test smells was proposed and discussed, both in literature and by practitioners over time Several tools for the automatic detection of some test smells have been proposed GAMIFY 2024, Vienna, September 17th, 2024

4. tsDetect Recently, Peruma et al. proposed tsDetect , a test smell detection tool able to recognize 20 different types of smells tsDetect is freely available at https://testsmells.org/ It can be used directly from command line It can be easily integrated in any other tool or framework A. Peruma, K. Almalki, C. D. Newman, M. W. Mkaouer, A. Ouni e F. Palomba, «TsDetect: An open source test smells detection tool,» in ESEC/FSE 2020 - Proceedings of the 28th ACM Joint Meeting European Software Engineering Conference and Symposium on the Foundations of Software Engineering, 2020. GAMIFY 2024, Vienna, September 17th, 2024

5. Teaching test smells Differently from code smells, there are almost no academic courses providing significant space to test smells teaching and exercising We have found in literature only the experience of Aniche et al., for which a Learning Goal of their course is to write maintainable test code by avoiding well-known test code smells V. Gar ousi , B. Kucuk e M. Felderer , «What we know about smells in software test code,» IEEE Software, vol. 36, n. 3, pp. 61-73, 2019 Porfirio Tramontana, Beatriz Marín, Ana C. R. Paiva, AlexandraMendes , Tanja E. J. Vos, Domenico Amalfitano, Felix Cammaerts, Monique Snoeck , and Anna Rita Fasolino . 2024. State of the Practice in Software Testing Teaching in Four European Countries. In 17th IEEE International Conference on Software Testing, Verification and Validation (ICST) 2024. https://doi.org/10.1109/ICST60714.2024.00015 M. Aniche , F. Hermans e A. van Deursen , «Pragmatic Software Testing Education,» in Proceedings of the 50th ACM Technical Symposium on Computer Science Education (SIGCSE '19), 2019. GAMIFY 2024, Vienna, September 17th, 2024

6. ENACTEST ENACTEST is an ERASMUS+ European Project involving a consortium wit h four academic partners GAMIFY 2024, Vienna, September 17th, 2024

7. Goals of the Enactest project GAMIFY 2024, Vienna, September 17th, 2024

8. Objective Our objective is to provide a teaching capsule based on a gamification approach supporting teaching and exercising for: Understanding test quality Recognizing test smells Fixing test smells by refactoring test code GAMIFY 2024, Vienna, September 17th, 2024

9. Our contribution: TSGame We have implemented a tool called TSGame supporting students in: practicing test smells detection practicing test smell refactoring sharing with other students the experiences made in refactoring existing test smells and receive their comments on the proposed solutions GAMIFY 2024, Vienna, September 17th, 2024

10. TSGame in a nutshell Interactive Learning Environment Learning objective : Raise students' awareness about the test quality problem, by stimulating them in identifying and removing test smells  Subjects : Student of a Software Testing course Prerequisites : Java programming, JUnit programming, knowledge about unit test quality and test smells GAMIFY 2024, Vienna, September 17th, 2024

11. TSGame offered Features GAMIFY 2024, Vienna, September 17th, 2024

12. List of Supported Test Smells Assertion Roulette Conditional Test Logic Constructor Initialization Default Test Dependent Test Duplicate Assert Eager Test EmptyTest Exception Catching Throwing General Fixture IgnoredTest Lazy Test Magic Number Test Mystery Guest Print Statement Redundant Assertion Resource Optimism Sensitive Equality Sleepy Test Unknown Test Verbose Test We considered the complete list of smells found by tsDetect by static analysis of the source code of the test class and of the class under test GAMIFY 2024, Vienna, September 17th, 2024

13. Test Smell Detection Quiz It is recommended that the student has learned the list of Test Smells from other teaching materials and now assesses his knowledge by the Quiz Given a set of JUnit test class ( including Test Smells ) , the student has to guess which test smells they contain TSGame presents to the student the code of a Class Under Test and the JUnit Class to be analysed The student detects the Smell At the end, a feedback about the correctly detected smells is presented to the student This feature can be carried out: As a first learning scenario before learning fixing test smells As an auto- assessment after practicing with test smells refactoring and solution sharing GAMIFY 2024, Vienna, September 17th, 2024

14. Test Smell Detection Quiz A list of possible Test Smells to be checked is proposed by the Tool GAMIFY 2024, Vienna, September 17th, 2024

15. Test Smell Refactoring The student has to refactor the code of a test class in order to remove its Test Smells The student has to avoid reducing the code coverage of the original Test Class After the student submits the refactored code, the Game provides a feedback, showing the updated number of test smells present in the updated version of the code GAMIFY 2024, Vienna, September 17th, 2024

16. 1. The student can edit the code of the Junit Test and Compile them 2. The Tool provides the Compiler and Test Execution output , showing the List of present Smells , and a Score 3. The student can iteratively edit the code. The Score reports the current number of Smells Test Smell Refactoring GAMIFY 2024, Vienna, September 17th, 2024

17. Browsing of Test Smell Refactoring solutions The student can browse existing solutions of Test Smell refactoring made by other students and rate/ comment / improve them GAMIFY 2024, Vienna, September 17th, 2024

18. TSGame Architecture Web- Based Architecture Seven main components (Front-End, TSDetect , Compiler, RefactoringChecker , Authentication Service, TestSmell Solution Service, ExerciseRepository Service) We have two different deploy solutions Thin -Client ( all the components are deployed on the Server Side) Fat - Client (Front-End, TSDetect , Compiler, RefactoringChecker are deployed on the Client Side) GAMIFY 2024, Vienna, September 17th, 2024

19. Deploy : Browser Based GAMIFY 2024, Vienna, September 17th, 2024

20. Deploy : Desktop Game GAMIFY 2024, Vienna, September 17th, 2024

21. Validation A preliminary version of the tool was proposed in december 2023 to students of the Software Testing Course of the Master Degree in Computer Engineering of the University Federico II the students had a 2-hours traditional lecture presenting test smells basics, a taxonomy of test smells, and examples of how to remove them by refactoring; the students were presented the TSGame app in a practical 1-hour lecture , where they learned how to use the TSGame features; the students were assigned a Game Session homework requiring to solve a realistic test smell refactoring problem on a Java class using TSGame . There was no specific time constraints except that the exercise had to be completed in two weeks; after playing the game, the students answered a post- questionnaire designed to collect their perception about the usefulness and the usability of TSGame for learning test smells and to confirm the refactoring results achieved and the actual effort dedicated to complete the task. GAMIFY 2024, Vienna, September 17th, 2024

22. Validation Objects Assignment Class 1 #Smells Class 2 #Smells Total Smells 1 FTPFileTest.java 225 WeakHashtableTest.java 34 259 2 IntHashMapTest.java 221 HSLColorTest.java 45 266 3 FontInfoTest.java 210 ChunkedLongArrayTest.java 55 265 4 ByteVectorTest.java 170 TimeStampTest.java 57 227 5 ParameterParserTest.java 157 SubjectParserTest.java 66 223 6 ByteArrayHashMapTest.java 147 XMLParserTest.java 77 224 7 XmlElementTest.java 146 InflectionTest.java 77 223 8 RangeTest.java 124 HierarchyPropertyParserTest.java 81 205 9 ImprovedStreamTokenizerTest.java 123 RationalNumberTest.java 123 246 10 WeakHashtableTest.java 34 FTPFileTest.java 225 259 11 HSLColorTest.java 45 IntHashMapTest.java 221 266 12 ChunkedLongArrayTest.java 55 FontInfoTest.java 210 265 13 TimeStampTest.java 57 ByteVectorTest.java 170 227 14 SubjectParserTest.java 66 ParameterParserTest.java 157 223 15 XMLParserTest.java 77 ByteArrayHashMapTest.java 147 224 16 InflectionTest.java 77 XmlElementTest.java 146 223 17 HierarchyPropertyParserTest.java 81 RangeTest.java 124 205 18 RationalNumberTest.java 123 ImprovedStreamTokenizerTest.java 123 246 GAMIFY 2024, Vienna, September 17th, 2024 The classes under test was from the SF-110 repository The test classes were written by students while playing Code Defenders Arcuri, G. Fraser and A. 2014. "“A large scale evaluation of automated unit test generation using evosuite ”." ACM Transactions on Software Engineering and Methodology (TOSEM), 24(2). José Miguel Rojas, Thomas White, Benjamin Clegg and Gordon Fraser. 2017. "“Code Defenders: Crowdsourcing Effective Tests and Subtle Mutants with a Mutation Testing Game,”." Proc. of the International Conference on Software Engineering (ICSE). IEEE. 677-688.

23. Results 22 out of 36 students reached in fixing all the test smells without losses in coverage. 28 out of 36 students ( 78% ) reached to fix all the test smells found by TSGame , whereas 4 of the remaining ones fixed all the smells except one or two and the last 4 students missed in refactoring a larger number of smells. The refactoring activities carried out by 28 out of 36 students did not cause reductions in the achieved coverage, whereas in the other 8 cases the achieved coverage reduced of amounts between 2% and 26%. The effort of this task varied from 30 minutes to 12 hours, with an average of about 3.4 hours . Some of the students declared problems about Eager Test smells (14 students) whereas only two of ones mentioned problems in fixing Sensitive Equality smell, and only one of them encountered problems in removing Exception Catching Test Smell. GAMIFY 2024, Vienna, September 17th, 2024

24. Perceived Satisfaction GAMIFY 2024, Vienna, September 17th, 2024 The same students solved another refactoring task without the support of TSGame The perceived satisfaction was evaluated by a post-questionnaire according to a five-value Likert scale (Strongly agree = 5. Agree =4, Neutral =3, Disagree=2, Strongly Disagree = 1).

25. Conclusions and Future Works GAMIFY 2024, Vienna, September 17th, 2024