Pragmatic look at education and training of software test engineers: Further cooperation of academia and industry is needed

Presented in: The Third International Software Testing Education Workshop (TestEd 2024), co-located with the 17th IEEE International Conference on Software Testing, Verification and Validation (ICST 2024),
May 28, 2024
Toronto, Canada

1. Pragmatic look at education and training of software test engineers: Further cooperation of academia and industry is needed Presented in: The Third International Software Testing Education Workshop (TestEd 2024), co-located with the 17th IEEE International Conference on Software Testing, Verification and Validation (ICST 2024), May 28, 2024 Toronto, Canada

2. About us – with extensive international experience Started his software engineering / testing career in 2012 Has been providing both industry and academic test training since 2020 Roughly: 80% industrialist (practitioner), 20% academic (lecturer) Work experience in: Started his software engineering / testing career in 1998 Wrote his first automated test case using IBM Rational Tester in that year. Roughly: 60% academic, 40% industrialist (practitioner) Has been providing both industry and academic test training since 2004 Work experience in: Model-based testing training, Istanbul, Turkiye, 2019

3. Background, and Goal of this paper A 2019 survey paper compiled the set of 204 papers published in the area, published between 1992-2018 --- some past works of the authors below: This current work takes a “pragmatic” high-level overview on the issue p resenting a number of high-level observations and challenges, and recommendations using evidence from our experience hopes to trigger further discussions in the community, between industry and academia, on how to further improve the status-quo and to find further best practices for more effective education and training of software testers Discussions are based on: … combined ~40 years of the two authors’ technical experience in test engineering, and their ~30 years of experience in providing testing education and training in more than six countries.

4. Context of software-testing education, training and certification This is quite a large and complex model ;) Let’s review piece by piece… At the center , is: Most of the papers in the community are about this “path” This work takes a “pragmatic” high-level overview on the issue: P resenting a number of observations and challenges, from our experience (combined 40 years) Competency profiling of software-testing knowledge and skills Recommendations

5. A number of observations / challenges Software-testing education in academia: (-) Tendency to focus on theoretical concepts (some educators see this as a good thing, but not others) - Also practical exercises are often rather small-scale (-) Largely non-uniform, i.e., contents’ coverage and depth can vary significantly from university to university and by the educator (who is teaching).  One reason why the ISTQB certificate has been very popular and in-demand (is uniform) Software-testing training and certification in industry: (+) Compared to academia, contents coverage and depth are more uniform / standardized (especially in the ISTQB certificates). This often has many advantages, e.g., everyone can use the same vocabulary, and when a tester has gained a certificate, employers already know her/his level of knowledge quickly (-) Although important testing concepts such as test-case “design” techniques are covered in trainings, e.g., ISTQB, those techniques are not actively encouraged for usage, and thus many certified testers do not (properly) use those techniques (-) Due to the above mindset, most testing job postings and interviews largely focus on test “tools”. When practitioners talk about testing, the discussions tend to mostly focus on which tools to use, but not “how” to properly use it, how to design test cases, etc. (+) ISTQB certifications have been established quite well in some countries, in the hiring process of many companies; and many employers / testers are already happy with the certification scheme, and its impact in the career path, etc.

6. A number of observations / challenges, about both camps combined (academia and industry) (-) The two groups (“camps”) prefer to continue operating (training testers) separately (it is a sad reality). Aside from separately educating and training testers, it is also an unfortunate reality that, industry and academia (overall in the world) do not cooperate actively in other activities: neither in research, nor in conferences In some countries / regions (e.g. Scandinavia), the situation is quite positive – there are active collaborations (-) Due to non-uniform software-testing education in academia and highly fragmented streams of testers’ entry to industry, industry is providing its own training (ISTQB) from “scratch”, as if university testing education does not exist

7. Context of software-testing education, training and certification This work takes a “pragmatic” high-level overview on the issue: P resenting a number of observations and challenges, from our experience (combined 40 years) Competency profiling of software-testing knowledge and skills Why?  To better characterize Software-testing education in academia; versus its training and certification in industry Recommendations

8. Competency profiling of software-testing knowledge and skills Why?  To better characterize Software-testing education in academia; versus its training and certification in industry There are MANY syllabi (syllabuses; what should be taught) for software testing, at least three: Software Engineering Body of Knowledge (SWEBOK) v3.0, 2014 Software testing competencies of the Software Engineering Competency Model (SWECOM) v1.0, 2014 v4.0, 2024

9. Competency profiling of software-testing knowledge and skills How to objectively quantify / assess one’s knowledge versus skill level? A lot of work has been done in other fields to characterize and quantify competency, e.g., Individual Competency Index (ICI) – below: www.bimframework.info/competency Building Information Modelling (BIM) 

10. Competency profiling of software-testing knowledge and skills (Or a human tester with low knowledge in both aspects) Can consider two perspectives: If a learner fully learns everything taught in the course, the two would be the same and different, if not

11. Competency profiling of software-testing knowledge and skills – Typical industry training

12. Competency profiling of software-testing knowledge and skills – Impossible to teach both aspects in depth

13. Competency profiling of software-testing knowledge and skills – all cases together Ironically, certifications such as ISTQB are mostly theoretical study and exams. ISTQB essentially “confirms” the theoretical knowledge of a tester, for recruiters, and then s/he needs to take further training in applied skills, mainly test automation. * *

14. Context of software-testing education, training and certification Most of our papers in the community are about this “path” This work takes a “pragmatic” high-level overview on the issue: P resenting a number of observations and challenges, from our experience (combined 40 years) Competency profiling of software-testing knowledge and skills Summary, Recommendations and Discussions

15. Summary, Recommendations, Discussions, Q/A Summary: The situation is quite OK ! Academia is providing testing education as it can, and industry continues the training and when it can, and when needed, it “adds up” on top of university education Consider 4 years of undergraduate versus ~30-40 years of professional career of a typical software engineer Recommendations We should consider to have the “right” mix of theory-practice in both university and industry courses Roles of academia and industry: Would be better to see further symbiotic collaboration of academia and industry in this endeavour to… ensure effective education of young test engineers who possess “pragmatic” testing theory, plus skills on how to conduct different types of testing using modern test tools and processes Discussions : It’s time for interaction discussions – Q / A Calgary Baku Ankara Belfast Cities that this work has some forms of roots in … Ottawa İ stanbul