How AI is impacting Software Engineering

Artificial Intelligence is reshaping how we design, develop, test, maintain and operate the software systems that underpin our modern life. In this seminar, Dr. Vahid Garousi will explore how AI is transforming software engineering, from helping developers write and test code faster to enabling entirely new ways of ensuring software reliability and quality. Drawing on global industry trends, his own ongoing research in this field via various ongoing European R&D projects, and collaborations with leading companies, he will share insights about how and where AI is being applied in software engineering. The session will highlight opportunities and risks: what software firms stand to gain, what challenges remain, and how software engineers’ roles are evolving in the age of AI. The seminar will also discuss potential opportunities for collaboration between academia (QUB) and local industry in NI, including technology transfer and the development of joint follow-up projects. This talk will offer practical takeaways for industry leaders, policymakers, and software engineers on how to harness AI responsibly in the engineering of software that increasingly runs our world.

1. 1 ✉ [email protected] 🌐 www.vgarousi.com

2. The speaker: Profile and international experience 2 International work e xperience : Have worked as a Educator, Researcher and Professor in four countries: Queen’s University Belfast, UK , since 2019 Invited Visiting Professor, Azerbaijan Technical University, since 2025 Wageningen University, Netherlands , 2017-2019 Hacettepe University , Ankara, Türkiye , 2015-2017 University of Calgary, Canada, 2006-2014 ... Vision / interest in Software Engineering : Working in the intersection of industry academic collaborations: Since 1998, my vision has been to link industry and academic in Software Engineering research and education --- innovating better approaches to Software Engineering --- training the Software Engineers of tomorrow 50+ industry partners, over 30+ R&D projects, research grants combined over 15+ million GBP. I also provide consulting on a regular basis Helping software companies develop and test software systems more effectively and efficiently. 2013 2019 2015 2009 Citizens of: Azerbaijan, Canada, UK

3. Please introduce yourselves… 3

4. Opening Dialogue AI is making fundamental shifts in every phase of Software Engineering (SE) Fact: AI is impacting (revolutionizing) every discipline Many GenAI / LLM choices … Also, we need to differentiate general-purpose versus specific-purpose AI tools for SE Let’s see an example interaction between human (software engineer)-AI: 4

5. Opening Dialogue 5 Thinking longer for a better answer

6. Opening Dialogue So much (mixed) discussions ( + / - ) are going on this topic  6 In favour of AI in SE Critical of AI in SE Critical of AI in SE Critical of AI in SE Critical of AI in SE

7. We will frame the talk and the Q/A session using these questions Which Software Engineering (SE) tasks are you already using AI for, and to what extent? How much do you trust the quality of AI-generated artifacts (e.g., code, test cases, documentation, requirements, etc. )? Do you agree that AI is making software engineers more productive (but also less skilled) over time? Which research collaboration opportunities can we identify between QUB researchers and NI software industry in AI-assisted SE? 7

8. AI across the SDLC AI can help in every phase of the SDLC, not just for coding Q1-Which Software Engineering (SE) tasks are you already using AI for, and to what extent? 8

9. Using AI in Software Engineering Sharing experience from one project: Context and need: Our industry collaborators wanted to find and use the “right” AI-powered software testing tools, and asked for our expertise Work done: We conducted a systematic review, identified 66 AI-powered software testing tools in the market (by end of 2024), categorized their features, and assessed two of those tools, in pilot studies in depth 9

10. Using AI in Software Testing We assessed two of those tools, in pilot studies: Tool 1-Parasoft Selenic: AI feature for self-healing of UI test-code  Motivation for this feature: Updating million lines of automated test code as the production code of software changes, costs companies millions of $£ each year Tool 2-SmartBear VisualTest: AI feature for visual testing  Motivation for this feature: Having humans do visual verification of UI screens costs millions of $£ Our empirical assessments: Results: + AI-powered testing can speed up test execution and reduce maintenance effort - But still struggles with complex UI changes, have limited contextual understanding, and erroneous outputs caused by AI misclassifications. - Testers must carefully review, inspect and correct AI-generated artifacts, reducing the net efficiency gains promised by these tools 10 Read full-text PDF in: arxiv.org/abs/2409.00411

11. Maturity models for AI-powered SDLC Maturity models are everywhere… (in every subject) Do you know of any? Maturity models are also being proposed for assessing maturity of using AI across the SDLC. Let’s review a few of them next 11

12. Maturity models for AI-powered SE A “very” comprehensive MM  An example SDLC phase: 12 defra.github.io/ai-sdlc-maturity-assessment/

13. A new maturity model, developed by us recently at QUB We are currently applying it in two industrial settings … The partners have already started to see the benefits Baseline and benchmarking. Organizations get an objective picture of where they stand, and can reuse for future comparisons. Actionable pointers. By exposing dimension-specific strengths and weaknesses, the model highlights concrete areas for improvement Cultural signal. The assessment itself is viewed as a sign that AI adoption is taken seriously at both the individual and organizational levels 13

14. Quality of AI-generated artifacts (e.g., code, test cases) Gettings tons of low-quality stuff from AI fast is not particularly a useful situation! Many causes could lead to poor quality of AI-generated SE artifacts, e.g., hallucinations, not providing enough detail / context by the software engineer to the AI Two insights: Quality of prompts and the provided “context” often corelates with the quality of AI-generated SE artifacts AI often does better job in generating / analysing small pieces of SE artifacts (e.g., small code blocks), and often would start to do poor job when the size of SE artifacts increase. Our role as “integrators” Question 2-How much do you trust the quality of AI-generated artifacts (e.g., code, test cases)? Also, what ratio of errors have you observed in AI-generated SE artifacts? 14

15. Quality of AI-generated artifacts (e.g., code, test cases) Lots of studies are being done on this issue, in both academia and industry An example study: 15 Mostly Basic Python Programming (MBPP)

16. Man versus machine: AI versus Software Engineers Question 3-Do you agree that AI is making software engineers more productive (and less skilled) over time? AI can make software engineers more productive, but it depends, “there is a catch”  Cost vs. benefit – Advantages vs. disadvantages 16

17. Let’s collaborate … Question 4-Which research collaboration opportunities can we identify between QUB researchers and NI software industry in AI-assisted SE? [Are there issues that you want to better understand and/or improve in this area?] [Some generic ideas below] Approaches and heuristics for increasing quality of AI-generated code Prompt engineering for software testing and QA Understanding cognitive-debt in AI-assisted SE Human-in-the-loop SE governance framework AI-supported requirements and design exploration Industrial training and upskilling in AI-assisted SE Benchmarking NI’s AI-adoption maturity in SE In-house training of LLM models for SE Responsible AI policies and standards for SE practice Empirical studies on productivity vs. cognitive debt in AI-assisted SE … End of the talk 17

18. backup 18

19. Man versus machine: AI versus Software Engineers Question 4-How do you see the role of software engineers evolving in the age of AI? 19

20. Maturity models for AI-powered SDLC (#1) 20 medium.com/@rama.sathish/ai-powered-software-development-life-cycle-1ac599ad38bb