NL-based Software Engineering (NLBSE) '25

NL-based Software Engineering (NLBSE) '25
April 27th, 2025, 09:15 - 15:30 UTC-4
https://nlbse2025.github.io/

1. NL-based Software Engineering (NLBSE)
'25
April 27th, 2025, 09:15 -15:30 UTC-4
https://nlbse2025.github.io/
@NLBSE_workshop
Sebastiano
Panichella
Andrea
Di Sorbo
Maliheh
Izadi

2. General format
Presentations:
Live presentations
In-person: Room 214
Questions:
After each presentation we have allocated time for Q & A.
Participants:
-We encourage you to ask questionsto the speakers

3. Participation (Paper Authors)
10 submissions, 6 were accepted
6 tool competition papers, 1 withdrawn

4. Participation (PC Members)

5. Thanks to our sponsors!
IEEE Technical Community on Software Engineering
(TCSE)
ACM Special Interest Group on Software Engineering
(SIGSOFT)

6. Schedule (UTC-4)
09:15 -9:30 → Opening
09:30 -10:30 → Live Keynote: "Breaking the Silence: the Threats of
Using LLMs
in Software Engineering"
byAnnibale Panichella &June Sallou
10:30 -11:00 → Break
11:00 -12:30 → Session: Research Papers
12:30 -14:00 → Lunch
14:00 -15:10 → Tool Competition
15:10 -15:20 → Closing
Room 214
https://conf.researchr.org/program/icse-2025/program-icse-2025/ https://nlbse2025.github.io/

7. Keynote (09:30)
Breaking the Silence: the Threats of Using
LLMs in Software Engineering
LargeLanguageModels(LLMs)aretransformingtheSoftware
Engineering(SE)landscape,influencingtaskssuchascode
generation,programrepair,andtestcasecreation.Whilethese
modelspromiseenhancedproductivityandinnovation,theyalso
introducesignificantchallenges.Issueslikedataleakage,closed-
sourcemodels,andthereproducibilityofresultsraisecritical
concernsforresearchersandpractitionersalike.Inthiskeynote,
wewillexploretheevolvingroleofLLMsinSE,sheddinglighton
thehiddencomplexitiesandpotentialrisksthatcomewiththeir
use.Moreimportantly,wewillpresentpracticalguidelinesforboth
researchersandindustryprofessionalstoensurerigorous,
transparent,andreproducibleLLM-basedresearch.Bydrawingon
real-worldexamplesandbestpractices,thistalkaimstosparka
broaderdiscussiononhowwe,asacommunity,canresponsibly
harnessthepowerofLLMsforsoftwareengineering.
Bios:
AnnibalePanichellaisanassociateprofessor
atDelftUniversityofTechnology(TUDelft)in
theNetherlands.WithinSERG,heleadsthe
Computation IntelligenceforSoftware
EngineeringLab(CISELab)andfocuseson
advancingthefieldsofsoftwaretesting,
search-basedsoftwareengineering,software
testingforAI,andempiricalsoftware
engineering.
JuneSallouisanAssistantProfessorin
Sustainable Software Engineering at
Wageningen University&Research,inthe
InformationTechnologyresearchgroup.

8. Break (10:30 –11:00)

9. Research papers
Full papers (20 minutes):
-15 minutes for talk
-5 minutes for questions
Short papers (15 minutes):
-10 minutes for talk
-5 minutes for questions
Position papers (10 minutes):
-5 minutes for talk
-5 minutes for questions
Tool papers (10 minutes):
-7 minutes for talk
-3 minutes for questions

10. Research papers: schedule (11:00)
Analyzing Toxicity in Open Source Software Communications Using Psycholinguistics and Moral Foundations Theory
(full) Ramtin Ehsani, Shadi Rezapour and Preetha ChatterjeeDrexel University (virtual/video presentation)
SkillScope: A Tool to Predict Fine-Grained Skills Needed to Solve Issues on GitHub(short)
Benjamin C. Carter
1
, Jonathan Rivas Contreras
1
, Carlos A. Llanes
1
, Pawan Acharya
2
, Jack Utzerath
1
, Adonijah O. Farner
1
,
Hunter Jenkins
1
, Dylan Johnson
1
, Jacob Penney
2
, Igor Steinmacher
2
, Marco A. Gerosa
2
and Fabio Santos
3
[1] Grand Canyon
University, [2] Northern Arizona University, [3] Colorado State University
Synthesizing Access Control Policies using Large Language Models (short) (virtual/video presentation)
Adarsh Vatsa, Pratyush Patel and William EiersStevens Institute of Technology
Semantic API Alignment: Linking High-level User Goals to APIs(short)
Robert Feldt
1
and Riccardo Coppola
2
[1] Blekinge Institute of Technology, [2] Politecnico di Torino
Evaluating Large Language Models in Exercises of UML Use Case Diagrams Modeling (short)
Giacomo Garaccione, Pablo Federico Vega Carrazan, Riccardo Coppola and Luca ArditoPolitecnico di Torino
Automating Benchmark Generation for LLMs in Software Engineering: Challenges and Opportunities (position)
Nimrod Busany
1
, Hananel Hadad
1
, Gil Rosenblum
1
, Zofia Maszlanka
2
, Rohit Shelke
3
, Okhaide Akhigbe
3
and Daniel Amyot
3
[1] Accenture, [2] Avanade, [3] Ottawa University

11. Lunch (12:30)

12. Tool competition (14:00)
Competition reports:
-10 minutes per paper (inclusive)
Tool Competition Co-chairs
Tool Chairs

13. Tool Competition schedule
Opening
Giuseppe Colavito
1
, Pooja Rani
2
, Ali Al-Kaswan
3
and Natalia Stulova
4
[1] University of Bari, [2] University of Zurich, [4] Delft University
of Technology, [4] MacPaw
Code Comment Classification with Data Augmentation and Transformer -Based Models
Mushfiqur Rahman
1
and Mohammed Latif Siddiq
2
[1] Bangladesh University of Engineering and Technology, [2] University of Notre
Dame
GRAPHiC: Utilizing Graph Structures and Class Weights in Code Comment Classification with Pretrained BERT
Models
Pir Sami Ullah Shah, Shahela Saif, Muhammad Haris Athar, Muhammad Riyaan Tariq and Abdur Rehman Afzal National University
of Computer and Emerging Sciences
Evaluating the Performance and Efficiency of Sentence-BERT for Code Comment Classification
Fabian C. Peña and Steffen HerboldUniversity of Passau
Optimizing Deep Learning Models to Address Class Imbalance in Code Comment Classification
Moritz Mock, Thomas Borsani, Giuseppe Di Fatta and Barbara Russo Free University of Bozen-Bolzano
CodeComClassify: Automating Code Comments Classification using BERT -based Language Models
Khubaib Amjad Alam
1
, Wajid Ali
2
, Nadeem Abbas
3
, Muhammad Haroon
2
, Summan Aziz
2
, Meer Hashaam Khan
2
and Zahoor Ahmad
2
[1] College of Engineering, Al Ain University, [2] National University of Computer & Emerging Sciences, [3] Linnaeus University
Closing
Giuseppe Colavito
1
, Pooja Rani
2
, Ali Al-Kaswan
3
and Natalia Stulova
4
[1] University of Bari, [2] University of Zurich, [4] Delft University
of Technology, [4] MacPaw
Tool Chairs

14. Closing (15:10)

15. Closing (15:10)
Allregistered participants specifically for NLBSE
Sebastiano

16. Thanks to: All of you
for joining!

17. Thanks to: Our Keynote Speakers
for giving a stimulating, enlightening, and instructive talk!

18. Thanks to: the Tool Competition Co-chairs
for organizing an exciting and relevant tool competition!

19. Thanks to: our Web Chairs
for their support with the website!

20. Thanks to: the Program Committee members
for their support in reviewing papers!

21. What’s Next?
Special issue at Science of Computer Programming:
“NLBSE’25: Natural Language-based Software to
Support Software Engineering Processes”
Open Call!
Short papers with a great focus on software and replication packages
Submission Dates: October 1st, 2025
Slides of our Workshop will be made available in the webpage
Link to submission page will be posted on Twitter and the NLBSE web page

22. What’s Next?
•Createthe Steering Committee of NLBSE
•Involvenew people in the organizing committee of the workshop.
•Organizea symposium exploring the intersection of language and code
models to foster knowledge sharing and community growth.
•Involvemore industrial subjects and practitioners.
•Encouragethe design, implementation, and public availability of usable and
high-quality tools to deal with NLBSE-related challenges.
•We are generally open to ideas or new NLBSE tool competition/challenges
(contact us)!

23. We are looking for new chairs for
NL-based Software Engineering (NLBSE)
'26
Giuseppe
Colavito
Nataliia
Stulova
?

24. Dinner
(NLBSE, ICSR
and MO2RE)
CaféNostalgica(https://nostalgica.ca),
603CumberlandStreet,Ottawa.
About1.1km/17minwalkfromthe
conventioncenter.Pleasearrive
between18:00and18:30.

25. Thank you all for participating!
See you next year in Rio de
Janeiro
at NLBSE 2026!