Object-Centric Debugging: Empirical experiment - preliminary results

Talk from ESUG 2024

"Object-Centric Debugging: Empirical experiment - preliminary results"

PDF: http://archive.esug.org/ESUG2024/day4/02-ObjectCentric-Debugging.pdf

1. Object-Centric Debugging
Empirical experiment - preliminary results
ESUG - 11 July 2024
Univ. Lille, Inria, CNRS, Centrale Lille, UMR 9189 - CRIStAL
1
Valentin Bourcier, Steven Costiou, Pooja Rani, Alberto Bacchelli

2. What is object-centric debugging?
Ressia. Object-centric debugging
2

3. What is object-centric debugging?
3
Object-centric breakpoints in Pharo

4. What is object-centric debugging?
4
Object-centric breakpoints in Pharo

5. •Insights, examples
•Empirical evaluations
Object-centric debugging
- Faster debugging
- Fewer actions
Ressia. Object-centric debugging
The hypotheses
5

6. The design
Empirical evaluation: object-centric debugging
Ammolite Lights Out
6

7. The design
Empirical evaluation: object-centric debugging
Ammolite Lights Out
7

8. The design
Empirical evaluation: object-centric debugging
Ammolite Lights Out
8

9. The design
Empirical evaluation: object-centric debugging
Ammolite Lights Out
3 pilot participations
Hypothesis testing
Wilcoxon signed rank
(59 participations)
9
Time
Actions

10. The design
Empirical evaluation: object-centric debugging
Object-centric breakpoints
10

11. The design
Empirical evaluation: object-centric debugging
Object-centric breakpoints
11

12. The design
Empirical evaluation: object-centric debugging
Object-centric breakpoints
12

13. Empirical evaluation: object-centric debugging
13

14. The design
Empirical evaluation: object-centric debugging
Per-participant analysis
Object-centric breakpoints
14

15. Per-participant, N=59
Empirical evaluation: object-centric debugging
15

16. Per-participant, N=59
Empirical evaluation: object-centric debugging
No significant difference
16

17. Per-group, task-sequence analysis
Empirical evaluation: object-centric debugging
GROUP 1 GROUP 2
17

18. Per-group, task-sequence analysis
Empirical evaluation: object-centric debugging
GROUP 1 GROUP 2
Higher values on the left side
18

19. Per-group, task-sequence analysis
Empirical evaluation: object-centric debugging
GROUP 1 GROUP 2
Higher values on the left side
Negative difference Positive difference
19

20. Per-group, task-sequence analysis
Empirical evaluation: object-centric debugging
GROUP 1 GROUP 2
Higher values on the left side
Negative difference Positive difference Object-centric debugging seems to help with
Ammolite, but NOT Lights Out
REALLY?
20

21. Work-in-progress
Empirical evaluation: object-centric debugging
Per-task analysis
Object-centric breakpoints
21

22. Per-task analysis
Empirical evaluation: object-centric debugging
AMMOLITE LIGHTS OUT
Negative differencePositive difference
22

23. 23
What causes this difference?
Empirical evaluation: object-centric debugging
AMMOLITE LIGHTS OUT
Code Browser more efficient than the debugger for Lights Out?
Negative differencePositive difference

24. 24
Empirical evaluation: object-centric debugging
AMMOLITE LIGHTS OUT
Bugs with a different nature

(reported by a dozen participants)
Application
Initialisation
Application
Initialisation
Failing object
Root cause
What causes this difference?

25. 25
What’s next?
A deep dive in the data:
•Per-expertise analysis
•Robustness analysis
International Conference
on
Software Engineering
New object-centric tools
&
New experiments

26. Object-Centric Debugging
Empirical experiment - preliminary results
ESUG - 11 July 2024
Univ. Lille, Inria, CNRS, Centrale Lille, UMR 9189 - CRIStAL
Valentin Bourcier, Steven Costiou, Pooja Rani, Alberto Bacchelli