How difficult is to get a JIT right? Talk from ESGU 2024

Talk from ESGU 2024: How difficult is to get a JIT right?

PDF: http://archive.esug.org/ESUG2024/day3/02-guille-vm-profiling.pdf

1. How difficult is

to get a JIT right?
Guillermo Polito - ESUG’24
[email protected]
Evref
fervE

2. Quick About Me: Guille
2
Evref
fervE
•Pronounced giʃe (guichet in FR, ~ghisheh in EN?)
•Now: Researcher at Inria - Lille
•Pharo Contributor since ~2010
•Keywords: compilers, testing, test generation
•Interests: tooling, benchmarking, ???, board games, batman, concurrency
If any of that interests you, come talk to me!
[email protected]
@guillep

3. Debugging Assembly Code
3
AARCH64X64IA32

4. Debugging Assembly Code
Without looking at it
4
AARCH64X64IA32

5. 5
The Pharo VM
INTERPRETER
JIT COMPILER
GARBAGE
COLLECTOR
Back
Front
FFI
Concurrency
…

6. INTERPRETER
JIT COMPILER
GARBAGE
COLLECTOR
Back
Front
FFI
Concurrency
…
6
DRUID
Input
Output I am performed at VM
building time (AoT)
VM developers avoid to
write and maintain the
frontend of the JIT
compiler code
(language dependencies)
Context: Druid JIT compiler generation

7. 7
DRUID
Input
Output I am performed at VM
building time (AoT)
VM developers avoid to
write and maintain the
frontend of the JIT
compiler code
(language dependencies)
Context: Druid JIT compiler generation
INTERPRETER
JIT COMPILER
GARBAGE
COLLECTOR
Back
Front
FFI
Concurrency
…

8. JIT Compiler
Interpreter
Druid by example: the addition primitive
8

9. A Couple of Months Ago
9
INTERPRETER
JIT COMPILER
223 bytecodes and 10 primitives
225 bytecodes and 130 primitives
46%
70%

10. 10
INTERPRETER
JIT COMPILER
223 bytecodes and 10 primitives
225 bytecodes and 130 primitives
46%
70%
A Couple of Months Ago

11. Generated JIT-Compiler
11

12. Generated JIT-Compiler
12
…

13. Chameneos KNucleotide RegexDNA
Richards File Tests Kernel Tests
Opal Tests
Some Initial benchmarks

14. À la par with the interpreter
14
Chameneos KNucleotide RegexDNA
Richards File Tests Kernel Tests
Opal Tests

15. Slightly faster?
15
Chameneos KNucleotide RegexDNA
Richards File Tests Kernel Tests
Opal Tests

16. And much slower too!
16
Chameneos KNucleotide RegexDNA
Richards File Tests Kernel Tests
Opal Tests

17. Where does the time go?
17

18. Analysing Instruments Profiles
18
$ xctrace export

19. Analyzing Samples
19
Sample 1
Sample 2
…
…
Sample N

20. Analyzing Samples
20
Sample 1
Sample 2
…
…
Sample N
Stack Traces

21. Analyzing Samples
21
Sample 1
Sample 2
…
…
Sample N
Stack Traces

22. Analyzing Samples
22
Sample 1
Sample 2
…
…
Sample N
Stack Traces
Primitive from
Machine Code
Interpreter

23. Group Traces Using Heuristics
23
Sample 1
Sample 2
…
…
Sample N
JIT compilation
Interpreter
GC
Sample 1
Sample 7
Sample 2
Sample 18992
…
…
…

24. High-level VM Profile
•Time spent in
•Interpreter
•JIT compilation
•JIT compiled code
•GC
•Primitives
•…
24
Some Bench

25. Scenario 1: Cross-JIT Profiling
25

26. Hot Paths and Our Partial JIT Implementation
•Cogit is all or nothing compiler
•Hot path is not compiled!
26

27. Scenario 2: Cross-Version Profiling
27

28. Differential Profiling + Absolute Values
28
Stock Druid Interpreter Stock Druid Interpreter

29. Differential Profiling + Absolute Values
29
Worse Quality MC
Stock Druid Interpreter Stock Druid Interpreter

30. Differential Profiling
30
More time in Primitives
Stock Druid Interpreter Stock Druid Interpreter

31. Drill-down in MC -> Primitives
31
Sample 1
Sample 2
…
…
Sample N
Stack Traces
Primitives!
Primitive Samples

32. Differential MC->Primitive Profiling
32

33. Differential MC->Primitive Profiling
33
Low-hanging fruits

34. •2x faster!

than interpreter on avg
•Almost there:
•~0.7x manual JIT
•Missing
•static type predictions
•peephole optimizations on conditionals
After Some Bit of “well-placed” Work :)
34

35. What’s next?
•Linux integration:
•Perf support
•Matéo Boury
•Tracking Pharo’s performance:
•Performance dashboards
•Benchmark Generation
•daily, monthly, yearly
35

36. Takeaways
•Integrate with tools that do their job well (Instruments, Perf)
•Simple custom tools help debugging complex VM scenarios
•Tests first for good behavior
•Bench first for good performance!
36