Troubleshooting, Monitoring and Profiling with Java Flight Recorder, Mission Control and Cryostat

Slides from JConfdev 2024
Java Flight Recorder and JDK Mission Control enable continuous collection of low level, detailed runtime information for incident analysis. JFR is included with the JDK, has a tiny footprint, and Java Flight Recorder is a profiling and event collection framework built into the JDK. JDK Mission Control contains a set of tools that enable detailed analysis of the data collected by Java Flight Recorder. Cryotstat enables secure managemant of JFR recordings for containerized Java apps. In this presentation, we will dive into Java Flight Recorder, Java Mission Control, and Cryostat. You will leave knowing how start, stop, and visualize data about your Java applications running in and out of containers.

1. Update confidential designator here
Version number here V00000
1
JDK Flight Recorder
and Cryostat

2. Update confidential designator here
Version number here V00000
Me
2
Jeremy Davis
Chief Architect, Red Hat
[email protected]

@argntprgrmr
(Threads, Instagram, x, Twitter, Bluesky)
@[email protected] (Mastodon)

3. Update confidential designator here
Version number here V00000
3
JDK Flight Recorder
and Cryostat

Jeremy Davis
Chief Architect, Red Hat
[email protected]
@argntprgrmr (Threads, Instagram, x, Twitter)
@[email protected]

4. Update confidential designator here
Version number here V00000
The TQM Guy
4
“Without data you are just
another person with an
opinion”
W. Edwards Deming

5. Update confidential designator here
Version number here V00000
It’s old
5

6. Update confidential designator here
Version number here V00000
We’re not in JRockit anymore!
6
JDK Flight Recorder

7. Update confidential designator here
Version number here V00000
Really 3 Things
7
Source:
Insert source data here
Insert source data here
▸JFR (JDK Flight Recorder)
▸jcmd
▸JMC (Java Mission Control)
Toolkit

8. Update confidential designator here
Version number here V00000
8
It’s Been Around a Long Time
Background
▸BEA JRockit Flight Recorder
▸Oracle JRockit
▸2018 OpenSource

9. Update confidential designator here
Version number here V00000
9
Deets
Details
▸Event Based
▸Events aggregated as “recordings”
▸API’s for producing and analyzing events and
event data
▸CLI, GUI, or custom
▸Designed for production

10. Update confidential designator here
Version number here V00000
10
Seriously!
Designed for Production
▸Designed for production
▸JVM Event (occur anyway)
▸Events saved into thread local buffers
▸Copied to global buffers
▸Written to disk or overwritten

11. Update confidential designator here
Version number here V00000
11
Real numbers
Production
▸Continuous capture works for occasional bugs
▸Terabytes of JFR data per minute
▸5 MiB per minute (2 MiB compressed)
▸< 2% overhead

12. Update confidential designator here
Version number here V00000
12
Events Are Core
Events
▸Event Id
▸Timestamp (CPU ticks)
▸Duration (CPU ticks)
▸Thread Id
▸Stack Trace Id
▸Event Specific Payload

13. Update confidential designator here
Version number here V00000
13
How to
Recording
▸java -XX:StartFlightRecording
▸java -XX:StartFlightRecording:filename=myrecording.jfr,dumponexit=true
▸jcmd 60128 JFR.start
▸jcmd 60128 JFR.dump name=1 filename=60128.jfr

14. Update confidential designator here
Version number here V00000
Java Mission Control

Making Sense
14

15. Update confidential designator here
Version number here V00000
cryostat.io
15
Cryostat

16. Update confidential designator here
Version number here V00000
16
cryostat.io
Cryostat UI
▸Dashboard
▸Automated Analysis
▸Grafana
▸Automated Rules
▸Recordings
▸Archive

17. Update confidential designator here
Version number here V00000
17
Recordings

18. Update confidential designator here
Version number here V00000
18
Automated Reports

19. Update confidential designator here
Version number here V00000
19
Event Types

20. Update confidential designator here
Version number here V00000
20
Event Templates

21. Update confidential designator here
Version number here V00000
Grafana
Who doesn’t love a good dashboard?
21

22. Update confidential designator here
Version number here V00000
Finally
22
You Can Leave It Running
< 1% overhead
Make Custom Events!
Kubernetes == Cryostat
All the goodness in k8s

23. Update confidential designator here
Version number here V00000
It won’t work (on a Mac) without this!
23
…
<array>
<string>-keyring</string>
<string>~/.eclipse_keyring</string>
<string>-vm</string>
<string>/Users/jeremyrdavis/.sdkman/candidates/java/current/bin/java</string>
</array>
…
/Applications/JDK\ Mission\ Control.app/Contents/Info.plist

24. Update confidential designator here
Version number here V00000
Links
Agenda
24
▸https://docs.oracle.com/javacomponents/jmc-5-4/jfr-runtime-guide/toc.htm
▸https://cryostat.io
▸https://www.youtube.com/watch?v=XEKkUpPnf4Q
▸https://github.com/theygreystone/jmc-tutorial
▸https://developers.redhat.com/blog/2020/08/25/get-started-with-jdk-flight-recorder-in-openjdk-8u
▸https://www.youtube.com/watch?v=oPCTkW7S7T4
▸https://inside.java/2021/02/22/podcast-013/