stackconf 2024 | Buzzing across the eBPF Landscape and into the Hive by Bill Mulligan

The buzz around the Linux kernel technology eBPF is growing quickly and it can be hard to know where to start or how to keep up with this technology that is reshaping our infrastructure stack. In this talk, Bill will trace how he got into eBPF, explore some of the applications leveraging eBPF today, and teach others how to dive into the hive of activity around eBPF. People just beginning with eBPF will learn how eBPF makes it possible to have efficient networking, observability without instrumentation, effortless tracing, and real-time security (among other things) without needing your own kernel team. Those already familiar with eBPF will get an overview of the eBPF landscape and learn about many new and expanding eBPF applications that allow them to harness the power without needing to dive into the bytecode. The audience will walk away with an understanding of the buzz around eBPF and knowledge of new tools that may solve some of their problems in networking, observability, and security.

1. Buzzing Across the eBPF
Landscape and Into the
Hive
Bill Mulligan | @breakawaybilly
Community Pollinator, Isovalent at Cisco

2. @breakawaybilly

3. @breakawaybilly
A Vision for eBPF Inside

4. eBPF
Buzz or Byte?

5. @breakawaybilly
Kernel Space/User Space “Paradox”
●Kernel: System awareness,
but lacks flexibility
●User space:
Programmable, but no
direct access to kernel
structures, resources
●Kernel modules: Difficult,
unsafe, not stable
Flexibility
Performance
Visibility

6. @breakawaybilly
Kernel Space/User Space “Paradox”
Can we have programmability in
the kernel?


Flexibility
Performance
Visibility

7. @breakawaybilly
What is ?

8. @breakawaybilly
What is ?
eBPF makes the Linux kernel programmable

9. @breakawaybilly
Makes the Linux kernel
programmable in a secure
and efficient way.
“What JavaScript is to the
browser, eBPF is to the
Linux Kernel”

10. @breakawaybilly
"BPF has actually been really useful, and the real power of it
is how it allows people to do specialized code that isn't
enabled until asked for"
- Linus Torvalds

11. Dynamically Change Kernel
Behavior

12. @breakawaybilly

13. @breakawaybilly

14. @breakawaybilly
“Superpowers for Linux”
- Brendan Gregg

15. @breakawaybilly
Vision

16. @breakawaybilly
eBPF is the Cloud Native App Store
https://commons.wikimedia.org/wiki/File:Steve_Jobs_presents_iPhone.jpg

17. Safe and Performant
Changes to Kernel Behavior

18. @breakawaybilly
Run eBPF programs on events
Attachment points
●Kernel functions (kprobes)
●Userspace functions
(uprobe)
●System calls
●Tracepoints
●Sockets (data level)
●Network devices (packet
level)
●Network device (DMA
level) [XDP]
●...

19. @breakawaybilly
Safety: eBPF Verifier
●Programs will run to completion
●Does not crash or otherwise harm the system
●Only priviledged users allowed to load

20. @breakawaybilly
Performance: In Kernel & JIT Compiler
●Translate program bytecode into the machine specific
instruction set to optimize execution speed of the program
●Runs as efficiently as natively compiled kernel code

21. @breakawaybilly
Performance

22. @breakawaybilly
Shared Memory: BPF maps
Between eBPF programs and between kernel and user space
●No data loss
●More performant

23. eBPF
Today

24. userspace
kernel
network call
read file
run process
podcontainer
pod containercontainer
One kernel
per host

25. @breakawaybilly
userspace
kernel
app
app
pods
networking
access files
create
containersKernel aware of
everything on the
host

26. userspace
app
kernel
app
pods
network call
read file
run process
eBPF programs
can see
everything
userspace
podcontainer
pod containercontainer

27. userspace
app
kernel
app
pods
network call
read file
run process
eBPF programs
can see
everything
No changes to
code or config
userspace
podcontainer
pod containercontainer

28. Flexibility
Performance
Visibility
eBPF in Kernel Space
In the kernel but flexible

Safety, performance, observability,
and programmability

Available by default

29. Real world
Use Cases

30. Networking

31. -Networking
-Security
-Observability
-Service Mesh & Ingress
-based:
Foundation Technology
31
Over 120 USERS.md entries

32. Vision
Inent & identity-based, high performance
container networking platform built using eBPF
The Beginning

33. Network Security 1.0

34. Cluster Mesh
Adding reach to end-to-end network security and observability

35. Load Balancer
Adding reach to end-to-end network security and observability

36. @breakawaybilly
Observability

37. Hubble

38. eBPF networking +
observability
$ hubble observe --namespace demo-app --label "app=frontend" --protocol tcp

Oct 12 06:06:05.907: demo-app/frontend-5f47758b4c-ltk46: 55028 (ID:14943) ->
demo-app/worker-dcdc797d9-n9fj2: 8000 (ID:15426) policy-verdict:L3-L4 ALLOWED (TCP Flags: SYN)
Oct 12 06:06:05.907: demo-app/frontend-5f47758b4c-ltk46: 55028 (ID:14943) ->
demo-app/worker-dcdc797d9-n9fj2: 8000 (ID:15426) policy-verdict:L3-L4 ALLOWED (TCP Flags: SYN)
Oct 12 06:06:05.907: demo-app/frontend-5f47758b4c-ltk46: 55028 (ID:14943) ->
demo-app/worker-dcdc797d9-n9fj2: 8000 (ID:15426) to-endpoint FORWARDED (TCP Flags: SYN)
Oct 12 06:06:06.920: demo-app/frontend-5f47758b4c-ltk46: 55028 (ID:14943) ->
demo-app/worker-dcdc797d9-n9fj2: 8000 (ID:15426) to-endpoint FORWARDED (TCP Flags: SYN)
Oct 12 06:06:08.934: demo-app/frontend-5f47758b4c-ltk46: 55028 (ID:14943) ->
demo-app/worker-dcdc797d9-n9fj2: 8000 (ID:15426) to-endpoint FORWARDED (TCP Flags: SYN)

39. eBPF networking +
observability

40. @breakawaybilly
Security

41. apiVersion: cilium.io/v1alpha1
kind: TracingPolicy
metadata:
name: "etc-files"
spec:
kprobes:
- call: "fd_install"
…
matchArgs:
- index: 1
operator: "Prefix"
values:
- "/etc/"
…

$ kubectl logs ds/tetragon -c export-stdout -f | tetragon
observe
?????? process default/xwing /usr/bin/vi /etc/passwd
?????? open default/xwing /usr/bin/vi /etc/passwd
?????? close default/xwing /usr/bin/vi
?????? open default/xwing /usr/bin/vi /etc/passwd
?????? write default/xwing /usr/bin/vi /etc/passwd 1275 bytes
?????? close default/xwing /usr/bin/vi
?????? exit default/xwing /usr/bin/vi /etc/passwd 0
Security observability

42. Runtime Enforcement

43. Security Observability &
Runtime Enforcement
43

44. @breakawaybilly
Tracing and Profiling

45. @breakawaybilly
Tracing and Profiling

46. @breakawaybilly
eBPF tools provide rich,
context-aware event streams

47. eBPF everything???
Or not?

48. @breakawaybilly
Still not a magic bullet

49. userspace
kernel
network call
read file
run process
podcontainer
pod containercontainer
“Add product X to cart”
open file 345
forward packet to 6.7.8.9
read file 111
create a socket
…
…
query service A
write to database
…

50. ●eBPF doesn’t have business context!
●Requires kernel knowledge
●the data volume can be absurd
eBPF is challenging

51. Buzzing
Community

52. @breakawaybilly
Kernel Community
Happy 10th Birthday ??????

53. @breakawaybilly
eBPF Landscape

54. @breakawaybilly
eBPF
Landscape

55. @breakawaybilly
eBPF Landscape

56. @breakawaybilly
eBPF Acquisitions
•Flowmill ➡ Splunk (November 2020) Security
•Kinvolk ➡ Microsoft (April 2021) Observability
•Pixie ➡ New Relic (July 2021) Observability
•Cmd ➡ Elastic (October 2021) Security
•Optimyze ➡ Elastic (August 2021) Profiling
•Seekret ➡ Datadog (August 2022) Observability
•Pyroscope ➡ Grafana (March 2023)Profiling
•Akita ➡ Postman (July 2023) Observability
•Isovalent ➡ Cisco (April 2024) Networking, Observability, Security
•StackState ➡ SUSE (June 2024) Observability

57. @breakawaybilly
eBPF for Windows

58. @breakawaybilly
eBPF Research Funding

59. Where to go
Next

60. @breakawaybilly
eBPF Wikipedia

61. @breakawaybilly
ebpf.io

62. @breakawaybilly
ebpf.io
en Frrançais
Italiano
Português
Swahili
Español
繁體中文
简体中文

63. @breakawaybilly
ebpf.io
Looking for
help
translating

64. @breakawaybilly
ebpf.io Labs

65. @breakawaybilly
eBPF Summit
September 11th

66. @breakawaybilly
eBPF Newsletter

67. @breakawaybilly
eBPF Documentary - ebpfdocumentary.com

68. @breakawaybilly
The Illustrated Children’s Guide to eBPF

69. @breakawaybilly
State of eBPF

70. A Vision for
eBPF Inside

71. @breakawaybilly
A Vision for eBPF Inside

72. Thank you
ebpf.io
@breakawaybilly
ebpf.io
Download from
isovalent.com