Taking Back Control of Your Telemetry Data with Fluent Bit

It's time you stopped letting your telemetry data pressure your budgets and get in the way of solving issues with agility! No more I say! Take back control of your telemetry data as we guide you through the open source project Fluent Bit. Learn how to manage your telemetry data from source to destination using the pipeline phases covering collection, parsing, aggregation, transformation, and forwarding from any source to any destination. Buckle up for a fun ride as you learn by exploring how telemetry pipelines work, how to set up your first pipeline, and exploring several common use cases that Fluent Bit helps solve. All this backed by a self-paced, hands-on workshop that attendees can pursue at home after this session (https://o11y-workshops.gitlab.io/workshop-fluentbit).

1. Taking Back Control of Your Telemetry Data with Fluent Bit Eric D. Schabell Director Evangelism, Chronosphere @ericschabell{@fosstodon.org} Cloud Native London Meetup - 7 Aug 2024

2. Telemetry in Phases

3. Pillars Phases

4. Pillar problems…

5. Car is on fire…

6. Troubleshooting workflow 1 2 3 telemetry data Root Cause Analysis Know Triage Understand Metrics: What is the symptom ? (MTTD-etection) Distributed Traces: Where is the problem ? (MTTR-emediate) Logs: Why was there a problem? (MTTR-epair)

7. Control with Pipelines

8. Input phase The first phase is INPUT, which is where Fluent Bit uses Input Plugins to gather information from specific sources.

9. Parser phase The second phase is PARSER, which is where unstructured input data is turned into structured data.

10. Filter phase Filtering phase is when we modify, enrich, or delete any of the collected Events.

11. Buffer phase Buffering is where the data is stored, using in-memory or the file system based options.

12. Routing phase This phase is where Fluent Bit uses Tag and Match concepts to determine which output destinations to send data.

13. Output Phase The final phase is where Fluent Bit uses Output Plugins to connect with specific destinations.

14. Let’s look under the hood Pipeline demo

15. Host P ipelines and Prometheus Observability Backend (Prometheus, Jaeger, etc.) , Telemetry pipeline scrape metrics metrics endpoint

16. Host Pipelines and Prometheus Telemetry pipeline scrape metrics metrics endpoint Alert Manager Dashboards WebUI Grafana PromLens Service Discovery Email PagerDuty Slack etc… Querying

17. Let’s look under the hood Pipeline with Prometheus demo

18. Pipelines and OpenTelemetry Host Observability Backend (Prometheus, Jaeger, etc.) , Telemetry pipeline OTel Collector OTLP OTLP

19. Console logging File logging: /tmp/output.json Pipelines and OpenTelemetry Host Telemetry pipeline OTel Collector OTLP OTLP

20. Let’s look under the hood Pipeline with OpenTelemetry demo

21. Hands-on Learning

22. Short link: bit.ly/fluentbit-workshop

23. Short link: https://o11y-workshops.gitlab.io

24. Questions? Eric D. Schabell Director Evangelism, Chronosphere @ericschabell{@fosstodon.org}