Observability For You and Me with OpenTelemetry

Are you interested in dipping your toes in the cloud native observability waters, but as an engineer you are not sure where to get started with tracing problems through your microservices and application landscapes on Kubernetes? Then this is the session for you, where we take you on your first steps in an active open-source project that offers a buffet of languages, challenges, and opportunities for getting started with telemetry data.

The project is called openTelemetry, but before diving into the specifics, we’ll start with de-mystifying key concepts and terms such as observability, telemetry, instrumentation, cardinality, percentile to lay a foundation. After understanding the nuts and bolts of observability and distributed traces, we’ll explore the openTelemetry community; its Special Interest Groups (SIGs), repositories, and how to become not only an end-user, but possibly a contributor.We will wrap up with an overview of the components in this project, such as the Collector, the OpenTelemetry protocol (OTLP), its APIs, and its SDKs.

Attendees will leave with an understanding of key observability concepts, become grounded in distributed tracing terminology, be aware of the components of openTelemetry, and know how to take their first steps to an open-source contribution!

Key Takeaways: Open source, vendor neutral instrumentation is an exciting new reality as the industry standardizes on openTelemetry for observability. OpenTelemetry is on a mission to enable effective observability by making high-quality, portable telemetry ubiquitous. The world of observability and monitoring today has a steep learning curve and in order to achieve ubiquity, the project would benefit from growing our contributor community.

1. Observability For
You and Me with
OpenTelemetry
Eric D. Schabell
Director Evangelism
@ericschabell{@fosstodon.org}
4 Jul 2024
Tech Meetup Glasgow

2. chronosphere.io
Cloud Native O11y (at scale)
Nuts & Bolts of OpenTelemetry
Getting Your Feet Wet

3. chronosphere.io
Cloud Native Observability at Scale

4. chronosphere.io
Cloud Native

5. chronosphere.io
Data volume
Experiment:
-Hello World app on 4 node
Kubernetes cluster with Tracing,
End User Metrics (EUM), Logs,
Metrics (containers / nodes)
-30 days == +450 GB

6. chronosphere.io
Retention
Retention
Retention
Retention
Retention
Retention
Retention

7. chronosphere.io
Cloud Native at Scale

8. chronosphere.io
Simple O11y

9. chronosphere.io
O11y at Scale (over time…)

10. chronosphere.io
“It’s remarkable how common this
situation is, where an organization
is paying more for their
observability data, than they do
for their production
infrastructure.”
?

11. chronosphere.io
O11y at Scale (need)

12. chronosphere.io
Telemetry is automatically collecting, transmitting
and measuring data from remote sources. It
transmits data back to a central location and
analyzes to monitor and control remote systems.

13. chronosphere.io
Nuts & Bolts of
OpenTelemetry

14. chronosphere.io
OpenTelemetry is not an
observability back-end.

15. chronosphere.io
https://opentelemetry.io

16. chronosphere.io
Traces give us the big picture and understanding
the full path a request takes in your application.
Spans represents a unit of work or operation and
are building blocks of traces.
Metrics are measurements of a service
captured at runtime.
Logs are a timestamped text record,
either structured (recommended) or
unstructured, with metadata.

17. chronosphere.io
●API - generating & correlating tracing data
●SDK - language specific impl
●Data - OpenTelemetry Protocol (OTLP)
Specification

18. chronosphere.io
Applications (Java)
OTel Auto Instrumentation (libraries)
OTel API
OTel SDK
OTel Collector
OTLP OTLP OTLP
(Auto) instrumentation

19. chronosphere.io
Host Observability Backend
(Prometheus, Jaeger, Fluent Bit, etc.),
Applications
OTel Auto Instrumentation
OTel API
OTel SDK
OTel Collector Agent
OTLP
OTLP
OTLP OTLP OTLP
Collector (agent)

20. chronosphere.io
Host
Host
Host
Observability Backend
(Prometheus, Jaeger, Fluent Bit, etc.),
Applications
OTel Auto Instrumentation
OTel API
OTel SDK
OTel Collector Agent
OTLP
OTLP OTLP OTLP
Collector (gateway)
OTel Collector Gateway

21. chronosphere.io
Observability Backend
(Prometheus, Jaeger, Fluent Bit, etc.),
OTel Collector
OTLP
Telemetry
(Spans)
Collector (internals)
ReceiverProcessorExporter

22. chronosphere.io
Spans and Traces

23. chronosphere.io
Metrics in OTel
●Counter - value accumulates over time
●Asynchronous Counter - Same as the Counter, but is
collected once for each export (aggregated value)
●UpDownCounter: example could be a queue length
●Asynchronous UpDownCounter - Same as the
UpDownCounter, but is collected once for each export
●Gauge: Measures a current value at the time it is read
●Histogram: A client-side aggregation of values, such as request
latencies (buckets)

24. chronosphere.io
OpenTelemetry’s approach with logs is different. Because existing logging
solutions are widespread in language and operational ecosystems,
OpenTelemetry acts as a “bridge” between those logs, the tracing and
metrics signals, and other OpenTelemetry components.

25. chronosphere.io
Kubernetes Operator
Collector
Auto-instrumentation
Collector
Auto-instrumentation
Collector
Auto-instrumentation
Collector
Auto-instrumentation

26. chronosphere.io
Getting Your Feet Wet

27. chronosphere.io
CNCF - cloud-native.slack.com

28. chronosphere.io
CNCF - End User Community

29. chronosphere.io
Workshop (aka Demo)
Demo application is a Flask web application
written in Python:
●http://localhost:8001/ - page load count
●http://localhost:8001/doggo - photo from the Dog API
●http://localhost:8001/rolldice - simulated dice roll
●http://localhost:16686 - Jaeger UI

30. Workshop?
Short link: bit.ly/opentelemetry-workshop

31. Questions?
Eric D. Schabell
Director Evangelism
@ericschabell{@fosstodon.org}