Apidays Singapore 2024 - OpenTelemetry for API Monitoring by Danielle Kayumbi, Capgemini

OpenTelemetry for API Monitoring
Danielle Kayumbi, Software Engineer - Capgemini

Apidays Singapore 2024: Connecting Customers, Business and Technology (April 17 & 18, 2024)

------

Check out our conferences at https://www.apidays.global/

Do you want to sponsor or talk at one of our conferences?
https://apidays.typeform.com/to/ILJeAaV8

Learn more on APIscene, the global media made by the community for the community:
https://www.apiscene.io

Explore the API ecosystem with the API Landscape:
https://apilandscape.apiscene.io/

1. OpenTelemetry For API Monitoring

2. Danielle KAYUMBI HODIEB Software Engineer danielleKayumbi

3. API Security breaches

4. Impact the business

5. AWS Historic Outage December 7th 2021

6. https://aws.amazon.com/fr/about-aws/global-infrastructure/localzones/locations/

7. . . .

8. Deliver packages

9. Netflix, Disney+ ... Stream movies

10. Play online Video games

11. https://www.supermarktblog.com/2017/11/13/antippen-statt-anstellen-wie-vapiano-und-mcdonalds-den-bestellprozess-im-schnellrestaurant-umkrempeln/ Order in fastfood restaurants

12. Delta, Southwest ... Buy airline tickets

13. Medical exams Were delayed

14. And so where Education exams

15. Deploy in prod New automatic job Scale up

16. Huge volume of requests

17. Huge volume of retry calls

18. Congestion

19. 9 hours to fix

20. How to prevent & discover ?

21. Identify outages Errors Latency Bottlenecks Third parties

22. Alert ! Send Relevant messages

23. Distributed systems

24. Amazon Microservices architecture https://www.divante.com/blog/10-companies-that-implemented-the-microservice-architecture-and-paved-the-way-for-others

25. Netflix More than 700 microservices https://www.divante.com/blog/10-companies-that-implemented-the-microservice-architecture-and-paved-the-way-for-others

26. Observe the system

27. Monitoring the operations from time to time

28. Monitoring is too deterministic

29. Analyse data continuously

30. Understand the relationships between systems

31. Observability The key success

32. Monitoring Observability

33. Traces Metrics Logs Telemetry data

34. Generate, collect, export Telemetry data

35. Help make a system observable

36. Accepted in 2019 Moved to matury level in 2021

37. OpenTracing OpenCensus

38. Fundamentals

39. Distributed tracing

40. https://eng.blackbuck.com/distributed-tracing-at-blackbuck/ Follow request journey across various systems

41. Log

42. Log Message emitted by services or other components

43. [ 2024-04-18T09:05:08.906198+00:00 ] app.INFO : Entity has been successfully persisted Log (Syslog sample)

44. Log Lack contextual information Tracking code execution

45. Log Correlated with trace & spans Included as part of a span

46. Span

47. Span Unit of work or operation . Tracks specific operations that a request makes.

48. { “name” : “get-customer-cart” “context” : { “trace_id” : “7bba9f33312b3dbb8b2c2c62bb7abe2d” , “span_id” : “086e83747d0e381e” }, “parent_id” : “” , “start_time” : “2024-04-18 16:04:01.209458162 +0000 UTC” , “end_time” : “2024-04-18 16:04:01.209458162 +0000 UTC” , “status_code” : “OK” , “attributes” : { “http.method” : “GET” , “user_id” : “123”, “ cart_id”: “456” }} } }

49. Sensitive services (command/query handlers…) Controllers (Http, AMQP actions…) Repositories (database, message broker, http client…)

50. Inject span

51. Annotation

52. Distributed trace

53. Distributed trace Records the paths taken by requests, Through multi-service architecture. Trace = collection of spans

54. https://opentelemetry.io/

55. Cart service Product service API [GET] product by id API [GET] cart by id

56. Get cart operation Get products Retrieve cart db Get product operation Retrieve product db Trace Cache product Context Product service Cart service

57. Context propagation Keep correlation between signals Handle by instrumentation librairies W3C Recommendation for interoperability

58. https://www.datadoghq.com/

59. OpenTelemetry collectors

60. https://docs.lightstep.com/docs/quick-start-collector

61. https://www.elastic.co/blog/best-practices-instrumenting-opentelemetry

62. https://www.datadoghq.com/

63. Service Level Management Four golden signals Monitoring Dashboards

64. Four Golden Signals Latency Traffic Saturation Errors

65. Latency https://cloud.google.com/blog/products/management-tools/the-right-metrics-to-monitor-cloud-data-pipelines

66. Traffic https://cloud.google.com/blog/products/management-tools/the-right-metrics-to-monitor-cloud-data-pipelines

67. Errors https://www.datadoghq.com/

68. Saturation https://cloud.google.com/blog/products/management-tools/the-right-metrics-to-monitor-cloud-data-pipelines

69. https://www.datadoghq.com/

70. Service level

71. Service Level Objective “Less than 1% of users should experience an idle of 5s”

72. Service Level Indicator Defined quantitative measure

73. Availability, latency “How many requests could be handled ?”

74. AWS outage resolution

75. https://www.thousandeyes.com/blog/aws-outage-analysis-dec-7-2021

76. Comprehensive observability

77. Thank you ! danielleKayumbi