Introduction to Apache Kafka

Apache Kafka is a distributed streaming Open source platform developed by Apache Software Foundation, written in Scala and Java.

1. Introduction to Apache Kafka By: Dhaval Thumar WWW.AIMDEK.COM

2. Agenda What is Apache Kafka? How Kafka works? History Kafka Products Kafka Core Components

3. What is Apache Kafka? Apache Kafka is a distributed streaming Open source platform developed by Apache Software Foundation, written in Scala and Java A distributed system is a system with multiple components located on different machines that communicate and coordinate actions via messages in order to appear as a single system to the end-user. Create real time data streams Processing real time data stream

4. What is Apache Kafka?

5. How Kafka Works? Kafka adopted Pub-Sub messaging architecture. Brocker Publisher Subscriber Message Broker Message Producer Message Consumer

6. History Developed by Linkedin in 2011 to solve data integration problem

7. History

8. Kafka Products Kafka Broker - Central server system Kafka Client - Producer & Consumer Java & Scala Library Kafka Connect - Data Integration tool Kafka Stream - Java & Scala library for Real time stream processing application KSQL

9. Kafka Core Concept Producer Consumer Broker/Server Cluster Topic Partition Offset Consumer Groups

10. Kafka Producer, Consumer, Broker

11. Kafka Topic Guys, I am collecting data from multiple producers. Which one do you want?

12. Kafka Topic Ok, give me the data sent by the smart-meter aproducer

13. Kafka Topic Well, the smart meter producer is sending three different types of data. Current-load-sent every minute Consumed-units-sent every hour Input-current-fluctuations-sent as and when it happens Which one do you want?

14. Kafka Topic Well, give me the current-load Message

15. Kafka Topic Ok, so you are looking for a current-load message. That’s great! But we have a bunch of smart meter producers, and all of them are sending the current-load message. so which one of those are you interested in?

16. Kafka Topic

17. Kafka Topic Give me all the data that you received in the current load

18. Live Demo

19. Questions and Answers

20. www.aimdek.com AIMDek Technologies Pvt. Ltd. [phone]   |  [phone] Sales:   [email protected] General:   [email protected] AIMDek Technologies Inc. [phone] Sales:   [email protected]   General:   [email protected] AIMDek Technologies Inc. [phone] Sales:   [email protected] General:   [email protected] www.aimdek.com www.aimdek.com Canada India USA