ZK Quorum size confluent kafka detailed explanation has been given beginners can use this it is very helpful

Quoram set up

1. Zookeeper Quorum setup

2. Role of a Zookeeper
❖Zookeeper provide multiple features for distributed platforms
•Distributed configuration management
•Self election /consensus building
•Coordination
•Key value store
Zookeeper Quorum setup
❖Zookeeper is used in many distributed platforms like Hadoop, kafka etc….
❖It’s Apache project that’s proven to be very stable and hasn’t had major release
for several years

3. Zookeeper Quorum setup
Structure of a Zookeeper
Zookeeper internal data structure is like tree
Each node is called as zNode
Each zNodehas a path
zNodecan be persistent or ephemeral
Each zNodecan store data
zNodecan’t be renamed
Each zNodecan be watched for changes

4. Zookeeper Quorum setup
Role of a Zookeeper in kafka cluster
Broker registration with heartbeat mechanism to keep the list current
Maintains list of topics along side
•Their configurations (Partitions, replication factor, additional configuration…..etc)
•The list of ISRs for partitions
Performs leader election in case of broker down
Stores kafka cluster id (randomly generated at 1
st
startup of the cluster)
Storing ACLs (Access Control List), if security enabled
•Topics
•Consumer groups
•Users

5. Zookeeper Quorum setup
Zookeeper Quorum Architecture size
Zookeeper quorum needs to have strict majority of servers up and running
There fore Zookeeper quorum have 1,3,5,7,9, (2N+1) server
This allows for 0,1,2,3,4,N servers can go down
Ex: If we have 2 zookeeper servers, if 1 zkserver is down and 1 zkserver is up then there is no strict majority
If we have 4 zookeeper servers, if 2 zkservers are down and 2 zkservers are up then also there is not strict majority
And hence even number of zookeeper quorum size is not recommended

6. Zookeeper Quorum setup

7. Zookeeper Quorum setup
Zookeeper configuration
Edit zookeeper.propertieswith much better values like below
# to store zookeeper data
dataDir=/data/zk
#clients can connect through 2181
clientPort=2181
# maximum number of clients is zero (it means allowing infinity connections)
maxClientCnxns=0
#To define heart beating stuff (2000ms equals to 2 seconds)
tickTime=2000
# How many ticks to take for the initial sync (10*2000(ticktime) ms==> 20,000 ms==> 20 Secs)
initLimit=10
# how many syncs(ticks) can be passed b/w request and acknowledgement(5*2000(ticktime) ==> 10000 ms==> 10 secs)
syncLimit=5
#zoo servers
server.1=kafka1:2888:3888
server.2=kafka2:2888:3888
server.3=kafka3:2888:3888
Note: Make sure the ports 2181, 2888 and 3888 are not blocked in firewall