What is the use of Apache Kafka in a Big Data Cluster

Question

I am a new learner of Big Data, I have set up a Hadoop Cluster and using Apache Kafka and ZooKeeper. Everything is fine, but, I don't know the functionality of the ZooKeeper and Kafka. Can anyone explain in a better and understandable way?

Answer 1

Kafka is a Distributed Messaging System which comprises the following components:

• Producer
• Broker
• Consumer

A Producer is the one who generates the message which is called as a topic.

The produced messages are stored in Kafka Clusters called as Brokers

The Consumer is the one who gets subscribed to the producer and consumes the message or the topic