Re-balancing error while reading messages from Kafka

0 votes

When I am trying to read messages from a Kafka topic, but the process gets killed with following error:

[2017-08-18 08:11:21,215] ERROR Error processing message, stopping consumer:  (kafka.consumer.ConsoleConsumer$)
kafka.common.ConsumerRebalanceFailedException: topic1 can't rebalance after 4 retries
    at kafka.consumer.ZookeeperConsumerConnector$ZKRebalancerListener.syncedRebalance(ZookeeperConsumerConnector.scala:428)
    at kafka.consumer.ZookeeperConsumerConnector.kafka$consumer$ZookeeperConsumerConnector$$reinitializeConsumer(ZookeeperConsumerConnector.scala:718)
    at kafka.consumer.ZookeeperConsumerConnector$WildcardStreamsHandler.<init>(ZookeeperConsumerConnector.scala:752)
    at kafka.consumer.ZookeeperConsumerConnector.createMessageStreamsByFilter(ZookeeperConsumerConnector.scala:142)
    at kafka.consumer.ConsoleConsumer$.main(ConsoleConsumer.scala:196)
    at kafka.consumer.ConsoleConsumer.main(ConsoleConsumer.scala)
Consumed 0 messages

Can anyone help me with this rebalancing issue?

Jul 10, 2018 in Apache Kafka by coldcode
• 2,080 points
2,156 views

1 answer to this question.

0 votes

rebalance.backoff.ms defines the time for which Kafka will wait before rebalancing & zookeeper.session.timeout.ms parameter specifies the time for which Kafka will wait to connect to zookeeper. So, try adjusting these two parameters.

Hope this will work

Join our Kafka course online today to become certified expert.

Thank You

answered Jul 10, 2018 by Shubham
• 13,490 points

Related Questions In Apache Kafka

+1 vote
1 answer

Kafka consumer not able to start reading messages from the last uncommitted offsets

Hi@prasant, I am not sure it will work ...READ MORE

answered Feb 18, 2020 in Apache Kafka by MD
• 95,380 points
7,956 views
0 votes
14 answers

How can I delete all the messages from a Kafka topic?

To delete all the messages from a Kafka topic.  There are many approach ...READ MORE

answered Jun 19, 2020 in Apache Kafka by Pawan
• 380 points
216,700 views
0 votes
1 answer

error message while running

Ideally, installing the urllib3 module should solve this error. ...READ MORE

answered Feb 25, 2019 in Apache Kafka by Omkar
• 69,210 points
220 views
0 votes
1 answer

How to check pending messages in KAFKA topic?

kafka-run-class.bat kafka.tools.ConsumerOffsetChecker --broker-info --group test-consumer-group --topic test ...READ MORE

answered May 29, 2020 in Apache Kafka by chandra
7,857 views
+2 votes
10 answers

Writing the Kafka consumer output to a file

System.out.println(String.valueOf(output.offset()) + ": " + new String(bytes, ...READ MORE

answered Dec 7, 2018 in Apache Kafka by Harsh
30,230 views
0 votes
1 answer

How can I create a topic in apache kafka?

Hi@akhtar, To create kafka topic you can use ...READ MORE

answered Feb 6, 2020 in Apache Kafka by MD
• 95,380 points
396 views
0 votes
2 answers

How can I create multiple consumers in apache kafka?

First of all, Consumer is not thread ...READ MORE

answered Jun 18, 2020 in Apache Kafka by Pawan
• 380 points
2,662 views
0 votes
1 answer

Not able to fetch messages from Kafka cluster

You need to add the hostname in ...READ MORE

answered Jul 10, 2018 in Apache Kafka by Shubham
• 13,490 points
2,219 views
0 votes
1 answer

How to reset the offset of messages consumed from Kafka?

The reset option only prints the result ...READ MORE

answered Jul 10, 2018 in Apache Kafka by Shubham
• 13,490 points
12,772 views
webinar REGISTER FOR FREE WEBINAR X
REGISTER NOW
webinar_success Thank you for registering Join Edureka Meetup community for 100+ Free Webinars each month JOIN MEETUP GROUP