Re-balancing error while reading messages from Kafka

0 votes

When I am trying to read messages from a Kafka topic, but the process gets killed with following error:

[2017-08-18 08:11:21,215] ERROR Error processing message, stopping consumer:  (kafka.consumer.ConsoleConsumer$)
kafka.common.ConsumerRebalanceFailedException: topic1 can't rebalance after 4 retries
    at kafka.consumer.ZookeeperConsumerConnector$ZKRebalancerListener.syncedRebalance(ZookeeperConsumerConnector.scala:428)
    at kafka.consumer.ZookeeperConsumerConnector.kafka$consumer$ZookeeperConsumerConnector$$reinitializeConsumer(ZookeeperConsumerConnector.scala:718)
    at kafka.consumer.ZookeeperConsumerConnector$WildcardStreamsHandler.<init>(ZookeeperConsumerConnector.scala:752)
    at kafka.consumer.ZookeeperConsumerConnector.createMessageStreamsByFilter(ZookeeperConsumerConnector.scala:142)
    at kafka.consumer.ConsoleConsumer$.main(ConsoleConsumer.scala:196)
    at kafka.consumer.ConsoleConsumer.main(ConsoleConsumer.scala)
Consumed 0 messages

Can anyone help me with this rebalancing issue?

Jul 10, 2018 in Apache Kafka by coldcode
• 2,080 points
1,930 views

1 answer to this question.

0 votes

rebalance.backoff.ms defines the time for which Kafka will wait before rebalancing & zookeeper.session.timeout.ms parameter specifies the time for which Kafka will wait to connect to zookeeper. So, try adjusting these two parameters.

Hope this will work

Join our Kafka course online today to become certified expert.

Thank You

answered Jul 10, 2018 by Shubham
• 13,490 points

Related Questions In Apache Kafka

+1 vote
1 answer

Kafka consumer not able to start reading messages from the last uncommitted offsets

Hi@prasant, I am not sure it will work ...READ MORE

answered Feb 18, 2020 in Apache Kafka by MD
• 95,340 points
7,119 views
0 votes
14 answers

How can I delete all the messages from a Kafka topic?

To delete all the messages from a Kafka topic.  There are many approach ...READ MORE

answered Jun 19, 2020 in Apache Kafka by Pawan
• 380 points
206,779 views
0 votes
1 answer

error message while running

Ideally, installing the urllib3 module should solve this error. ...READ MORE

answered Feb 25, 2019 in Apache Kafka by Omkar
• 69,190 points
173 views
0 votes
1 answer

How to check pending messages in KAFKA topic?

kafka-run-class.bat kafka.tools.ConsumerOffsetChecker --broker-info --group test-consumer-group --topic test ...READ MORE

answered May 29, 2020 in Apache Kafka by chandra
6,859 views
+2 votes
10 answers

Writing the Kafka consumer output to a file

System.out.println(String.valueOf(output.offset()) + ": " + new String(bytes, ...READ MORE

answered Dec 7, 2018 in Apache Kafka by Harsh
27,174 views
0 votes
1 answer

How can I create a topic in apache kafka?

Hi@akhtar, To create kafka topic you can use ...READ MORE

answered Feb 6, 2020 in Apache Kafka by MD
• 95,340 points
322 views
0 votes
2 answers

How can I create multiple consumers in apache kafka?

First of all, Consumer is not thread ...READ MORE

answered Jun 18, 2020 in Apache Kafka by Pawan
• 380 points
2,422 views
0 votes
1 answer

Not able to fetch messages from Kafka cluster

You need to add the hostname in ...READ MORE

answered Jul 10, 2018 in Apache Kafka by Shubham
• 13,490 points
1,982 views
0 votes
1 answer

How to reset the offset of messages consumed from Kafka?

The reset option only prints the result ...READ MORE

answered Jul 10, 2018 in Apache Kafka by Shubham
• 13,490 points
12,418 views
webinar REGISTER FOR FREE WEBINAR X
Send OTP
REGISTER NOW
webinar_success Thank you for registering Join Edureka Meetup community for 100+ Free Webinars each month JOIN MEETUP GROUP