Re-balancing error while reading messages from Kafka.

0 votes

When I am trying to read messages from a Kafka topic, but the process gets killed with following error:

[2017-08-18 08:11:21,215] ERROR Error processing message, stopping consumer:  (kafka.consumer.ConsoleConsumer$)
kafka.common.ConsumerRebalanceFailedException: topic1 can't rebalance after 4 retries
    at kafka.consumer.ZookeeperConsumerConnector$ZKRebalancerListener.syncedRebalance(ZookeeperConsumerConnector.scala:428)
    at kafka.consumer.ZookeeperConsumerConnector.kafka$consumer$ZookeeperConsumerConnector$$reinitializeConsumer(ZookeeperConsumerConnector.scala:718)
    at kafka.consumer.ZookeeperConsumerConnector$WildcardStreamsHandler.<init>(ZookeeperConsumerConnector.scala:752)
    at kafka.consumer.ZookeeperConsumerConnector.createMessageStreamsByFilter(ZookeeperConsumerConnector.scala:142)
    at kafka.consumer.ConsoleConsumer$.main(ConsoleConsumer.scala:196)
    at kafka.consumer.ConsoleConsumer.main(ConsoleConsumer.scala)
Consumed 0 messages

Can anyone help me with this rebalancing issue?

Jul 9, 2018 in Apache Kafka by coldcode
• 2,050 points
1,232 views

1 answer to this question.

0 votes

rebalance.backoff.ms defines the time for which Kafka will wait before rebalancing & zookeeper.session.timeout.ms parameter specifies the time for which Kafka will wait to connect to zookeeper. So, try adjusting these two parameters.

Hope this helps!

answered Jul 9, 2018 by Shubham
• 13,450 points

Related Questions In Apache Kafka

+1 vote
1 answer

Kafka consumer not able to start reading messages from the last uncommitted offsets

Hi@prasant, I am not sure it will work ...READ MORE

answered Feb 18 in Apache Kafka by MD
• 65,200 points
2,997 views
+1 vote
13 answers

How can I delete all the messages from a Kafka topic?

To delete all the messages from a Kafka topic.  There are many approach ...READ MORE

answered Jun 19 in Apache Kafka by Pawan
• 380 points
119,516 views
0 votes
1 answer

error message while running

Ideally, installing the urllib3 module should solve this error. ...READ MORE

answered Feb 25, 2019 in Apache Kafka by Omkar
• 69,030 points
99 views
0 votes
1 answer

How to check pending messages in KAFKA topic?

kafka-run-class.bat kafka.tools.ConsumerOffsetChecker --broker-info --group test-consumer-group --topic test ...READ MORE

answered May 29 in Apache Kafka by chandra
3,356 views
+1 vote
10 answers

Writing the Kafka consumer output to a file

System.out.println(String.valueOf(output.offset()) + ": " + new String(bytes, ...READ MORE

answered Dec 7, 2018 in Apache Kafka by Harsh
16,626 views
0 votes
1 answer

How can I create a topic in apache kafka?

Hi@akhtar, To create kafka topic you can use ...READ MORE

answered Feb 6 in Apache Kafka by MD
• 65,200 points
135 views
0 votes
2 answers

How can I create multiple consumers in apache kafka?

First of all, Consumer is not thread ...READ MORE

answered Jun 17 in Apache Kafka by Pawan
• 380 points
1,209 views
0 votes
1 answer

Not able to fetch messages from Kafka cluster

You need to add the hostname in ...READ MORE

answered Jul 9, 2018 in Apache Kafka by Shubham
• 13,450 points
1,238 views
0 votes
1 answer

How to reset the offset of messages consumed from Kafka?

The reset option only prints the result ...READ MORE

answered Jul 9, 2018 in Apache Kafka by Shubham
• 13,450 points
8,342 views