Re-balancing error while reading messages from Kafka.

0 votes

When I am trying to read messages from a Kafka topic, but the process gets killed with following error:

[2017-08-18 08:11:21,215] ERROR Error processing message, stopping consumer:  (kafka.consumer.ConsoleConsumer$)
kafka.common.ConsumerRebalanceFailedException: topic1 can't rebalance after 4 retries
    at kafka.consumer.ZookeeperConsumerConnector$ZKRebalancerListener.syncedRebalance(ZookeeperConsumerConnector.scala:428)
    at kafka.consumer.ZookeeperConsumerConnector.kafka$consumer$ZookeeperConsumerConnector$$reinitializeConsumer(ZookeeperConsumerConnector.scala:718)
    at kafka.consumer.ZookeeperConsumerConnector$WildcardStreamsHandler.<init>(ZookeeperConsumerConnector.scala:752)
    at kafka.consumer.ZookeeperConsumerConnector.createMessageStreamsByFilter(ZookeeperConsumerConnector.scala:142)
    at kafka.consumer.ConsoleConsumer$.main(ConsoleConsumer.scala:196)
    at kafka.consumer.ConsoleConsumer.main(ConsoleConsumer.scala)
Consumed 0 messages

Can anyone help me with this rebalancing issue?

Jul 9, 2018 in Apache Kafka by coldcode
• 2,050 points
766 views

1 answer to this question.

0 votes

rebalance.backoff.ms defines the time for which Kafka will wait before rebalancing & zookeeper.session.timeout.ms parameter specifies the time for which Kafka will wait to connect to zookeeper. So, try adjusting these two parameters.

Hope this helps!

answered Jul 9, 2018 by Shubham
• 13,380 points

Related Questions In Apache Kafka

+1 vote
1 answer

Kafka consumer not able to start reading messages from the last uncommitted offsets

Hi@prasant, I am not sure it will work ...READ MORE

answered Feb 18 in Apache Kafka by MD
• 7,790 points
273 views
+1 vote
12 answers

How can I delete all the messages from a Kafka topic?

As of kafka 2.3.0 version, there is ...READ MORE

answered Sep 8, 2019 in Apache Kafka by Brajkishore
• 240 points
72,316 views
0 votes
1 answer

error message while running

Ideally, installing the urllib3 module should solve this error. ...READ MORE

answered Feb 25, 2019 in Apache Kafka by Omkar
• 69,000 points
62 views
0 votes
0 answers

How to check pending messages in KAFKA topic?

Let say we have first_topic which has kafka_server_brokertopicmetrics_messagesin_total{instance="localhost:1120",job="kafka",topic="first_topic"}  ...READ MORE

May 2, 2019 in Apache Kafka by anonymous
1,517 views
+1 vote
10 answers

Writing the Kafka consumer output to a file

System.out.println(String.valueOf(output.offset()) + ": " + new String(bytes, ...READ MORE

answered Dec 7, 2018 in Apache Kafka by Harsh
11,182 views
0 votes
1 answer

How can I create a topic in apache kafka?

Hi@akhtar, To create kafka topic you can use ...READ MORE

answered Feb 6 in Apache Kafka by MD
• 7,790 points
49 views
0 votes
1 answer

How can I create multiple consumers in apache kafka?

Hi@akhtar, If you already created multiple producers then ...READ MORE

answered Feb 6 in Apache Kafka by MD
• 7,790 points
83 views
0 votes
1 answer

Not able to fetch messages from Kafka cluster

You need to add the hostname in ...READ MORE

answered Jul 9, 2018 in Apache Kafka by Shubham
• 13,380 points
843 views
0 votes
1 answer

How to reset the offset of messages consumed from Kafka?

The reset option only prints the result ...READ MORE

answered Jul 9, 2018 in Apache Kafka by Shubham
• 13,380 points
4,576 views