Kafka consumer not able to start reading messages from the last uncommitted offsets

+1 vote
I have a 3-node Kafka cluster setup. I am using Apache spark (consumer) to read messages from Kafka broker. Each topic has 6 partitions. everything was working fine. when logs are coming from Apache Nifi to Kafka queue, spark consumer can read the messages in offsets smoothly, but in case of consumer crash, the spark consumer will not be able to read the remaining messages from Kafka.

I am reading latest offsets from Kafka brokers to my consumer.

I am using Kafka 2.11
Feb 1, 2020 in Apache Kafka by Prashant
 • 130 points • 10,571 views

1 answer to this question.

0 votes

Hi@prasant,

I am not sure it will work or not. But you can increase your receive buffer size in the server.properties file.

Hope this works for you!

Get your Kafka certification today to become certified professional

Thank You

answered Feb 18, 2020 by MD
 • 95,460 points

Related Questions In Apache Kafka

0 votes
1 answer

How to reset the offset of messages consumed from Kafka?

The reset option only prints the result ...READ MORE

answered Jul 10, 2018 in Apache Kafka by Shubham
 • 13,490 points • 14,780 views
0 votes
2 answers

Not able to start Kafka service

If your service, in this case kafka, ...READ MORE

answered May 10, 2019 in Apache Kafka by Novice
 • 20,697 views
0 votes
1 answer

Is there any change in consumer offsets if a new partition(s) is added to a Kafka topic?

Yes, it stays the same. An offset is ...READ MORE

answered Jul 9, 2018 in Apache Kafka by nitinrawat895
 • 11,380 points • 3,100 views
0 votes
1 answer

Not able to download Kafka using curl

The output to download this is in ...READ MORE

answered Feb 21, 2019 in Apache Kafka by Omkar
 • 69,180 points • 3,070 views
0 votes
1 answer

Not able to fetch messages from Kafka cluster

You need to add the hostname in ...READ MORE

answered Jul 10, 2018 in Apache Kafka by Shubham
 • 13,490 points • 3,613 views
0 votes
1 answer

How to deploy kafka cluster on cloud?

You can run kafka in both ways ...READ MORE

answered Jun 14, 2018 in DevOps Tools by DareDev
 • 6,890 points • 1,182 views
0 votes
1 answer

Re-balancing error while reading messages from Kafka.

rebalance.backoff.ms defines the time for which Kafka ...READ MORE

answered Jul 10, 2018 in Apache Kafka by Shubham
 • 13,490 points • 3,467 views
+2 votes
8 answers

Writing the Kafka consumer output to a file

System.out.println(String.valueOf(output.offset()) + ": " + new String(bytes, ...READ MORE

answered Dec 7, 2018 in Apache Kafka by Harsh