Kafka consumer not able to start reading messages from the last uncommitted offsets

Question

I have a 3-node Kafka cluster setup. I am using Apache spark (consumer) to read messages from Kafka broker. Each topic has 6 partitions. everything was working fine. when logs are coming from Apache Nifi to Kafka queue, spark consumer can read the messages in offsets smoothly, but in case of consumer crash, the spark consumer will not be able to read the remaining messages from Kafka.

I am reading latest offsets from Kafka brokers to my consumer.

I am using Kafka 2.11

Answer 1

Hi@prasant,

I am not sure it will work or not. But you can increase your receive buffer size in the server.properties file.

Hope this works for you!

Get your Kafka certification today to become certified professional

Thank You