How to purge Kafka Topic?

0 votes

I want to push a message into a kafka topic on my local machine.

But every time I'm trying to do it, I'm getting an error:

kafka.common.InvalidMessageSizeException: invalid message size
Increasing the fetch.size is not ideal here, because I don't actually want to accept messages that big. 

Is there a way to purge the topic in kafka?

Thanks in advance!

Sep 4, 2018 in Apache Kafka by Shubham
• 13,380 points
806 views

1 answer to this question.

0 votes

For your case what you can do is, temporarily update the retention time on the topic to one second:

kafka-topics.sh --zookeeper localhost:2081 --alter --topic DemoTopic --config retention.ms=1000
then wait for the purge to take effect (about one minute). Once purged, restore the previous retention.ms value.

Hope this helps!

answered Sep 4, 2018 by nitinrawat895
• 10,870 points

Related Questions In Apache Kafka

0 votes
0 answers

How to check pending messages in KAFKA topic?

Let say we have first_topic which has kafka_server_brokertopicmetrics_messagesin_total{instance="localhost:1120",job="kafka",topic="first_topic"}  ...READ MORE

May 2, 2019 in Apache Kafka by anonymous
1,550 views
0 votes
1 answer

How to reset the offset of messages consumed from Kafka?

The reset option only prints the result ...READ MORE

answered Jul 9, 2018 in Apache Kafka by Shubham
• 13,380 points
4,637 views
0 votes
1 answer

How to commit message offsets in Kafka for reliable data pipeline?

You can use auto.commit.enable to allow Kafka ...READ MORE

answered Jul 9, 2018 in Apache Kafka by Shubham
• 13,380 points
604 views
+3 votes
1 answer

How to install Kafka on Windows System?

These are the steps to install Kafka ...READ MORE

answered Feb 25, 2019 in Apache Kafka by Ramya
5,916 views
+1 vote
12 answers

How can I delete all the messages from a Kafka topic?

As of kafka 2.3.0 version, there is ...READ MORE

answered Sep 8, 2019 in Apache Kafka by Brajkishore
• 240 points
73,234 views
0 votes
1 answer

Kafka Elements

The most important elements of Kafka: Topic – ...READ MORE

answered Sep 18, 2018 in Apache Kafka by kurt_cobain
• 9,310 points
241 views
0 votes
1 answer

Kafka vs Flume

Even though both are used for real-time ...READ MORE

answered Sep 24, 2018 in Big Data Hadoop by Data_Nerd
• 2,370 points
154 views
0 votes
1 answer

Integrating Arduino to Apache Kafka

If you are using Arduino Yun to build your IoT application(s), ...READ MORE

answered Sep 27, 2018 in IoT (Internet of Things) by anonymous2
• 4,280 points
1,164 views
0 votes
1 answer

How to delete a topic in Kafka 0.8.1.1?

Deleting topic isn't always working in 0.8.1.1 Deletion ...READ MORE

answered Sep 4, 2018 in Apache Kafka by nitinrawat895
• 10,870 points
383 views
0 votes
1 answer

Is there any change in consumer offsets if a new partition(s) is added to a Kafka topic?

Yes, it stays the same. An offset is ...READ MORE

answered Jul 9, 2018 in Apache Kafka by nitinrawat895
• 10,870 points
525 views