9419/writing-the-kafka-consumer-output-to-a-file
I want to write the messages which I am consuming using console consumer to a text file which I can keep for the future reference. Can anyone tell me how to do that?
Use the pipe operator when you are running the console consumer. It will log all the messages which are getting consumed, to a file.
kafka-console-producer.sh --broker-list localhost:9092 --topic Topic < abc.txt
Isn't that backward? The question is about outputting consumer messages to a text file.
That line is a producer loading the events from a file.
You can use a simple > for this:
kafka-console-consumer > file.txt
System.out.println(String.valueOf(output.offset()) + ": " + new String(bytes, "UTF-8")); outputmessage += new String(bytes, "UTF-8")+"\n";
writeDataToFile(outputmessage);
tail -n0 -F my_file.txt | kafka-console-producer.sh --broker-list localhost:9092 --topic my_topic
echo "message goes here" | kafka-console-consumer.sh --broker-list localhost:9092 --topic my_topic
You can create topic and then cat it to output file:
bin/kafka-topics.sh --create --zookeeper localhost:2181 --replication-factor 3 -partitions 1 --topic topic-name
bin/kafka-console-producer.sh --broker-list localhost:9092 --topic Multibrokerapplication > output.txt
Hi@ I don't know it will work or ...READ MORE
Yes, it stays the same. An offset is ...READ MORE
Let me explain to you about Apache ...READ MORE
Hi@prasant, I am not sure it will work ...READ MORE
Hi, I am new in Kafka. Can anyone ...READ MORE
First of all, Consumer is not thread ...READ MORE
Hi@akhtar, To know leader information you can use ...READ MORE
Hi@akhtar, If we try to use String key ...READ MORE
The reset option only prints the result ...READ MORE
To delete all the messages from a Kafka topic. There are many approach ...READ MORE
OR
Already have an account? Sign in.