9419/writing-the-kafka-consumer-output-to-a-file
I want to write the messages which I am consuming using console consumer to a text file which I can keep for the future reference. Can anyone tell me how to do that?
Use the pipe operator when you are running the console consumer. It will log all the messages which are getting consumed, to a file.
kafka-console-producer.sh --broker-list localhost:9092 --topic Topic < abc.txt
Isn't that backward? The question is about outputting consumer messages to a text file.
That line is a producer loading the events from a file.
You can use a simple > for this:
kafka-console-consumer > file.txt
System.out.println(String.valueOf(output.offset()) + ": " + new String(bytes, "UTF-8")); outputmessage += new String(bytes, "UTF-8")+"\n";
writeDataToFile(outputmessage);
tail -n0 -F my_file.txt | kafka-console-producer.sh --broker-list localhost:9092 --topic my_topic
echo "message goes here" | kafka-console-consumer.sh --broker-list localhost:9092 --topic my_topic
You can create topic and then cat it to output file:
bin/kafka-topics.sh --create --zookeeper localhost:2181 --replication-factor 3 -partitions 1 --topic topic-name
bin/kafka-console-producer.sh --broker-list localhost:9092 --topic Multibrokerapplication > output.txt
Yes, it stays the same. An offset is ...READ MORE
Let me explain to you about Apache ...READ MORE
In general, a single producer for all ...READ MORE
Deleting topic isn't always working in 0.8.1.1 Deletion ...READ MORE
Firstly you need to understand the concept ...READ MORE
org.apache.hadoop.mapred is the Old API org.apache.hadoop.mapreduce is the ...READ MORE
put syntax: put <localSrc> <dest> copy syntax: copyFr ...READ MORE
In your case there is no difference ...READ MORE
The reset option only prints the result ...READ MORE
As of kafka 2.3.0 version, there is ...READ MORE
OR
Already have an account? Sign in.