I want to write the messages which I am consuming using console consumer to a text file which I can keep for the future reference. Can anyone tell me how to do that?
Use the pipe operator when you are running the console consumer. It will log all the messages which are getting consumed, to a file.
kafka-console-producer.sh --broker-list localhost:9092 --topic Topic < abc.txt
Isn't that backward? The question is about outputting consumer messages to a text file.
That line is a producer loading the events from a file.
You can use a simple > for this:
kafka-console-consumer > file.txt
System.out.println(String.valueOf(output.offset()) + ": " + new String(bytes, "UTF-8"));
outputmessage += new String(bytes, "UTF-8")+"\n";
tail -n0 -F my_file.txt | kafka-console-producer.sh --broker-list localhost:9092 --topic my_topic
echo "message goes here" | kafka-console-consumer.sh --broker-list localhost:9092 --topic my_topic
You can create topic and then cat it to output file:
bin/kafka-topics.sh --create --zookeeper localhost:2181 --replication-factor 3
-partitions 1 --topic topic-name
bin/kafka-console-producer.sh --broker-list localhost:9092 --topic Multibrokerapplication > output.txt
I don't know it will work or ...READ MORE
Yes, it stays the same.
An offset is ...READ MORE
Let me explain to you about Apache ...READ MORE
I am not sure it will work ...READ MORE
I am new in Kafka. Can anyone ...READ MORE
If you already created multiple producers then ...READ MORE
To know leader information you can use ...READ MORE
If we try to use String key ...READ MORE
The reset option only prints the result ...READ MORE
As of kafka 2.3.0 version, there is ...READ MORE
Already have an account? Sign in.