I think I can help you with the difference between logstash and Kafka.
Logstash is a tool that can be used to collect, process and forward events and log messages.
Collection is accomplished through a number of input plugins.
Kafka can be used as as an input plugin, where it will read events from a Kafka topic.
Once an input plugin has collected data it can be processed by any number of filters which modify and annotate the event data. Finally events are routed to output plugins which can forward the events to a variety of external programs including Elasticsearch.
Where as Kafka is a messaging software that persists messages, has TTL, and the notion of consumers that pull data out of Kafka. Some of it's usages could be:
- Stream Processing
- Website Activity Tracking
- Metrics Collection and Monitoring
- Log Aggregation
So simply both of them have their own advantages and disadvantages. But then it depends on your requirements solely.
I hope the above information will be helpful to you.