Spark streaming with Kafka dependency error

Question

I am writing a program where I am trying to stream data using Spark streaming from Kafka. But when I am executing the program I am getting the following error:

An exception or error caused a run to abort: kafka.server.KafkaServer$.$lessinit$greater$default$2()Lorg/apache/kafka/common/utils/Time;
java.lang.NoSuchMethodError: kafka.server.KafkaServer$.$lessinit$greater$default$2()Lorg/apache/kafka/common/utils/Time;
    at net.manub.embeddedkafka.EmbeddedKafkaSupport$class.startKafka(EmbeddedKafka.scala:467)
    at net.manub.embeddedkafka.EmbeddedKafka$.startKafka(EmbeddedKafka.scala:38)
    at net.manub.embeddedkafka.EmbeddedKafka$.start(EmbeddedKafka.scala:55)
    at iris.orange.ScalaTest$$anonfun$1.apply$mcV$sp(ScalaTest.scala:10)

I am not sure but I think this error is regarding dependency. Can anyone help me in understanding the error? Thanks in advance.

Answer 1

Your error is with the version of spark-kafka-streaming. Trying using this Spark-streaming-Kafka jar. If It does not work then try changing the version of the jar accordingly.

<dependency>
    <groupId>org.apache.spark</groupId>
    <artifactId>spark-streaming-kafka-0-10_2.10</artifactId>
    <version>2.0.0</version>
</dependency>