In spark streaming with kafka, i am getting error “Exception in thread ”main“ java.lang.NoSuchMethodError: org.apache.spark.internal.Logging.$init$”

+1 vote

I am getting error "Exception in thread "main" java.lang.NoSuchMethodError: org.apache.spark.internal.Logging.$init$" while running the spark streaming integration with kafka program in intellij in scala. the code is fine. I have given the versions also right. Can you please anyone tell me the problem.

I have already tried with different versions and the code is not working.

I am getting error at this place. ssc(streamingcontext):

val messages = KafkaUtils.createDirectStream[String, String](
  ssc,
  LocationStrategies.PreferConsistent,
  ConsumerStrategies.Subscribe[String, String](topicsSet, kafkaParams))
Nov 7, 2019 in Apache Spark by c.kothamasu
• 160 points
568 views

What version of Scala are you using? I faced a similar problem. I was using scala 2.12 and the error was solved when I downgraded to Scale 2.12. Also check if the version is correctly mentioned in the build.sbt file.

1 answer to this question.

+1 vote
Try to put the kafka client for your version.

Ex:

<dependency>
    <groupId>org.apache.kafka</groupId>
    <artifactId>kafka-clients</artifactId>
    <version>0.11.0.3</version>
</dependency>
answered Dec 13, 2019 by Alexandru
• 510 points

Related Questions In Apache Spark

0 votes
1 answer
0 votes
1 answer

Getting error while connecting zookeeper in Kafka - Spark Streaming integration

I guess you need provide this kafka.bootstrap.servers ...READ MORE

answered May 24, 2018 in Apache Spark by Shubham
• 13,380 points
1,051 views
0 votes
1 answer

Spark streaming with Kafka dependency error

Your error is with the version of ...READ MORE

answered Jul 5, 2018 in Apache Spark by Shubham
• 13,380 points
337 views
0 votes
1 answer

"main" java.lang.NoClassDefFoundError: org/apache/hadoop/fs/FSDataInputStream

1. We will check whether master and ...READ MORE

answered Jul 29, 2019 in Apache Spark by Yogi
1,292 views
+1 vote
2 answers
0 votes
1 answer

What are the differences between Apache Kafka and RabbitMQ?

Hi@akhtar, Apache kafka and RabitMQ have their own ...READ MORE

answered Feb 6 in Apache Kafka by MD
• 22,020 points
41 views
+1 vote
1 answer

Cannot resolve Error In Spark when filter records with two where condition

Try df.where($"cola".isNotNull && $"cola" =!= "" && !$"colb".isin(2,3)) your ...READ MORE

answered Dec 13, 2019 in Apache Spark by Alexandru
• 510 points

edited Dec 13, 2019 by Alexandru 409 views
+1 vote
1 answer

Spark: java.io.FileNotFoundException

Hello, From the error I get that the ...READ MORE

answered Dec 13, 2019 in Apache Spark by Alexandru
• 510 points
822 views