Spark: java.sql.SQLException: No suitable driver

0 votes

I am facing below error while loading data from sql to spark for processing.
Could you please help me with the error.

val df = spark.read.format("jdbc").option("url", "jdbc:mysql://sqoopdb.edu.cloudlab.com/z_dinesh").option("dbtable", "t").option("user", "labuser").option("
password", "edureka").load()
java.sql.SQLException: No suitable driver
at java.sql.DriverManager.getDriver(DriverManager.java:315)
at org.apache.spark.sql.execution.datasources.jdbc.JDBCOptions$$anonfun$7.apply(JDBCOptions.scala:84)
at org.apache.spark.sql.execution.datasources.jdbc.JDBCOptions$$anonfun$7.apply(JDBCOptions.scala:84)
at scala.Option.getOrElse(Option.scala:121)
at org.apache.spark.sql.execution.datasources.jdbc.JDBCOptions.(JDBCOptions.scala:83)
at org.apache.spark.sql.execution.datasources.jdbc.JDBCOptions.(JDBCOptions.scala:34)
at org.apache.spark.sql.execution.datasources.jdbc.JdbcRelationProvider.createRelation(JdbcRelationProvider.scala:32)
at org.apache.spark.sql.execution.datasources.DataSource.resolveRelation(DataSource.scala:330)
at org.apache.spark.sql.DataFrameReader.load(DataFrameReader.scala:152)
at org.apache.spark.sql.DataFrameReader.load(DataFrameReader.scala:125)
... 48 elided
Jul 24 in Apache Spark by Tina
270 views

1 answer to this question.

0 votes

The missing driver is the JDBC one and you have to add it to the SparkSQL configuration. You can do it through your Properties object with this line:

prop.setProperty("driver", "com.mysql.jdbc.Driver") 

Or, in your case, the complete statement should be as below,

val df = spark.read.format("jdbc").option("url", "jdbc:mysql://sqoopdb.edu.cloudlab.com/z_dinesh").option("driver", "com.mysql.jdbc.Driver").option("dbtable", "t").option("user", "labuser").option("password", "edureka").load() 

Also, make sure that the MySQL JDBC connection jar is added to the program.

answered Jul 24 by John

Related Questions In Apache Spark

0 votes
1 answer

Get Spark SQL configuration in Java

You will need to use Spark session ...READ MORE

answered Mar 18 in Apache Spark by John
33 views
0 votes
1 answer

cache tables in apache spark sql

Caching the tables puts the whole table ...READ MORE

answered May 4, 2018 in Apache Spark by Data_Nerd
• 2,360 points
793 views
0 votes
1 answer

What's the difference between 'filter' and 'where' in Spark SQL?

Both 'filter' and 'where' in Spark SQL ...READ MORE

answered May 23, 2018 in Apache Spark by nitinrawat895
• 10,690 points
6,864 views
0 votes
1 answer

Spark Driver roles

A Spark driver (aka an application’s driver ...READ MORE

answered Jun 21, 2018 in Apache Spark by Ashish
• 2,630 points
45 views
0 votes
1 answer
0 votes
1 answer

Hadoop Mapreduce word count Program

Firstly you need to understand the concept ...READ MORE

answered Mar 16, 2018 in Data Analytics by nitinrawat895
• 10,690 points
3,097 views
0 votes
1 answer

hadoop.mapred vs hadoop.mapreduce?

org.apache.hadoop.mapred is the Old API  org.apache.hadoop.mapreduce is the ...READ MORE

answered Mar 16, 2018 in Data Analytics by nitinrawat895
• 10,690 points
349 views
0 votes
10 answers

hadoop fs -put command?

put syntax: put <localSrc> <dest> copy syntax: copyFr ...READ MORE

answered Dec 7, 2018 in Big Data Hadoop by Aditya
15,203 views
0 votes
1 answer

Spark error: Caused by: java.lang.RuntimeException: The root scratch dir: /tmp/hive on HDFS should be writable.

Give  read-write permissions to  C:\tmp\hive folder Cd to winutils bin folder ...READ MORE

answered Jul 11 in Apache Spark by Rajiv
187 views
0 votes
1 answer

Unable to run the java- spark standalone program

Though there is nothing wrong with the ...READ MORE

answered Jul 30 in Apache Spark by Lohit
41 views