Spark + Hive connectivity

0 votes

Hello Team,
I want to load data from hive into spark data frame. I have specified the hive.metastore.uris
thrift://192.168.1.30:9083
but now during starting spark sheel I am getting below error.

ERROR:
at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1521)
... 91 more
Caused by: org.apache.hadoop.hive.metastore.api.MetaException: Could not connect to meta store using any of the URIs provided. Most recent failure: org.apache.thrift.transport.TTransportException: java.net.ConnectException: Connection refused (Connection refused)
at org.apache.thrift.transport.TSocket.open(TSocket.java:226)
Aug 1 in Apache Spark by Jill
92 views

1 answer to this question.

0 votes

The problem is probably with the command. It would've been better if you had mentioned the command that you are using. Anyways, here's a working code:

Use the following commands to connect to hive.

importorg.apache.spark.sql.hive.HiveContext
val hiveContext = new org.apache.spark.sql.hive.HiveContext(sc)
import hiveContext._


val hive_tables=hiveContext.sql("show tables")
val first = hiveContext.sql("select * from olympics limit 5")
first.show()
answered Aug 1 by Rishni

Related Questions In Apache Spark

+1 vote
2 answers

Hadoop 3 compatibility with older versions of Hive, Pig, Sqoop and Spark

Hadoop 3 is not widely used in ...READ MORE

answered Apr 20, 2018 in Apache Spark by kurt_cobain
• 9,280 points
2,077 views
0 votes
1 answer

Spark to Hive Table creation

There's an easier way to achieve your ...READ MORE

answered Jul 23 in Apache Spark by Dinesh
44 views
0 votes
1 answer
0 votes
1 answer

Spark: Read from Hive, store in HDFS

Below is an example of reading data ...READ MORE

answered Jul 26 in Apache Spark by Lohit
74 views
+1 vote
1 answer
0 votes
1 answer

Hadoop Mapreduce word count Program

Firstly you need to understand the concept ...READ MORE

answered Mar 16, 2018 in Data Analytics by nitinrawat895
• 10,760 points
3,536 views
0 votes
1 answer

hadoop.mapred vs hadoop.mapreduce?

org.apache.hadoop.mapred is the Old API  org.apache.hadoop.mapreduce is the ...READ MORE

answered Mar 16, 2018 in Data Analytics by nitinrawat895
• 10,760 points
436 views
+1 vote
11 answers

hadoop fs -put command?

put syntax: put <localSrc> <dest> copy syntax: copyFr ...READ MORE

answered Dec 7, 2018 in Big Data Hadoop by Aditya
17,991 views
0 votes
1 answer

Spark error: Caused by: java.lang.RuntimeException: The root scratch dir: /tmp/hive on HDFS should be writable.

Give  read-write permissions to  C:\tmp\hive folder Cd to winutils bin folder ...READ MORE

answered Jul 11 in Apache Spark by Rajiv
379 views
0 votes
1 answer

Load .xlsx files to hive tables with spark scala

This should work: def readExcel(file: String): DataFrame = ...READ MORE

answered Jul 22 in Apache Spark by Kishan
313 views