How to create a parquet table in hive and store data in it from a hive table

0 votes

This is my hive table :

sqlContext.sql("select * from 20181121_SPARKHIVE_431591").show()
# I am storing data as a parquet file on hdfs
sel = sqlContext.sql("select * from 20181121_SPARKHIVE_431591")
sel.write.parquet("test/parquet_nw",partitionBy = "productID")

#trying to use prev location as parquet table location
sqlContext.sql("CREATE EXTERNAL TABLE 20181118_HIVE_431591 (code string,name string,quantity int, price float) PARTITIONED BY (`productID` int) STORED AS parquet LOCATION "/user/edureka_431591/test/parquet_nw"") 

#querying parquet table
sqlContext.sql("select * from 20181118_HIVE_431591").show() --> this returns nothing

How to store it in parquet?

Jan 28, 2019 in Big Data Hadoop by Param
18,025 views

1 answer to this question.

0 votes

Please use the code attached below for your reference:

To save the parquet file:

sqlContext.sql("SET hive.exec.dynamic.partition.mode= nonstrict")
sqlContext.sql("SET hive.exec.dynamic.partition = true")
sel.write.format("parquet").save("custResult.parquet")

Then you can use the command:

sqlContext.sql("CREATE EXTERNAL TABLE nedw_11 (code string,name string,quantity int, price float) PARTITIONED BY (`productID` int) STORED AS parquet LOCATION "/user/edureka_431591/'custResult.parquet'")

To know more, It's recommended to join Big Data Certification today. 

answered Jan 28, 2019 by Omkar
• 69,210 points

Related Questions In Big Data Hadoop

0 votes
1 answer

How to create a Hive Database and store it in HDFS?

You will have to exclusively mention the ...READ MORE

answered May 20, 2019 in Big Data Hadoop by Jishnu
2,131 views
0 votes
1 answer

How to create smaller table from big table in HIVE?

You could probably best use Hive's built-in sampling ...READ MORE

answered Sep 24, 2018 in Big Data Hadoop by digger
• 26,740 points
1,463 views
0 votes
1 answer

How to create a FileSystem object that can be used for reading from and writing to HDFS?

Read operation on HDFS In order to read ...READ MORE

answered Mar 21, 2018 in Big Data Hadoop by nitinrawat895
• 11,380 points

edited Mar 22, 2018 by nitinrawat895 2,660 views
0 votes
1 answer

What is Modeling data in Hadoop and how to do it?

I suggest spending some time with Apache ...READ MORE

answered Sep 19, 2018 in Big Data Hadoop by Frankie
• 9,830 points
1,555 views
+1 vote
1 answer

Hadoop Mapreduce word count Program

Firstly you need to understand the concept ...READ MORE

answered Mar 16, 2018 in Data Analytics by nitinrawat895
• 11,380 points
10,521 views
0 votes
1 answer

hadoop.mapred vs hadoop.mapreduce?

org.apache.hadoop.mapred is the Old API  org.apache.hadoop.mapreduce is the ...READ MORE

answered Mar 16, 2018 in Data Analytics by nitinrawat895
• 11,380 points
2,165 views
+2 votes
11 answers

hadoop fs -put command?

Hi, You can create one directory in HDFS ...READ MORE

answered Mar 16, 2018 in Big Data Hadoop by nitinrawat895
• 11,380 points
103,814 views
–1 vote
1 answer

Hadoop dfs -ls command?

In your case there is no difference ...READ MORE

answered Mar 16, 2018 in Big Data Hadoop by kurt_cobain
• 9,390 points
4,234 views
0 votes
1 answer

How to create a Hive table from sequence file stored in HDFS?

There are two SerDe for SequenceFile as ...READ MORE

answered Dec 18, 2018 in Big Data Hadoop by Omkar
• 69,210 points
4,511 views
0 votes
1 answer

How to create a managed table in Hive?

You can use this command: create table employee(Name ...READ MORE

answered Dec 14, 2018 in Big Data Hadoop by Omkar
• 69,210 points
5,097 views
webinar REGISTER FOR FREE WEBINAR X
REGISTER NOW
webinar_success Thank you for registering Join Edureka Meetup community for 100+ Free Webinars each month JOIN MEETUP GROUP