Json and snappy compression

0 votes

When trying to write json file using snappy compression the below method is not working.

sqlContext.setConf("spark.sql.json.compression.codec","snappy")
filterStatus.write.json("/user/hduser_212418/heorder_json")

what changes to be done for the above code for it to save in snappy compression format. Only the below one works.

filterStatus.toJSON.rdd.saveAsTextFile("/user/hduser_212418/heorder_json",classOf[org.apache.hadoop.io.compress.SnappyCodec])

Input to the abov is :

val filterStatus = rdFile.filter("order_status like "%Y%"")
filterStatus: org.apache.spark.sql.Dataset[org.apache.spark.sql.Row] = [order_id: int, order_date: bigint ... 2 more fields]
Jan 10 in Big Data Hadoop by digger
• 27,620 points
145 views

1 answer to this question.

Your answer

Your name to display (optional):
Privacy: Your email address will only be used for sending these notifications.
0 votes

Refer the below screenshot for your above requirement

image

image

image

The above approach works

answered Jan 10 by Omkar
• 66,910 points

Related Questions In Big Data Hadoop

0 votes
10 answers

What is the difference between Mongodb and Hadoop?

Apart from the similarity that they are ...READ MORE

answered Dec 6, 2018 in Big Data Hadoop by Deeraj
1,997 views
0 votes
1 answer

How can I download only hdfs and not hadoop?

No, you cannot download HDFS alone because ...READ MORE

answered Mar 15, 2018 in Big Data Hadoop by nitinrawat895
• 9,490 points
57 views
+2 votes
10 answers

Is there any difference between “hdfs dfs” and “hadoop fs” shell commands?

Yes, there's a difference between hadoop fs and ...READ MORE

answered Dec 7, 2018 in Big Data Hadoop by Kunal
8,039 views
0 votes
1 answer

How to install and configure a multi-node Hadoop cluster?

I would recommend you to install Cent ...READ MORE

answered Mar 21, 2018 in Big Data Hadoop by Shubham
• 12,890 points
504 views
0 votes
0 answers
0 votes
1 answer

Hadoop Mapreduce word count Program

Firstly you need to understand the concept ...READ MORE

answered Mar 16, 2018 in Data Analytics by nitinrawat895
• 9,490 points
1,845 views
0 votes
1 answer

hadoop.mapred vs hadoop.mapreduce?

org.apache.hadoop.mapred is the Old API  org.apache.hadoop.mapreduce is the ...READ MORE

answered Mar 16, 2018 in Data Analytics by nitinrawat895
• 9,490 points
160 views
0 votes
10 answers

hadoop fs -put command?

copy command can be used to copy files ...READ MORE

answered Dec 7, 2018 in Big Data Hadoop by Sujay
9,165 views
0 votes
3 answers

Hadoop hdfs: list all files in a directory and its subdirectories

You can do it using queue: private static ...READ MORE

answered Dec 4, 2018 in Big Data Hadoop by Ishwar
1,021 views
0 votes
1 answer

Hadoop: What is the difference between `hadoop dfs` and `hadoop fs`?

You can find the definition here: $HADOOP_HOME/bin/hadoop ... elif [ ...READ MORE

answered Nov 6, 2018 in Big Data Hadoop by Omkar
• 66,910 points
174 views

© 2018 Brain4ce Education Solutions Pvt. Ltd. All rights Reserved.
"PMP®","PMI®", "PMI-ACP®" and "PMBOK®" are registered marks of the Project Management Institute, Inc. MongoDB®, Mongo and the leaf logo are the registered trademarks of MongoDB, Inc.