How to convert JSON file to AVRO file and vise versa

+1 vote

Hi All,

I want to convert my json file to avro and vice versa but facing some difficulty. Can anyone please help me debugging this?

val df = spark.read.json(“path to the file”)
df.write.format(“com.databricks.spark.avro”).save(“destination location”)

But I am getting the below error:

org.apache.spark.sql.AnalysisException: Failed to find data source: com.databricks.spark.avro.
Aug 23, 2019 in Apache Spark by Prashant
306 views

1 answer to this question.

+1 vote

Try including the package while starting the spark-shell:

spark-shell --packages com.databricks:spark-avro_2.10:2.0.1

And then import:

import com.databricks.spark.avro._
answered Aug 26, 2019 by Karan

Related Questions In Apache Spark

+1 vote
1 answer
0 votes
1 answer

How to save and retrieve the Spark RDD from HDFS?

You can save the RDD using saveAsObjectFile and saveAsTextFile method. ...READ MORE

answered May 29, 2018 in Apache Spark by Shubham
• 13,370 points
3,122 views
0 votes
1 answer

How to convert rdd object to dataframe in spark

SqlContext has a number of createDataFrame methods ...READ MORE

answered May 30, 2018 in Apache Spark by nitinrawat895
• 10,840 points
1,795 views
+1 vote
1 answer
+1 vote
1 answer

Hadoop Mapreduce word count Program

Firstly you need to understand the concept ...READ MORE

answered Mar 16, 2018 in Data Analytics by nitinrawat895
• 10,840 points
3,859 views
0 votes
1 answer

hadoop.mapred vs hadoop.mapreduce?

org.apache.hadoop.mapred is the Old API  org.apache.hadoop.mapreduce is the ...READ MORE

answered Mar 16, 2018 in Data Analytics by nitinrawat895
• 10,840 points
526 views
+1 vote
11 answers

hadoop fs -put command?

put syntax: put <localSrc> <dest> copy syntax: copyFr ...READ MORE

answered Dec 7, 2018 in Big Data Hadoop by Aditya
20,390 views
0 votes
1 answer

How to create dataframe for the comma delimited file?

 Refer to the below command used: val df ...READ MORE

answered Jul 5, 2019 in Apache Spark by karan
43 views
0 votes
1 answer

How to load data of .csv file in MySQL Database Table?

You can do it using a code ...READ MORE

answered Jul 22, 2019 in Apache Spark by Vishwa
59 views