How to read json file as dictionary in scala?

0 votes
Hi,

How to read json file in Scala and use its content?
e. g. jason file having name: ABC, age: 25, Location: XYZ

read these details and print in Scala.
Jul 24 by Tina
504 views

1 answer to this question.

0 votes
val df1= sqlContext.read.json("file:///home/edureka/Desktop/datsets/world_bank.json")  // loads file, give appropriate path

This sql context is used in  version below  2.0 of spark , but this also will work.

  Using spark session fro 2.0 and above use below line

val df1 = spark.read.json("file:///home/edureka/Desktop/datsets/world_bank.json"

df1.printSchema() // provides the schema details

image

df1select("id","countrycode").show

output table: shown below

image

Like this if you have - numeric or integer data - use  data frame api '  s we can filter the data required as you have mentioned,    

or Create a temporary view using the DataFrame  

// Creates a temporary view using the DataFrame

df1.createOrReplaceTempView("table1")

Then use sql statements to query , if in case age field is in table -  for example

val age = spark.sql("SELECT name FROM table1 WHERE age > 25 ")

age. show()

To save this dataframe as for example csv format use below statement.

age.write.format("csv").save("file path/..././namesAndAges.csv")
answered Jul 24 by Firoz

Related Questions

+1 vote
1 answer
0 votes
1 answer

Scala: Convert text file data into ORC format using data frame

Converting text file to Orc: Using Spark, the ...READ MORE

answered Aug 1 in Apache Spark by Esha
324 views
0 votes
1 answer

Hadoop Mapreduce word count Program

Firstly you need to understand the concept ...READ MORE

answered Mar 16, 2018 in Data Analytics by nitinrawat895
• 10,800 points
3,579 views
0 votes
1 answer

hadoop.mapred vs hadoop.mapreduce?

org.apache.hadoop.mapred is the Old API  org.apache.hadoop.mapreduce is the ...READ MORE

answered Mar 16, 2018 in Data Analytics by nitinrawat895
• 10,800 points
455 views
0 votes
1 answer
0 votes
3 answers

How to read a Text File in Java?

You can use readAllLines and the join method to ...READ MORE

answered Jul 27, 2018 in Java by samarth295
• 2,190 points
130 views
0 votes
2 answers

How to read a text file in Java?

You can use Scanner class to read ...READ MORE

answered Aug 9, 2018 in Java by parth295
• 4,640 points
63 views
0 votes
1 answer

How to read an XML file using XPath in Java?

Import the packages required to work with ...READ MORE

answered Jun 14, 2018 in Java by Akrati
• 3,150 points
971 views
0 votes
1 answer
0 votes
1 answer

How to read file from subprocess.open in python?

you call: out, err = p.communicate() READ MORE

answered Aug 24, 2018 in Blockchain by digger
• 26,600 points
72 views