How to read json file as dictionary in scala?

+1 vote
Hi,

How to read json file in Scala and use its content?
e. g. jason file having name: ABC, age: 25, Location: XYZ

read these details and print in Scala.
Jul 24, 2019 by Tina
2,526 views

1 answer to this question.

0 votes
val df1= sqlContext.read.json("file:///home/edureka/Desktop/datsets/world_bank.json")  // loads file, give appropriate path

This sql context is used in  version below  2.0 of spark , but this also will work.

  Using spark session fro 2.0 and above use below line

val df1 = spark.read.json("file:///home/edureka/Desktop/datsets/world_bank.json"

df1.printSchema() // provides the schema details

image

df1select("id","countrycode").show

output table: shown below

image

Like this if you have - numeric or integer data - use  data frame api '  s we can filter the data required as you have mentioned,    

or Create a temporary view using the DataFrame  

// Creates a temporary view using the DataFrame

df1.createOrReplaceTempView("table1")

Then use sql statements to query , if in case age field is in table -  for example

val age = spark.sql("SELECT name FROM table1 WHERE age > 25 ")

age. show()

To save this dataframe as for example csv format use below statement.

age.write.format("csv").save("file path/..././namesAndAges.csv")
answered Jul 24, 2019 by Firoz

Related Questions

+1 vote
2 answers
+1 vote
1 answer

Scala: Convert text file data into ORC format using data frame

Converting text file to Orc: Using Spark, the ...READ MORE

answered Aug 1, 2019 in Apache Spark by Esha
1,251 views
+1 vote
1 answer

Hadoop Mapreduce word count Program

Firstly you need to understand the concept ...READ MORE

answered Mar 16, 2018 in Data Analytics by nitinrawat895
• 10,950 points
6,115 views
0 votes
1 answer

hadoop.mapred vs hadoop.mapreduce?

org.apache.hadoop.mapred is the Old API  org.apache.hadoop.mapreduce is the ...READ MORE

answered Mar 16, 2018 in Data Analytics by nitinrawat895
• 10,950 points
933 views
0 votes
1 answer
0 votes
1 answer

how to read csv file form sftp connection and store into string object in java code and convert into json.....post it using rest api

Hey, @Pooja, Before starting with anything you should ...READ MORE

answered May 12 in Java by Roshni
• 4,690 points
507 views
0 votes
1 answer

How to read an external local JSON file in JavaScript?

Hello @kartik, For reading the external Local JSON ...READ MORE

answered Sep 3 in Java-Script by Niroj
• 67,020 points
2,076 views
0 votes
3 answers

How to read a Text File in Java?

You can use readAllLines and the join method to ...READ MORE

answered Jul 27, 2018 in Java by samarth295
• 2,200 points
367 views
0 votes
2 answers

How to read a text file in Java?

You can use Scanner class to read ...READ MORE

answered Aug 9, 2018 in Java by Parth
• 4,640 points
152 views
0 votes
1 answer

How to read an XML file using XPath in Java?

Import the packages required to work with ...READ MORE

answered Jun 14, 2018 in Java by Akrati
• 3,170 points
1,737 views