How to Query hdfs with Spark Sql

0 votes

I have a csv file in hdfs, how can I query this file with spark SQL?

Sep 20, 2018 in Blockchain by digger
• 26,600 points
500 views

1 answer to this question.

0 votes

Yyou can achieve this by creating Dataframe.

val dataFrame = spark.sparkContext
  .textFile("examples/src/main/resources/people.csv")
  .map(_.split(","))
  .map(attributes => Person(attributes(0), attributes(1).trim.toInt))
  .toDF()

dataFrame.sql("<sql query>");
answered Sep 20, 2018 by slayer
• 29,240 points

Related Questions In Blockchain

0 votes
1 answer
0 votes
1 answer

How to interact with blockchain using Java web app?

You can interact with the blockchain using ...READ MORE

answered Jul 16, 2018 in Blockchain by digger
• 26,600 points
232 views
0 votes
1 answer
0 votes
1 answer

Hadoop Mapreduce word count Program

Firstly you need to understand the concept ...READ MORE

answered Mar 16, 2018 in Data Analytics by nitinrawat895
• 10,760 points
3,551 views
+1 vote
11 answers

hadoop fs -put command?

put syntax: put <localSrc> <dest> copy syntax: copyFr ...READ MORE

answered Dec 7, 2018 in Big Data Hadoop by Aditya
18,189 views
0 votes
1 answer

Hadoop dfs -ls command?

In your case there is no difference ...READ MORE

answered Mar 16, 2018 in Big Data Hadoop by kurt_cobain
• 9,280 points
1,318 views
0 votes
1 answer
0 votes
1 answer

How to use std::iterator with basic type uint8_t*?

std::reverse_iterator takes the original iterator type as ...READ MORE

answered Aug 24, 2018 in Blockchain by slayer
• 29,240 points
89 views
0 votes
1 answer

How to open bitcoin-qt wallet with bitcoinj?

I found this issue solved at the ...READ MORE

answered Aug 24, 2018 in Blockchain by slayer
• 29,240 points
74 views