How to Query hdfs with Spark Sql

0 votes

I have a csv file in hdfs, how can I query this file with spark SQL?

Sep 20, 2018 in Blockchain by digger
• 26,550 points
398 views

1 answer to this question.

0 votes

Yyou can achieve this by creating Dataframe.

val dataFrame = spark.sparkContext
  .textFile("examples/src/main/resources/people.csv")
  .map(_.split(","))
  .map(attributes => Person(attributes(0), attributes(1).trim.toInt))
  .toDF()

dataFrame.sql("<sql query>");
answered Sep 20, 2018 by slayer
• 29,050 points

Related Questions In Blockchain

0 votes
1 answer
0 votes
1 answer

How to interact with blockchain using Java web app?

You can interact with the blockchain using ...READ MORE

answered Jul 16, 2018 in Blockchain by digger
• 26,550 points
188 views
0 votes
1 answer
0 votes
1 answer

Hadoop Mapreduce word count Program

Firstly you need to understand the concept ...READ MORE

answered Mar 16, 2018 in Data Analytics by nitinrawat895
• 10,670 points
2,745 views
0 votes
10 answers

hadoop fs -put command?

put syntax: put <localSrc> <dest> copy syntax: copyFr ...READ MORE

answered Dec 7, 2018 in Big Data Hadoop by Aditya
13,598 views
0 votes
1 answer

Hadoop dfs -ls command?

In your case there is no difference ...READ MORE

answered Mar 16, 2018 in Big Data Hadoop by kurt_cobain
• 9,240 points
999 views
0 votes
1 answer
0 votes
1 answer

How to use std::iterator with basic type uint8_t*?

std::reverse_iterator takes the original iterator type as ...READ MORE

answered Aug 24, 2018 in Blockchain by slayer
• 29,050 points
65 views
0 votes
1 answer

How to open bitcoin-qt wallet with bitcoinj?

I found this issue solved at the ...READ MORE

answered Aug 24, 2018 in Blockchain by slayer
• 29,050 points
57 views