How to Query hdfs with Spark Sql

0 votes

I have a csv file in hdfs, how can I query this file with spark SQL?

Sep 20, 2018 in Blockchain by digger
• 26,700 points
1,055 views

1 answer to this question.

0 votes

Yyou can achieve this by creating Dataframe.

val dataFrame = spark.sparkContext
  .textFile("examples/src/main/resources/people.csv")
  .map(_.split(","))
  .map(attributes => Person(attributes(0), attributes(1).trim.toInt))
  .toDF()

dataFrame.sql("<sql query>");
answered Sep 20, 2018 by slayer
• 29,300 points

Related Questions In Blockchain

0 votes
1 answer
0 votes
1 answer

How to interact with blockchain using Java web app?

You can interact with the blockchain using ...READ MORE

answered Jul 16, 2018 in Blockchain by digger
• 26,700 points
485 views
0 votes
1 answer

How to achieve consensus with peers from different departments?

Defining one MSP to represent each division. ...READ MORE

answered Jul 16, 2018 in Blockchain by digger
• 26,700 points
158 views
+1 vote
1 answer

Hadoop Mapreduce word count Program

Firstly you need to understand the concept ...READ MORE

answered Mar 16, 2018 in Data Analytics by nitinrawat895
• 11,380 points
6,862 views
+2 votes
11 answers

hadoop fs -put command?

Hi, You can create one directory in HDFS ...READ MORE

answered Mar 16, 2018 in Big Data Hadoop by nitinrawat895
• 11,380 points
48,505 views
–1 vote
1 answer

Hadoop dfs -ls command?

In your case there is no difference ...READ MORE

answered Mar 16, 2018 in Big Data Hadoop by kurt_cobain
• 9,390 points
2,603 views
0 votes
1 answer
0 votes
1 answer

How to use std::iterator with basic type uint8_t*?

std::reverse_iterator takes the original iterator type as ...READ MORE

answered Aug 24, 2018 in Blockchain by slayer
• 29,300 points
253 views
0 votes
1 answer

How to open bitcoin-qt wallet with bitcoinj?

I found this issue solved at the ...READ MORE

answered Aug 24, 2018 in Blockchain by slayer
• 29,300 points
189 views