How to Query hdfs with Spark Sql

0 votes

I have a csv file in hdfs, how can I query this file with spark SQL?

Sep 20, 2018 in Blockchain by digger
• 27,620 points
230 views

1 answer to this question.

Your answer

Your name to display (optional):
Privacy: Your email address will only be used for sending these notifications.
0 votes

Yyou can achieve this by creating Dataframe.

val dataFrame = spark.sparkContext
  .textFile("examples/src/main/resources/people.csv")
  .map(_.split(","))
  .map(attributes => Person(attributes(0), attributes(1).trim.toInt))
  .toDF()

dataFrame.sql("<sql query>");
answered Sep 20, 2018 by slayer
• 29,040 points

Related Questions In Blockchain

0 votes
1 answer
0 votes
1 answer

How to interact with blockchain using Java web app?

You can interact with the blockchain using ...READ MORE

answered Jul 16, 2018 in Blockchain by digger
• 27,620 points
104 views
0 votes
1 answer
0 votes
1 answer

Hadoop Mapreduce word count Program

Firstly you need to understand the concept ...READ MORE

answered Mar 16, 2018 in Data Analytics by nitinrawat895
• 9,030 points
1,661 views
0 votes
10 answers

hadoop fs -put command?

copy command can be used to copy files ...READ MORE

answered Dec 7, 2018 in Big Data Hadoop by Sujay
8,049 views
0 votes
1 answer

Hadoop dfs -ls command?

In your case there is no difference ...READ MORE

answered Mar 16, 2018 in Big Data Hadoop by kurt_cobain
• 9,260 points
561 views
0 votes
1 answer
0 votes
1 answer

How to use std::iterator with basic type uint8_t*?

std::reverse_iterator takes the original iterator type as ...READ MORE

answered Aug 24, 2018 in Blockchain by slayer
• 29,040 points
40 views
0 votes
1 answer

How to open bitcoin-qt wallet with bitcoinj?

I found this issue solved at the ...READ MORE

answered Aug 24, 2018 in Blockchain by slayer
• 29,040 points
36 views

© 2018 Brain4ce Education Solutions Pvt. Ltd. All rights Reserved.
"PMP®","PMI®", "PMI-ACP®" and "PMBOK®" are registered marks of the Project Management Institute, Inc. MongoDB®, Mongo and the leaf logo are the registered trademarks of MongoDB, Inc.