Error : split value is not a member of org.apache.spark.sql.Row

0 votes

Hi,

Error : split value is not a member of org.apache.spark.sql.Row

I am working on eclipse EdurekaVM, when adding split(",") i am getting this error, please help.

val rdd2col = spark.read.csv("file:///home/edureka/rk/2Cols.csv")
val names = rdd2col.map(line => line.split(",")(0))
Jul 10, 2019 in Apache Spark by Kiran
3,369 views

2 answers to this question.

0 votes

spark.read.csv is used when loading into a dataframe. The error message that you are getting states that the 'split' function that you are using is not a member of spark.sql. It is searching split in spark.sql because of the loading technique.

Please refer to the below commands for your requirement:

val rdd2col = sc.textFile("file:///home/edureka/Documents/datasets/emp.csv")
val names = rdd2col.map(line => line.split(",")(1))
names.collect()
answered Jul 10, 2019 by Rishi
What does (1) mean in map(line => line.split(",")(1))?
It indicates the index value. It means the above code splits the data first and return only those values which are in index 1. You can change it with -1, +1 etc. according to your need.
0 votes
var d=rdd2col.rdd.map(x=>x.split(","))

or

val names=rdd2col.select("name").map(line => line.getString(0)).collect.toList
answered Aug 5 by Ramkumar Ramasamy.

Hey,

Thank you for your contribution to the Edureka Community.

Register/Sign up on the community to gain points for further contributions. You may ask questions, answer, upvote, and downvote an answer. Each of these would fetch you points and you could be among the top contributors and win exciting merchandise from Edureka.

Cheers!

Related Questions In Apache Spark

+1 vote
1 answer

Error: value textfile is not a member of org.apache.spark.SparkContext

Hi, Regarding this error, you just need to change ...READ MORE

answered Jul 4, 2019 in Apache Spark by Gitika
• 41,360 points
1,493 views
0 votes
1 answer

Scala: 30: error: value partitions is not a member of String

Try this code: val rdd= sc.textFile (“file.txt”, 5) rdd.partitions.size Output ...READ MORE

answered Jul 29, 2019 in Apache Spark by Nijit
534 views
0 votes
1 answer

Spark: Error while instantiating "org.apache.spark.sql.hive.HiveSessionState"

Seems like you have not started the ...READ MORE

answered Jul 25, 2019 in Apache Spark by Rohit
3,196 views
+1 vote
2 answers
+1 vote
1 answer

Hadoop Mapreduce word count Program

Firstly you need to understand the concept ...READ MORE

answered Mar 16, 2018 in Data Analytics by nitinrawat895
• 10,950 points
6,070 views
0 votes
1 answer

hadoop.mapred vs hadoop.mapreduce?

org.apache.hadoop.mapred is the Old API  org.apache.hadoop.mapreduce is the ...READ MORE

answered Mar 16, 2018 in Data Analytics by nitinrawat895
• 10,950 points
925 views
+1 vote
11 answers

hadoop fs -put command?

put syntax: put <localSrc> <dest> copy syntax: copyF ...READ MORE

answered Dec 7, 2018 in Big Data Hadoop by Aditya
38,999 views
0 votes
1 answer

Error : split value is not a member of org.apache.spark.sql.Row

spark.read.csv is used when loading into a ...READ MORE

answered Jul 22, 2019 in Apache Spark by Firoz
1,140 views
0 votes
1 answer

Scala: error: value unary_+ is not a member of (Int, Int)

All prefix operators' symbols are predefined: +, -, ...READ MORE

answered Jul 22, 2019 in Apache Spark by karan
514 views