Error split value is not a member of org apache spark sql Row

0 votes

Hi,

Error : split value is not a member of org.apache.spark.sql.Row

I am working on eclipse EdurekaVM, when adding split(",") i am getting this error, please help.

val rdd2col = spark.read.csv("file:///home/edureka/rk/2Cols.csv")
val names = rdd2col.map(line => line.split(",")(0))
Jul 10, 2019 in Apache Spark by Kiran
4,359 views

2 answers to this question.

0 votes

spark.read.csv is used when loading into a dataframe. The error message that you are getting states that the 'split' function that you are using is not a member of spark.sql. It is searching split in spark.sql because of the loading technique.

Please refer to the below commands for your requirement:

val rdd2col = sc.textFile("file:///home/edureka/Documents/datasets/emp.csv")
val names = rdd2col.map(line => line.split(",")(1))
names.collect()
answered Jul 10, 2019 by Rishi
What does (1) mean in map(line => line.split(",")(1))?
It indicates the index value. It means the above code splits the data first and return only those values which are in index 1. You can change it with -1, +1 etc. according to your need.
0 votes
var d=rdd2col.rdd.map(x=>x.split(","))

or

val names=rdd2col.select("name").map(line => line.getString(0)).collect.toList
answered Aug 5, 2020 by Ramkumar Ramasamy.

Hey,

Thank you for your contribution to the Edureka Community.

Register/Sign up on the community to gain points for further contributions. You may ask questions, answer, upvote, and downvote an answer. Each of these would fetch you points and you could be among the top contributors and win exciting merchandise from Edureka.

Cheers!

Related Questions In Apache Spark

+1 vote
1 answer

Error: value textfile is not a member of org.apache.spark.SparkContext

Hi, Regarding this error, you just need to change ...READ MORE

answered Jul 4, 2019 in Apache Spark by Gitika
• 65,870 points
1,848 views
0 votes
1 answer

Scala: 30: error: value partitions is not a member of String

Try this code: val rdd= sc.textFile (“file.txt”, 5) rdd.partitions.size Output ...READ MORE

answered Jul 29, 2019 in Apache Spark by Nijit
807 views
0 votes
1 answer

Spark: Error while instantiating "org.apache.spark.sql.hive.HiveSessionState"

Seems like you have not started the ...READ MORE

answered Jul 25, 2019 in Apache Spark by Rohit
3,917 views
+1 vote
2 answers
+1 vote
1 answer

Hadoop Mapreduce word count Program

Firstly you need to understand the concept ...READ MORE

answered Mar 16, 2018 in Data Analytics by nitinrawat895
• 11,380 points
6,814 views
0 votes
1 answer

hadoop.mapred vs hadoop.mapreduce?

org.apache.hadoop.mapred is the Old API  org.apache.hadoop.mapreduce is the ...READ MORE

answered Mar 16, 2018 in Data Analytics by nitinrawat895
• 11,380 points
1,092 views
+2 votes
11 answers

hadoop fs -put command?

Hi, You can create one directory in HDFS ...READ MORE

answered Mar 16, 2018 in Big Data Hadoop by nitinrawat895
• 11,380 points
47,859 views
0 votes
1 answer

Error : split value is not a member of org.apache.spark.sql.Row

spark.read.csv is used when loading into a ...READ MORE

answered Jul 22, 2019 in Apache Spark by Firoz
1,439 views
0 votes
1 answer

Scala: error: value unary_+ is not a member of (Int, Int)

All prefix operators' symbols are predefined: +, -, ...READ MORE

answered Jul 22, 2019 in Apache Spark by karan
703 views