Scala 30 error value partitions is not a member of String

0 votes
val emp_data =spark.sparkContext.textFile("dataset");

val emp_header=emp_data.first()

val emp_data_without_header =emp_header.filter(x=>(!x.equals(emp_header)))

println("No.ofpartition="+emp_data_without_header.partitions.size)
30: error: value partitions is not a member of String emp_data_without_header.partitions.size ^

How to check the partitions for the above statement?

Jul 29, 2019 in Apache Spark by Kamal
1,565 views

1 answer to this question.

0 votes

Try this code:

val rdd= sc.textFile (“file.txt”, 5)

rdd.partitions.size

Output = 5

Hope it helps!

If you want to know more about Apache Spark Scala, It's highly recommended to go for Apache Spark certification course today.

Thanks!!

answered Jul 29, 2019 by Nijit

Related Questions In Apache Spark

0 votes
1 answer

Scala: error: value unary_+ is not a member of (Int, Int)

All prefix operators' symbols are predefined: +, -, ...READ MORE

answered Jul 22, 2019 in Apache Spark by karan
1,484 views
+1 vote
1 answer

Error: value textfile is not a member of org.apache.spark.SparkContext

Hi, Regarding this error, you just need to change ...READ MORE

answered Jul 4, 2019 in Apache Spark by Gitika
• 65,850 points
2,911 views
0 votes
2 answers

Error : split value is not a member of org.apache.spark.sql.Row

var d=rdd2col.rdd.map(x=>x.split(",")) or val names=rd ...READ MORE

answered Aug 5, 2020 in Apache Spark by Ramkumar Ramasamy.
7,806 views
0 votes
1 answer

Error : split value is not a member of org.apache.spark.sql.Row

spark.read.csv is used when loading into a ...READ MORE

answered Jul 22, 2019 in Apache Spark by Firoz
2,129 views
+1 vote
2 answers
0 votes
1 answer
+1 vote
1 answer

Hadoop Mapreduce word count Program

Firstly you need to understand the concept ...READ MORE

answered Mar 16, 2018 in Data Analytics by nitinrawat895
• 11,380 points
8,788 views
0 votes
1 answer

hadoop.mapred vs hadoop.mapreduce?

org.apache.hadoop.mapred is the Old API  org.apache.hadoop.mapreduce is the ...READ MORE

answered Mar 16, 2018 in Data Analytics by nitinrawat895
• 11,380 points
1,561 views
0 votes
1 answer
0 votes
1 answer

Appending " to a string in Scala

1) Use the concat() function. Refer to the below ...READ MORE

answered Jul 23, 2019 in Apache Spark by Ritu
185 views
webinar REGISTER FOR FREE WEBINAR X
Send OTP
REGISTER NOW
webinar_success Thank you for registering Join Edureka Meetup community for 100+ Free Webinars each month JOIN MEETUP GROUP