Scala 30 error value partitions is not a member of String

0 votes
val emp_data =spark.sparkContext.textFile("dataset");

val emp_header=emp_data.first()

val emp_data_without_header =emp_header.filter(x=>(!x.equals(emp_header)))

println("No.ofpartition="+emp_data_without_header.partitions.size)
30: error: value partitions is not a member of String emp_data_without_header.partitions.size ^

How to check the partitions for the above statement?

Jul 29, 2019 in Apache Spark by Kamal
1,209 views

1 answer to this question.

0 votes

Try this code:

val rdd= sc.textFile (“file.txt”, 5)

rdd.partitions.size

Output = 5

Hope it helps!

If you want to know more about Apache Spark Scala, It's highly recommended to go for Apache Spark certification course today.

Thanks!!

answered Jul 29, 2019 by Nijit

Related Questions In Apache Spark

0 votes
1 answer

Scala: error: value unary_+ is not a member of (Int, Int)

All prefix operators' symbols are predefined: +, -, ...READ MORE

answered Jul 22, 2019 in Apache Spark by karan
1,118 views
+1 vote
1 answer

Error: value textfile is not a member of org.apache.spark.SparkContext

Hi, Regarding this error, you just need to change ...READ MORE

answered Jul 4, 2019 in Apache Spark by Gitika
• 65,970 points
2,510 views
0 votes
2 answers

Error : split value is not a member of org.apache.spark.sql.Row

var d=rdd2col.rdd.map(x=>x.split(",")) or val names=rd ...READ MORE

answered Aug 5, 2020 in Apache Spark by Ramkumar Ramasamy.
6,377 views
0 votes
1 answer

Error : split value is not a member of org.apache.spark.sql.Row

spark.read.csv is used when loading into a ...READ MORE

answered Jul 22, 2019 in Apache Spark by Firoz
1,900 views
+1 vote
2 answers
0 votes
1 answer
+1 vote
1 answer

Hadoop Mapreduce word count Program

Firstly you need to understand the concept ...READ MORE

answered Mar 16, 2018 in Data Analytics by nitinrawat895
• 11,380 points
8,034 views
0 votes
1 answer

hadoop.mapred vs hadoop.mapreduce?

org.apache.hadoop.mapred is the Old API  org.apache.hadoop.mapreduce is the ...READ MORE

answered Mar 16, 2018 in Data Analytics by nitinrawat895
• 11,380 points
1,366 views
0 votes
1 answer
0 votes
1 answer

Appending " to a string in Scala

1) Use the concat() function. Refer to the below ...READ MORE

answered Jul 23, 2019 in Apache Spark by Ritu
142 views