1 Given sfpd RDD to create a pair RDD consisting of tuples of the form Category 1 in scala which of the following is used

0 votes
1)Given sfpd RDD, to create a pair RDD consisting of tuples of the form (Category. 1) in scala ,which of the following is used?

a) val pairs =sfpd.parallelize()

b) val pairs =sfpd.map(x=>(x(Category),1))

c) val pairs =sfpd.map(x=>x.parallize))

d) val pairs =sfpd.map(x=>x.unparallelize))
Nov 22, 2020 in Apache Spark by ritu
• 980 points
162 views

1 answer to this question.

0 votes

Hi, @Ritu,

When creating a pair RDD from an in-memory collection in Scala and Python, we only need to call SparkContext.parallelize() on a collection of pairs. To create a pair RDD in Java from an in-memory collection, we instead use SparkContext.parallelizePairs().

Option c for your query.

answered Nov 23, 2020 by Gitika
• 65,870 points

Related Questions In Apache Spark

0 votes
1 answer

Which one of the following commands is used to start python-spark?

Hi@ritu, To start your python spark shell, you ...READ MORE

answered Nov 26, 2020 in Apache Spark by MD
• 95,060 points
66 views
0 votes
1 answer

Which syntax to use to take the sum of list of collection in scala?

Hi, You can see this example to get ...READ MORE

answered Jul 5, 2019 in Apache Spark by Gitika
• 65,870 points
650 views
0 votes
1 answer

How to use yield keyword in scala and why it is used instead of println?

Hi, The yield keyword is used because the ...READ MORE

answered Jul 5, 2019 in Apache Spark by Gitika
• 65,870 points
444 views
0 votes
1 answer
0 votes
1 answer

which one of the following commands is used to see the structure of the Dataframe?

Hi @Ritu If you want to see the ...READ MORE

answered Nov 25, 2020 in Apache Spark by Gitika
• 65,870 points
96 views