How can I compare the elements of the RDD using MapReduce?

Question

How can I find the maximum number of the elements present in the RDD. I want to compare the RDD elements. Can I use loops or conditional statements to compare the elements?

Answer 1

You have to use the comparison operator in a reduce operation.

x = sc.parallelize([1,2,4,5])
max_val = x.reduce(lambda a, b: a if a > b else b)