You have to use the comparison operator in a reduce operation.
x = sc.parallelize([1,2,4,5])
max_val = x.reduce(lambda a, b: a if a > b else b)
How can I import zip files and ...READ MORE
A Dataframe can be created from an ...READ MORE
You can create a data frame from ...READ MORE
rdd.mapPartitions(iter => Array(iter.size).iterator, true)
This command will ...READ MORE
Firstly you need to understand the concept ...READ MORE
You can create one directory in HDFS ...READ MORE
In your case there is no difference ...READ MORE
The distributed copy command, distcp, is a ...READ MORE
// Collect data from input avro file ...READ MORE
Yes, you can go ahead and write ...READ MORE
Already have an account? Sign in.