You have to use the comparison operator in a reduce operation.
x = sc.parallelize([1,2,4,5])
max_val = x.reduce(lambda a, b: a if a > b else b)
How can I import zip files and ...READ MORE
rdd.mapPartitions(iter => Array(iter.size).iterator, true)
This command will ...READ MORE
You can select the column and apply ...READ MORE
Assuming your RDD[row] is called rdd, you ...READ MORE
Firstly you need to understand the concept ...READ MORE
put <localSrc> <dest>
copyF ...READ MORE
In your case there is no difference ...READ MORE
The distributed copy command, distcp, is a ...READ MORE
// Collect data from input avro file ...READ MORE
Yes, you can go ahead and write ...READ MORE
Already have an account? Sign in.