What is reduce action in Spark

Question

Can anyone explain how reduce() function works in Spark?

Answer 1

Hey,

It takes a function that operates on two elements of the type in your RDD and returns a new element of the same type. A simple example of such a function is +, which we can use to sum our RDD.

 With reduce(), we can easily sum the elements of our RDD, count the number of elements, and perform other types of aggregations.

Comments

Can you explain in brief like, for the following example of code

rdd = sc.parallelize([1,2,3,4])

rdd.reduce(lambda a, b: a + b)

Can you explain in which way reduce performs sum operation according to below options.

option 1 - (1+2)+3)+4

option 2 - (1+2)+(3+4)

---

It happens as option 1