What is meant by Transformation Give some examples

Question

Can anyone explain what is transformation in Spark?

Answer 1

Hi,

The transformations are the functions that are applied to an RDD (resilient distributed dataset). The transformation results in another RDD. A transformation is not executed until an action follows.

Some examples of transformation are:

1. map() – applies the function passed to it on each element of RDD resulting in a new RDD.
2. filter() – creates a new RDD by picking the elements from the current RDD which pass the function provided as an argument