How to compute the square root of sum of squares of numbers

Question

Hi,

I have a list of numbers in a file in hdfs, each line has one number so how can I compute the square root of sum of squares of numbers? Can anyone suggest?

Answer 1

Hey,

You need to follow some steps to complete the work, here you can follow:

• First load the file as RDD from HDFS on spark using the below code:

numsAsText = sc.textFile(“hdfs://hadoop1.knowbigdata.com/user/student/sgiri/mynumbersfile.txt”);

• Second, define the function to compute the squaresdef to SqInt(str):

v = int(str);

return v*v;

• Third, run the function on spark rdd as transformation:

nums = numsAsText.map(toSqInt);

• Fourth, Run the summation as reduce action:

total = nums.reduce(sum)

• Fifth, finally compute the square root. For which we need to import math:

import math;

print math.sqrt(total);

Hope this helps.