How to compute the square root of sum of squares of numbers?

0 votes
Hi,

I have a list of numbers in a file in hdfs, each line has one number so how can I compute the square root of sum of squares of numbers? Can anyone suggest?
Jul 22 in Apache Spark by Liza
18 views

1 answer to this question.

0 votes

Hey,

You need to follow some steps to complete the work, here you can follow:

  • First load the file as RDD from HDFS on spark using the below code:
numsAsText = sc.textFile(“hdfs://hadoop1.knowbigdata.com/user/student/sgiri/mynumbersfile.txt”);
  • Second, define the function to compute the squaresdef to SqInt(str):

v = int(str);

return v*v;
  • Third, run the function on spark rdd as transformation:
nums = numsAsText.map(toSqInt);
  • Fourth, Run the summation as reduce action:
total = nums.reduce(sum)
  • Fifth, finally compute the square root. For which we need to import math:
import math;

print math.sqrt(total);

Hope this helps.

answered Jul 22 by Gitika
• 25,300 points

Related Questions In Apache Spark

0 votes
1 answer

How to get the number of elements in partition?

rdd.mapPartitions(iter => Array(iter.size).iterator, true) This command will ...READ MORE

answered May 8, 2018 in Apache Spark by kurt_cobain
• 9,240 points
135 views
0 votes
1 answer
0 votes
1 answer
0 votes
7 answers

How to print the contents of RDD in Apache Spark?

Simple and easy: line.foreach(println) READ MORE

answered Dec 10, 2018 in Apache Spark by Kuber
8,163 views
0 votes
1 answer
0 votes
1 answer

Hadoop Mapreduce word count Program

Firstly you need to understand the concept ...READ MORE

answered Mar 16, 2018 in Data Analytics by nitinrawat895
• 10,490 points
2,305 views
0 votes
1 answer

hadoop.mapred vs hadoop.mapreduce?

org.apache.hadoop.mapred is the Old API  org.apache.hadoop.mapreduce is the ...READ MORE

answered Mar 16, 2018 in Data Analytics by nitinrawat895
• 10,490 points
235 views
0 votes
10 answers

hadoop fs -put command?

put syntax: put <localSrc> <dest> copy syntax: copyFr ...READ MORE

answered Dec 7, 2018 in Big Data Hadoop by Aditya
11,893 views
0 votes
1 answer

How to calculate the result of formula with Scala?

Hi, You can use a simple mathematical calculation ...READ MORE

answered Jul 1 in Apache Spark by Gitika
• 25,300 points
37 views
0 votes
1 answer

Which syntax to use to take the sum of list of collection in scala?

Hi, You can see this example to get ...READ MORE

answered Jul 5 in Apache Spark by Gitika
• 25,300 points
32 views