How to compute the square root of sum of squares of numbers?

0 votes
Hi,

I have a list of numbers in a file in hdfs, each line has one number so how can I compute the square root of sum of squares of numbers? Can anyone suggest?
Jul 22, 2019 in Apache Spark by Liza
279 views

1 answer to this question.

0 votes

Hey,

You need to follow some steps to complete the work, here you can follow:

  • First load the file as RDD from HDFS on spark using the below code:
numsAsText = sc.textFile(“hdfs://hadoop1.knowbigdata.com/user/student/sgiri/mynumbersfile.txt”);
  • Second, define the function to compute the squaresdef to SqInt(str):

v = int(str);

return v*v;
  • Third, run the function on spark rdd as transformation:
nums = numsAsText.map(toSqInt);
  • Fourth, Run the summation as reduce action:
total = nums.reduce(sum)
  • Fifth, finally compute the square root. For which we need to import math:
import math;

print math.sqrt(total);

Hope this helps.

answered Jul 22, 2019 by Gitika
• 25,460 points

Related Questions In Apache Spark

0 votes
1 answer

How to get the number of elements in partition?

rdd.mapPartitions(iter => Array(iter.size).iterator, true) This command will ...READ MORE

answered May 8, 2018 in Apache Spark by kurt_cobain
• 9,290 points
310 views
0 votes
1 answer
0 votes
1 answer
0 votes
7 answers

How to print the contents of RDD in Apache Spark?

Save it to a text file: line.saveAsTextFile("alicia.txt") Print contains ...READ MORE

answered Dec 10, 2018 in Apache Spark by Akshay
16,272 views
+1 vote
1 answer
+1 vote
1 answer

Hadoop Mapreduce word count Program

Firstly you need to understand the concept ...READ MORE

answered Mar 16, 2018 in Data Analytics by nitinrawat895
• 10,840 points
3,907 views
0 votes
1 answer

hadoop.mapred vs hadoop.mapreduce?

org.apache.hadoop.mapred is the Old API  org.apache.hadoop.mapreduce is the ...READ MORE

answered Mar 16, 2018 in Data Analytics by nitinrawat895
• 10,840 points
536 views
+1 vote
11 answers

hadoop fs -put command?

put syntax: put <localSrc> <dest> copy syntax: copyFr ...READ MORE

answered Dec 7, 2018 in Big Data Hadoop by Aditya
20,778 views
0 votes
1 answer

How to calculate the result of formula with Scala?

Hi, You can use a simple mathematical calculation ...READ MORE

answered Jul 1, 2019 in Apache Spark by Gitika
• 25,460 points
65 views
0 votes
1 answer

Which syntax to use to take the sum of list of collection in scala?

Hi, You can see this example to get ...READ MORE

answered Jul 5, 2019 in Apache Spark by Gitika
• 25,460 points
51 views