How to compute the square root of sum of squares of numbers?

0 votes
Hi,

I have a list of numbers in a file in hdfs, each line has one number so how can I compute the square root of sum of squares of numbers? Can anyone suggest?
Jul 22, 2019 in Apache Spark by Liza
1,283 views

1 answer to this question.

0 votes

Hey,

You need to follow some steps to complete the work, here you can follow:

  • First load the file as RDD from HDFS on spark using the below code:
numsAsText = sc.textFile(“hdfs://hadoop1.knowbigdata.com/user/student/sgiri/mynumbersfile.txt”);
  • Second, define the function to compute the squaresdef to SqInt(str):

v = int(str);

return v*v;
  • Third, run the function on spark rdd as transformation:
nums = numsAsText.map(toSqInt);
  • Fourth, Run the summation as reduce action:
total = nums.reduce(sum)
  • Fifth, finally compute the square root. For which we need to import math:
import math;

print math.sqrt(total);

Hope this helps.

answered Jul 22, 2019 by Gitika
• 34,050 points

Related Questions In Apache Spark

0 votes
1 answer

How to get the number of elements in partition?

rdd.mapPartitions(iter => Array(iter.size).iterator, true) This command will ...READ MORE

answered May 8, 2018 in Apache Spark by kurt_cobain
• 9,310 points
500 views
0 votes
1 answer
0 votes
1 answer
0 votes
7 answers

How to print the contents of RDD in Apache Spark?

Save it to a text file: line.saveAsTextFile("alicia.txt") Print contains ...READ MORE

answered Dec 10, 2018 in Apache Spark by Akshay
29,942 views
+1 vote
2 answers
+1 vote
1 answer

Hadoop Mapreduce word count Program

Firstly you need to understand the concept ...READ MORE

answered Mar 16, 2018 in Data Analytics by nitinrawat895
• 10,920 points
5,530 views
0 votes
1 answer

hadoop.mapred vs hadoop.mapreduce?

org.apache.hadoop.mapred is the Old API  org.apache.hadoop.mapreduce is the ...READ MORE

answered Mar 16, 2018 in Data Analytics by nitinrawat895
• 10,920 points
818 views
+1 vote
11 answers

hadoop fs -put command?

put syntax: put <localSrc> <dest> copy syntax: copyF ...READ MORE

answered Dec 7, 2018 in Big Data Hadoop by Aditya
34,318 views
0 votes
1 answer

How to calculate the result of formula with Scala?

Hi, You can use a simple mathematical calculation ...READ MORE

answered Jul 1, 2019 in Apache Spark by Gitika
• 34,050 points
144 views
0 votes
1 answer

Which syntax to use to take the sum of list of collection in scala?

Hi, You can see this example to get ...READ MORE

answered Jul 5, 2019 in Apache Spark by Gitika
• 34,050 points
250 views