How to compute the square root of sum of squares of numbers

0 votes
Hi,

I have a list of numbers in a file in hdfs, each line has one number so how can I compute the square root of sum of squares of numbers? Can anyone suggest?
Jul 22, 2019 in Apache Spark by Liza
2,656 views

1 answer to this question.

0 votes

Hey,

You need to follow some steps to complete the work, here you can follow:

  • First load the file as RDD from HDFS on spark using the below code:
numsAsText = sc.textFile(“hdfs://hadoop1.knowbigdata.com/user/student/sgiri/mynumbersfile.txt”);
  • Second, define the function to compute the squaresdef to SqInt(str):

v = int(str);

return v*v;
  • Third, run the function on spark rdd as transformation:
nums = numsAsText.map(toSqInt);
  • Fourth, Run the summation as reduce action:
total = nums.reduce(sum)
  • Fifth, finally compute the square root. For which we need to import math:
import math;

print math.sqrt(total);

Hope this helps.

answered Jul 22, 2019 by Gitika
• 65,870 points

Related Questions In Apache Spark

0 votes
1 answer

How to get the number of elements in partition?

rdd.mapPartitions(iter => Array(iter.size).iterator, true) This command will ...READ MORE

answered May 8, 2018 in Apache Spark by kurt_cobain
• 9,390 points
764 views
0 votes
1 answer
0 votes
1 answer
+1 vote
8 answers

How to print the contents of RDD in Apache Spark?

Save it to a text file: line.saveAsTextFile("alicia.txt") Print contains ...READ MORE

answered Dec 10, 2018 in Apache Spark by Akshay
43,376 views
+1 vote
2 answers
+1 vote
1 answer

Hadoop Mapreduce word count Program

Firstly you need to understand the concept ...READ MORE

answered Mar 16, 2018 in Data Analytics by nitinrawat895
• 11,380 points
7,201 views
0 votes
1 answer

hadoop.mapred vs hadoop.mapreduce?

org.apache.hadoop.mapred is the Old API  org.apache.hadoop.mapreduce is the ...READ MORE

answered Mar 16, 2018 in Data Analytics by nitinrawat895
• 11,380 points
1,168 views
+2 votes
11 answers

hadoop fs -put command?

Hi, You can create one directory in HDFS ...READ MORE

answered Mar 16, 2018 in Big Data Hadoop by nitinrawat895
• 11,380 points
52,743 views
0 votes
1 answer

How to calculate the result of formula with Scala?

Hi, You can use a simple mathematical calculation ...READ MORE

answered Jul 1, 2019 in Apache Spark by Gitika
• 65,870 points
324 views
0 votes
1 answer

Which syntax to use to take the sum of list of collection in scala?

Hi, You can see this example to get ...READ MORE

answered Jul 5, 2019 in Apache Spark by Gitika
• 65,870 points
764 views