How to compute the square root of sum of squares of numbers

0 votes
Hi,

I have a list of numbers in a file in hdfs, each line has one number so how can I compute the square root of sum of squares of numbers? Can anyone suggest?
Jul 23, 2019 in Apache Spark by Liza
5,931 views

1 answer to this question.

0 votes

Hey,

You need to follow some steps to complete the work, here you can follow:

  • First load the file as RDD from HDFS on spark using the below code:
numsAsText = sc.textFile(“hdfs://hadoop1.knowbigdata.com/user/student/sgiri/mynumbersfile.txt”);
  • Second, define the function to compute the squaresdef to SqInt(str):

v = int(str);

return v*v;
  • Third, run the function on spark rdd as transformation:
nums = numsAsText.map(toSqInt);
  • Fourth, Run the summation as reduce action:
total = nums.reduce(sum)
  • Fifth, finally compute the square root. For which we need to import math:
import math;

print math.sqrt(total);

Hope this helps.

answered Jul 23, 2019 by Gitika
• 65,770 points

Related Questions In Apache Spark

0 votes
1 answer

How to get the number of elements in partition?

rdd.mapPartitions(iter => Array(iter.size).iterator, true) This command will ...READ MORE

answered May 8, 2018 in Apache Spark by kurt_cobain
• 9,350 points
2,227 views
0 votes
1 answer
0 votes
1 answer
+1 vote
8 answers

How to print the contents of RDD in Apache Spark?

Save it to a text file: line.saveAsTextFile("alicia.txt") Print contains ...READ MORE

answered Dec 10, 2018 in Apache Spark by Akshay
61,868 views
+1 vote
2 answers
+1 vote
1 answer

Hadoop Mapreduce word count Program

Firstly you need to understand the concept ...READ MORE

answered Mar 16, 2018 in Data Analytics by nitinrawat895
• 11,380 points
11,077 views
0 votes
1 answer

hadoop.mapred vs hadoop.mapreduce?

org.apache.hadoop.mapred is the Old API  org.apache.hadoop.mapreduce is the ...READ MORE

answered Mar 16, 2018 in Data Analytics by nitinrawat895
• 11,380 points
2,575 views
+2 votes
11 answers

hadoop fs -put command?

Hi, You can create one directory in HDFS ...READ MORE

answered Mar 16, 2018 in Big Data Hadoop by nitinrawat895
• 11,380 points
109,074 views
0 votes
1 answer

How to calculate the result of formula with Scala?

Hi, You can use a simple mathematical calculation ...READ MORE

answered Jul 1, 2019 in Apache Spark by Gitika
• 65,770 points
1,303 views
0 votes
1 answer

Which syntax to use to take the sum of list of collection in scala?

Hi, You can see this example to get ...READ MORE

answered Jul 5, 2019 in Apache Spark by Gitika
• 65,770 points
1,886 views
webinar REGISTER FOR FREE WEBINAR X
REGISTER NOW
webinar_success Thank you for registering Join Edureka Meetup community for 100+ Free Webinars each month JOIN MEETUP GROUP