How to decide number of mappers?

0 votes

While running a mapreduce job in Hadoop, how to decide how many mappers are needed?

Jan 30, 2019 in Big Data Hadoop by Raksha
642 views

1 answer to this question.

+1 vote

The number of mapper depends on the total size of the input. i.e. the total number of blocks of the input files.

Mapper= {(total data size)/ (input split size)}

If data size= 1 Tb and input split size= 100 MB

Then, Mapper= (1000*1000)/100= 10,000

answered Jan 30, 2019 by Omkar
• 69,000 points

Related Questions In Big Data Hadoop

0 votes
1 answer

How to set the number of Map & Reduce tasks?

The map tasks created for a job ...READ MORE

answered Apr 18, 2018 in Big Data Hadoop by Shubham
• 13,380 points
139 views
0 votes
1 answer

How to calculate number of slave nodes?

Total number of slave nodes in your ...READ MORE

answered Aug 16, 2018 in Big Data Hadoop by nitinrawat895
• 10,870 points
70 views
0 votes
1 answer

How to find the number of blocks a hdfs file is divided into?

Yes. you can use the hadoop fsck command to do ...READ MORE

answered Nov 29, 2018 in Big Data Hadoop by Omkar
• 69,000 points
1,245 views
+1 vote
1 answer

How to limit the number of rows per each item in a Hive QL?

SELECT a_id, b, c, count(*) as sumrequests FROM ...READ MORE

answered Nov 30, 2018 in Big Data Hadoop by Omkar
• 69,000 points
4,811 views
+1 vote
1 answer

Hadoop Mapreduce word count Program

Firstly you need to understand the concept ...READ MORE

answered Mar 16, 2018 in Data Analytics by nitinrawat895
• 10,870 points
4,612 views
0 votes
1 answer

hadoop.mapred vs hadoop.mapreduce?

org.apache.hadoop.mapred is the Old API  org.apache.hadoop.mapreduce is the ...READ MORE

answered Mar 16, 2018 in Data Analytics by nitinrawat895
• 10,870 points
654 views
+1 vote
11 answers

hadoop fs -put command?

put syntax: put <localSrc> <dest> copy syntax: copyFr ...READ MORE

answered Dec 7, 2018 in Big Data Hadoop by Aditya
26,234 views
–1 vote
1 answer

Hadoop dfs -ls command?

In your case there is no difference ...READ MORE

answered Mar 16, 2018 in Big Data Hadoop by kurt_cobain
• 9,310 points
1,728 views
+1 vote
1 answer

How to count number of rows in alias in PIG?

COUNT is part of pig LOGS= LOAD 'log'; LOGS_GROUP= ...READ MORE

answered Oct 15, 2018 in Big Data Hadoop by Omkar
• 69,000 points
310 views
0 votes
1 answer

Hadoop hdfs: How to count number of lines?

You can count the number of lines ...READ MORE

answered Nov 19, 2018 in Big Data Hadoop by Omkar
• 69,000 points
165 views