I am install hadoop on signal node and run program but delay time if have dataset size 3.3 GB take time about 2 hours

0 votes
Apr 12, 2019 in Big Data Hadoop by Faris
• 120 points
117 views

1 answer to this question.

0 votes

Hey @Faris!

Try increasing the number of reducers. When you increase the number of reducers, the distribution is better. I have seen in a few cases that only 1 reducer is used and this causes delay. You can set the number of reducers while running the MapReduce program. Refer to the below command:

$ hadoop <the options that you usually pass> -D mapred.reduce.tasks=<number of reducers>
answered Apr 12, 2019 by Omkar
• 69,040 points
Mean increasing number slaves?
No, I mean the reducer. You are running a mapreduce program. And the job will be divided to mappers and reducers. So, I think increasing the number of reducers might help.
How i can increasing number reduce ;please help me
  1. Export your MapReduce Java project to JAR file. (Refer to this: https://www.tutorialspoint.com/eclipse/eclipse_create_jar_files.htm)
  2. Open terminal and go to the directory where you have exported the JAR file. 
  3. Run the MapReduce using the following command syntax:
$ hadoop jar <jar file name> <Java class name with packagename(Ex:com.hadoop.WordCount)> <hdfs input path> <hdfs output path>

Related Questions In Big Data Hadoop

0 votes
1 answer

I have installed Hadoop on Ubuntu but name node is not running.

If you are using Hadoop version-2.7.7, then ...READ MORE

answered Apr 30, 2019 in Big Data Hadoop by Gitika
• 31,390 points
362 views
+1 vote
1 answer

Hadoop Mapreduce word count Program

Firstly you need to understand the concept ...READ MORE

answered Mar 16, 2018 in Data Analytics by nitinrawat895
• 10,920 points
5,278 views
0 votes
1 answer

hadoop.mapred vs hadoop.mapreduce?

org.apache.hadoop.mapred is the Old API  org.apache.hadoop.mapreduce is the ...READ MORE

answered Mar 16, 2018 in Data Analytics by nitinrawat895
• 10,920 points
770 views
+1 vote
11 answers

hadoop fs -put command?

put syntax: put <localSrc> <dest> copy syntax: copyF ...READ MORE

answered Dec 7, 2018 in Big Data Hadoop by Aditya
32,154 views
–1 vote
1 answer

Hadoop dfs -ls command?

In your case there is no difference ...READ MORE

answered Mar 16, 2018 in Big Data Hadoop by kurt_cobain
• 9,310 points
1,986 views
0 votes
1 answer

Hadoop: “Unhealthy Node local-dirs and log-dirs are bad”

You can increase the threshold in yarn-site.xml <property> ...READ MORE

answered Nov 8, 2018 in Big Data Hadoop by Omkar
• 69,040 points
1,259 views
0 votes
1 answer

Class not found exception when I am running my Word Count Program jar file

You have forgotten to include the package name ...READ MORE

answered Jan 18, 2019 in Big Data Hadoop by Omkar
• 69,040 points
95 views