Iterate twice on values MapReduce

0 votes

I receive an iterator as argument and I would like to iterate on values twice.

public void reduce(Pair<String,String> key, Iterator<IntWritable> values,
                   Context context)

Is it possible ? How ? The signature is imposed by the framework I am using (namely Hadoop).

Oct 1, 2018 in Big Data Hadoop by slayer
• 29,350 points
1,882 views

1 answer to this question.

0 votes

We have to cache the values from the iterator if you want to iterate again. At least we can combine the first iteration and the caching:

Iterator<IntWritable> it = getIterator();
List<IntWritable> cache = new ArrayList<IntWritable>();

// first loop and caching
while (it.hasNext()) {
   IntWritable value = it.next();
   doSomethingWithValue();
   cache.add(value);
}

// second loop
for(IntWritable value:cache) {
   doSomethingElseThatCantBeDoneInFirstLoop(value);
}
answered Oct 1, 2018 by digger
• 26,740 points
"Por supuesto respetarlo, ser coherente con el planteamiento y yo que desde el primer momento he hech."
zapatillas nike air force mujer 2018 http://www.shoesharer.com/D-zapatillas-nike-air-force-mujer-2018

Related Questions In Big Data Hadoop

0 votes
1 answer
0 votes
1 answer

Running Mapreduce on compressed data

It is very straight forward, no need ...READ MORE

answered Jul 24, 2019 in Big Data Hadoop by Nanda
636 views
0 votes
1 answer

How to install Hadoop on Ubuntu?

You can refer to this blog by ...READ MORE

answered Mar 21, 2018 in Big Data Hadoop by nitinrawat895
• 11,380 points
1,188 views
0 votes
1 answer

Hadoop MapReduce vs MongoDB MapReduce

Hadoop vs MongoDB MR: 1) Hadoop's MR can ...READ MORE

answered Mar 26, 2018 in Big Data Hadoop by nitinrawat895
• 11,380 points
1,177 views
+1 vote
1 answer

Hadoop Installation Issue on Windows

Below is the main error you are ...READ MORE

answered Mar 26, 2018 in Big Data Hadoop by nitinrawat895
• 11,380 points
5,683 views
+1 vote
1 answer

Hadoop Mapreduce word count Program

Firstly you need to understand the concept ...READ MORE

answered Mar 16, 2018 in Data Analytics by nitinrawat895
• 11,380 points
10,558 views
+2 votes
11 answers

hadoop fs -put command?

Hi, You can create one directory in HDFS ...READ MORE

answered Mar 16, 2018 in Big Data Hadoop by nitinrawat895
• 11,380 points
104,213 views
–1 vote
1 answer

Hadoop dfs -ls command?

In your case there is no difference ...READ MORE

answered Mar 16, 2018 in Big Data Hadoop by kurt_cobain
• 9,390 points
4,260 views
0 votes
1 answer
webinar REGISTER FOR FREE WEBINAR X
REGISTER NOW
webinar_success Thank you for registering Join Edureka Meetup community for 100+ Free Webinars each month JOIN MEETUP GROUP