Iterate twice on values (MapReduce)

0 votes

I receive an iterator as argument and I would like to iterate on values twice.

public void reduce(Pair<String,String> key, Iterator<IntWritable> values,
                   Context context)

Is it possible ? How ? The signature is imposed by the framework I am using (namely Hadoop).

Oct 1, 2018 in Big Data Hadoop by slayer
• 29,170 points
233 views

1 answer to this question.

0 votes

We have to cache the values from the iterator if you want to iterate again. At least we can combine the first iteration and the caching:

Iterator<IntWritable> it = getIterator();
List<IntWritable> cache = new ArrayList<IntWritable>();

// first loop and caching
while (it.hasNext()) {
   IntWritable value = it.next();
   doSomethingWithValue();
   cache.add(value);
}

// second loop
for(IntWritable value:cache) {
   doSomethingElseThatCantBeDoneInFirstLoop(value);
}
answered Oct 1, 2018 by digger
• 26,550 points
"Por supuesto respetarlo, ser coherente con el planteamiento y yo que desde el primer momento he hech."
zapatillas nike air force mujer 2018 http://www.shoesharer.com/D-zapatillas-nike-air-force-mujer-2018

Related Questions In Big Data Hadoop

0 votes
1 answer
0 votes
1 answer

Running Mapreduce on compressed data

It is very straight forward, no need ...READ MORE

answered Jul 24 in Big Data Hadoop by Nanda
22 views
0 votes
1 answer

How to install Hadoop on Ubuntu?

You can refer to this blog by ...READ MORE

answered Mar 21, 2018 in Big Data Hadoop by nitinrawat895
• 10,730 points
369 views
0 votes
1 answer

Hadoop MapReduce vs MongoDB MapReduce

Hadoop vs MongoDB MR: 1) Hadoop's MR can ...READ MORE

answered Mar 26, 2018 in Big Data Hadoop by nitinrawat895
• 10,730 points
198 views
+1 vote
1 answer

Hadoop Installation Issue on Windows

Below is the main error you are ...READ MORE

answered Mar 26, 2018 in Big Data Hadoop by nitinrawat895
• 10,730 points
854 views
0 votes
1 answer

Hadoop Mapreduce word count Program

Firstly you need to understand the concept ...READ MORE

answered Mar 16, 2018 in Data Analytics by nitinrawat895
• 10,730 points
3,356 views
0 votes
10 answers

hadoop fs -put command?

put syntax: put <localSrc> <dest> copy syntax: copyFr ...READ MORE

answered Dec 7, 2018 in Big Data Hadoop by Aditya
16,627 views
0 votes
1 answer

Hadoop dfs -ls command?

In your case there is no difference ...READ MORE

answered Mar 16, 2018 in Big Data Hadoop by kurt_cobain
• 9,260 points
1,214 views
0 votes
1 answer