How can we send data from MongoDB to Hadoop?

0 votes

In my scenario, I want to get the daily inserted data from MongoDB (roughly around 10MB) and put that all into Hadoop. Also MongoDB node and Hadoop node runs on the same server.

I'm not getting how to do this? Please help me out.

Mar 26, 2018 in Big Data Hadoop by Atul
• 5,530 points
192 views

1 answer to this question.

0 votes
The MongoDB Connector for Hadoop reads data directly from MongoDB. You can configure multiple input splits to read data from the same collection in parallel. The Mapper and Reducer jobs are run by Hadoop's Map/Reduce engine, not MongoDB's Map/Reduce.

If your data estimate is correct (only 10MB per day?) that is a small amount to ingest and the job may be faster if you don't have any input splits calculated.

You should be wary of Hadoop and MongoDB competing for resources on the same server, as contention for memory or disk can affect the efficiency of your data transfer.
answered Mar 26, 2018 by nitinrawat895
• 10,800 points

Related Questions In Big Data Hadoop

0 votes
1 answer

How can you transfer data from hive to HDFS ?

Hey, Data in Hive tables reside on HDFS, ...READ MORE

answered May 16 in Big Data Hadoop by Gitika
• 25,420 points
159 views
0 votes
1 answer
0 votes
1 answer

How can we transfer a PDF file to HDFS?

You can easily upload any file to ...READ MORE

answered Apr 13, 2018 in Big Data Hadoop by nitinrawat895
• 10,800 points
91 views
0 votes
1 answer
0 votes
1 answer

Hadoop dfs -ls command?

In your case there is no difference ...READ MORE

answered Mar 16, 2018 in Big Data Hadoop by kurt_cobain
• 9,280 points
1,325 views
0 votes
1 answer

Hadoop Mapreduce word count Program

Firstly you need to understand the concept ...READ MORE

answered Mar 16, 2018 in Data Analytics by nitinrawat895
• 10,800 points
3,563 views
0 votes
1 answer

How to get started with Hadoop?

Well, hadoop is actually a framework that ...READ MORE

answered Mar 21, 2018 in Big Data Hadoop by coldcode
• 2,040 points
114 views
+1 vote
11 answers

hadoop fs -put command?

put syntax: put <localSrc> <dest> copy syntax: copyFr ...READ MORE

answered Dec 7, 2018 in Big Data Hadoop by Aditya
18,323 views
0 votes
1 answer

How to move data from Oracle database to Hadoop?

Yes, you heard it correctly. Apache Sqoop is ...READ MORE

answered Apr 11, 2018 in Big Data Hadoop by nitinrawat895
• 10,800 points
2,356 views
0 votes
1 answer

How to create a FileSystem object that can be used for reading from and writing to HDFS?

Read operation on HDFS In order to read ...READ MORE

answered Mar 21, 2018 in Big Data Hadoop by nitinrawat895
• 10,800 points

edited Mar 21, 2018 by nitinrawat895 401 views