How can we send data from MongoDB to Hadoop

0 votes

In my scenario, I want to get the daily inserted data from MongoDB (roughly around 10MB) and put that all into Hadoop. Also MongoDB node and Hadoop node runs on the same server.

I'm not getting how to do this? Please help me out.

Mar 26, 2018 in Big Data Hadoop by Atul
• 5,550 points
793 views

1 answer to this question.

0 votes
The MongoDB Connector for Hadoop reads data directly from MongoDB. You can configure multiple input splits to read data from the same collection in parallel. The Mapper and Reducer jobs are run by Hadoop's Map/Reduce engine, not MongoDB's Map/Reduce.

If your data estimate is correct (only 10MB per day?) that is a small amount to ingest and the job may be faster if you don't have any input splits calculated.

You should be wary of Hadoop and MongoDB competing for resources on the same server, as contention for memory or disk can affect the efficiency of your data transfer.
answered Mar 26, 2018 by nitinrawat895
• 11,380 points

Related Questions In Big Data Hadoop

0 votes
1 answer

How can you transfer data from hive to HDFS ?

Hey, Data in Hive tables reside on HDFS, ...READ MORE

answered May 16, 2019 in Big Data Hadoop by Gitika
• 65,850 points
721 views
0 votes
1 answer

How can I import data from mysql to hive tables with incremental data?

Hi@dharmendra, It is common to perform one-time ingestion ...READ MORE

answered Nov 23, 2020 in Big Data Hadoop by MD
• 94,990 points
78 views
0 votes
1 answer
0 votes
1 answer

How can we transfer a PDF file to HDFS?

You can easily upload any file to ...READ MORE

answered Apr 13, 2018 in Big Data Hadoop by nitinrawat895
• 11,380 points
195 views
–1 vote
1 answer

Hadoop dfs -ls command?

In your case there is no difference ...READ MORE

answered Mar 16, 2018 in Big Data Hadoop by kurt_cobain
• 9,390 points
2,508 views
+1 vote
1 answer

Hadoop Mapreduce word count Program

Firstly you need to understand the concept ...READ MORE

answered Mar 16, 2018 in Data Analytics by nitinrawat895
• 11,380 points
6,634 views
0 votes
1 answer

How to get started with Hadoop?

Well, hadoop is actually a framework that ...READ MORE

answered Mar 21, 2018 in Big Data Hadoop by coldcode
• 2,080 points
324 views
+2 votes
11 answers

hadoop fs -put command?

Hi, You can create one directory in HDFS ...READ MORE

answered Mar 16, 2018 in Big Data Hadoop by nitinrawat895
• 11,380 points
45,310 views
0 votes
1 answer

How to move data from Oracle database to Hadoop?

Yes, you heard it correctly. Apache Sqoop is ...READ MORE

answered Apr 11, 2018 in Big Data Hadoop by nitinrawat895
• 11,380 points
4,194 views
0 votes
1 answer

How to create a FileSystem object that can be used for reading from and writing to HDFS?

Read operation on HDFS In order to read ...READ MORE

answered Mar 21, 2018 in Big Data Hadoop by nitinrawat895
• 11,380 points

edited Mar 21, 2018 by nitinrawat895 1,208 views