Can I run Hadoop with Docker for both DEV and PROD environments?

0 votes
Is there an easiest way that works for both dev' and real environments so that to run Hadoop with Docker?

This query is for the local development environment and real environment difference should be reflected only on the destination machine.
Jun 26 in Big Data Hadoop by nitinrawat895
• 10,150 points
15 views

1 answer to this question.

0 votes

Hadoop Docker Hub provides Hadoop Images, You can try them out or if you have something suitable for different environments, then you might require to run on a fully distributed environment.i.e. with a container for the HDFS and YARN master node and multiple containers for the worker nodes. 

I can help you with an Image that I have. You can see from the Dockerfile that it starts from the Java base image, installs Hadoop and uses a startup script so containers can be run as a master or a worker.

That means you can run a distributed cluster with Docker, using the latest networking stack:

docker network create hadoop

docker run -d -p 50070:50070 -p 8088:8088 \
    --network hadoop --name hadoop-dotnet-master \
    sixeyed/hadoop-dotnet master

docker run -d -p 50075:50075 -p 8142:8042 -p 19888:19888 \
    --network hadoop 
    sixeyed/hadoop-dotnet worker

You can run a fully-distributed cluster with a Docker Compose file.

I hope this should help. 

Happy Learning...;-)

answered Jun 26 by ravikiran
• 3,560 points

Related Questions In Big Data Hadoop

0 votes
0 answers

Can I run Hadoop with Docker for both DEV and PROD environments?

Is there an easiest way that works ...READ MORE

Jun 26 in Big Data Hadoop by nitinrawat895
• 10,150 points
11 views
0 votes
1 answer

How can I download only hdfs and not hadoop?

No, you cannot download HDFS alone because ...READ MORE

answered Mar 15, 2018 in Big Data Hadoop by nitinrawat895
• 10,150 points
68 views
0 votes
1 answer

How can I download hadoop documentation for a specific version?

You can go through this SVN link:- ...READ MORE

answered Mar 21, 2018 in Big Data Hadoop by Shubham
• 13,190 points
66 views
0 votes
1 answer

Hadoop Mapreduce word count Program

Firstly you need to understand the concept ...READ MORE

answered Mar 16, 2018 in Data Analytics by nitinrawat895
• 10,150 points
2,059 views
0 votes
1 answer

hadoop.mapred vs hadoop.mapreduce?

org.apache.hadoop.mapred is the Old API  org.apache.hadoop.mapreduce is the ...READ MORE

answered Mar 16, 2018 in Data Analytics by nitinrawat895
• 10,150 points
198 views
0 votes
10 answers

hadoop fs -put command?

copy command can be used to copy files ...READ MORE

answered Dec 7, 2018 in Big Data Hadoop by Sujay
10,546 views
0 votes
1 answer

Hadoop dfs -ls command?

In your case there is no difference ...READ MORE

answered Mar 16, 2018 in Big Data Hadoop by kurt_cobain
• 9,240 points
771 views
0 votes
1 answer

How do I run Hadoop with Docker for both DEV and PROD environments?

Your requirements show that you are using ...READ MORE

answered May 30 in Big Data Hadoop by ravikiran
• 3,560 points
27 views