Hadoop in the AWS free tier?

0 votes

For a Java class final project, we need to setup Hadoop and implement an n-gram processor. I have found a number of 'Hadoop on AWS' tutorials, but am uncertain how to deploy Hadoop while staying in the free tier. I tried a while ago, and received a bill for over $250 USD. Now I am too nervous to test, and would like help to ensure I do not rack up another bill.

From what I understand, these are the limits of the free tier: Allowable Instances: t1.micro or t2.micro Maximum instance-hours per month: 750 OS: Standard-priced versions of Linux or Windows on EC2 Storage: 30 GB of EBS Many of the Hadoop tutorials use instances other than those two shown above. The AWS tutorial here mentions it will cost about $1, stating it will run for one hour. I need to keep this/these instances active for over 2 weeks, but only really use them for a few minutes at a time. I do not believe that I will exceed even one-tenth of that 750 hours.

We can get bonus points if we use "more than 1 machine". Can I do that within the free tier? Does the free tier have a limit on the number of instances it can spin up?

Does anyone have a tutorial that stays in the free tier? Or should I skip AWS and try a local Hadoop solution?

Aug 16, 2018 in AWS by bug_seeker
• 15,350 points
165 views

1 answer to this question.

0 votes

If you want to limit your hadoop cluster nodes only to t2.micro instances and total EBS volumes size to 30 GB, then you can run [in theory] a hadoop cluster within free tier. Do note that the hardware on t2.micro are of meagre.

The thing about free tier on AWS is that you are allowed only t2.micro for 750 hours per month. That means you can run for example 10 nodes for 75 hrs in a month for free after which you would be billed.

Here is a post that does exactly what you want with 4 nodes "Spinning Up a Free Hadoop Cluster: Step by Step". So you should be able to run this cluster within free limit for around 1 week.

answered Aug 16, 2018 by Priyaj
• 56,920 points

Related Questions In AWS

0 votes
1 answer
0 votes
2 answers

How many instances can be created under AWS free tier account

Ans 1. Amazon charges based on usage per ...READ MORE

answered Jun 7, 2018 in AWS by MISHA
• 520 points
154 views
0 votes
1 answer

AWS EC2 Free Tier- how many instances can i run?

Amazon has a handy page that details ...READ MORE

answered Jul 6, 2018 in AWS by Priyaj
• 56,920 points
472 views
+13 votes
2 answers

Git management technique when there are multiple customers and need multiple customization?

Consider this - In 'extended' Git-Flow, (Git-Multi-Flow, ...READ MORE

answered Mar 26, 2018 in DevOps & Agile by DragonLord999
• 8,380 points
214 views
0 votes
1 answer

How to checkout Hadoop 2.6.0 from git

Clone the following Git repository: git clone git ...READ MORE

answered Apr 23, 2018 in Big Data Hadoop by kurt_cobain
• 9,260 points
93 views
0 votes
1 answer

AWS email saying have exceeded the free-tier limit.

I got a similar message, except I ...READ MORE

answered Jul 9, 2018 in AWS by Priyaj
• 56,920 points
37 views
+1 vote
1 answer

How many times can one receive the AWS free tier?

Yes, it is possible to create a ...READ MORE

answered Jul 27, 2018 in AWS by Priyaj
• 56,920 points
38 views