How to create a EMR Cluster using Java AWS SDK?

0 votes
Can someone help me with the java code to create a EMR cluster? I am using Java AWS SDK. Any help is appreciated.
Feb 26 in AWS by Reshmi Bhattacharya
53 views

1 answer to this question.

Your answer

Your name to display (optional):
Privacy: Your email address will only be used for sending these notifications.
0 votes

The Java code for creating an EMR Cluster is as follows:-

import java.io.IOException;
import com.amazonaws.auth.AWSCredentials;
import com.amazonaws.auth.BasicAWSCredentials;
import com.amazonaws.auth.PropertiesCredentials;
import com.amazonaws.services.elasticmapreduce.*;
import com.amazonaws.services.elasticmapreduce.model.AddJobFlowStepsRequest;
import com.amazonaws.services.elasticmapreduce.model.AddJobFlowStepsResult;
import com.amazonaws.services.elasticmapreduce.model.HadoopJarStepConfig;
import com.amazonaws.services.elasticmapreduce.model.StepConfig;
import com.amazonaws.services.elasticmapreduce.util.StepFactory;

public class main {
    private static final AWSCredentials AWS_CREDENTIALS;
    
    static {
        // Your accesskey and secretkey
        AWS_CREDENTIALS = new BasicAWSCredentials(
                "Access Key",
                "Secret Key"
        );
    }

    public static void main(String[] args) {
        
        
        AmazonElasticMapReduce client = new AmazonElasticMapReduceClient(AWS_CREDENTIALS);
        StepConfig hive = new StepConfig("Hive", new StepFactory().newInstallHiveStep());

        HadoopJarStepConfig hadoopConfig1 = new HadoopJarStepConfig()
            .withJar("s3://mybucket/my-jar-location1");
        StepConfig customStep = new StepConfig("Step1", hadoopConfig1);

        AddJobFlowStepsResult result = client.addJobFlowSteps(new AddJobFlowStepsRequest()
          .withJobFlowId("j-id")
          .withSteps(hive, customStep));
        
          System.out.println(result.getStepIds());

    }
}

This will create a 2 node cluster, 1 master and 1 slave. Hope this helps.

answered Feb 27 by Priyaj
• 56,140 points

Related Questions In AWS

0 votes
1 answer
0 votes
1 answer
0 votes
1 answer

How to create EMR cluster using AWS CLI?

The command to create EMR cluster using ...READ MORE

answered Feb 27 in AWS by Priyaj
• 56,140 points
54 views
0 votes
1 answer
+1 vote
2 answers
+1 vote
1 answer
0 votes
1 answer

How to create a VPC using Java SDK?

You can create a VPC using the ...READ MORE

answered Feb 21 in AWS by Priyaj
• 56,140 points
72 views

© 2018 Brain4ce Education Solutions Pvt. Ltd. All rights Reserved.
"PMP®","PMI®", "PMI-ACP®" and "PMBOK®" are registered marks of the Project Management Institute, Inc. MongoDB®, Mongo and the leaf logo are the registered trademarks of MongoDB, Inc.