The Java code for creating an AWS EMR Cluster is as follows:-
import java.io.IOException;
import com.amazonaws.auth.AWSCredentials;
import com.amazonaws.auth.BasicAWSCredentials;
import com.amazonaws.auth.PropertiesCredentials;
import com.amazonaws.services.elasticmapreduce.*;
import com.amazonaws.services.elasticmapreduce.model.AddJobFlowStepsRequest;
import com.amazonaws.services.elasticmapreduce.model.AddJobFlowStepsResult;
import com.amazonaws.services.elasticmapreduce.model.HadoopJarStepConfig;
import com.amazonaws.services.elasticmapreduce.model.StepConfig;
import com.amazonaws.services.elasticmapreduce.util.StepFactory;
public class main {
private static final AWSCredentials AWS_CREDENTIALS;
static {
// Your accesskey and secretkey
AWS_CREDENTIALS = new BasicAWSCredentials(
"Access Key",
"Secret Key"
);
}
public static void main(String[] args) {
AmazonElasticMapReduce client = new AmazonElasticMapReduceClient(AWS_CREDENTIALS);
StepConfig hive = new StepConfig("Hive", new StepFactory().newInstallHiveStep());
HadoopJarStepConfig hadoopConfig1 = new HadoopJarStepConfig()
.withJar("s3://mybucket/my-jar-location1");
StepConfig customStep = new StepConfig("Step1", hadoopConfig1);
AddJobFlowStepsResult result = client.addJobFlowSteps(new AddJobFlowStepsRequest()
.withJobFlowId("j-id")
.withSteps(hive, customStep));
System.out.println(result.getStepIds());
}
}
This will create a 2 node cluster, 1 master and 1 slave. Hope this helps.
