How I can kill the jobs using jobID running in local mode with Hadoop

Question

I am Running hadoop jobs in local mode i.e jobs are running in single JVM of linux machine. When I try to kill the running jobs using JobClient then the RunningJob object is returning null. Below is the code which I am using:

**String jobDetails="job_local65122045_0001";

JobID jobID = JobID.forName(jobDetails);

JobClient jobClient = new JobClient(conf);

RunningJob runningJob = jobClient.getJob(jobId);**

I am getting RunningJob object as null though the job is running.However, I am successfully able to get the RunningJob object incase of hadoop Jobs.

Could anyone please help to fix the problem.

Comments

Hi, @kamboj,

You can use this to kill a Hadoop job:

hadoop job -kill $jobId

 

---

Hi @Reshma,

I do not want to kill the job through command. My requirement is to kill job though Hadoop API's. I also used LocalJobRunner but  did not get success.

Moreover, just to check the output, I run the command mentioned by you and below is the exception I have faced.

--------------------------------------------------------------------------------------------

[linux]# hadoop job -kill job_local5123645_0008

DEPRECATED: Use of this script to execute mapred command is deprecated.

Instead use the mapred command for it.

20/08/26 06:03:24 INFO client.MapRZKBasedRMFailoverProxyProvider: Updated RM address to xen-mapr-centos7/IP8032

Exception in thread "main" java.lang.NumberFormatException: For input string: "local5123645"

at java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)

at java.lang.Long.parseLong(Long.java:589)

at java.lang.Long.parseLong(Long.java:631)

at org.apache.hadoop.mapreduce.TypeConverter.toClusterTimeStamp(TypeConverter.java:92)

at org.apache.hadoop.mapreduce.TypeConverter.toYarn(TypeConverter.java:81)

at org.apache.hadoop.mapred.ClientServiceDelegate.<init>(ClientServiceDelegate.java:120)

at org.apache.hadoop.mapred.ClientCache.getClient(ClientCache.java:68)

at org.apache.hadoop.mapred.YARNRunner.getJobStatus(YARNRunner.java:612)

at org.apache.hadoop.mapreduce.Cluster.getJob(Cluster.java:186)

at org.apache.hadoop.mapreduce.tools.CLI.getJob(CLI.java:625)

at org.apache.hadoop.mapreduce.tools.CLI.run(CLI.java:355)

at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)

at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)

at org.apache.hadoop.mapred.JobClient.main(JobClient.java:127

 -----------------------------------------------------

Could you please let me know what is wrong with this.

---

Hi, @Kanmboj,

I guess, Hadoop old version that supports mapred has this (hadoop job ) command.

The new version of Hadoop does not support this functionality because of YARN. You can go for this:

mapred job -kill $jobId

---

@Reshma,

I will try this as well. But my main concern here is to kill the job which is running locally(local mode). Could you please let me know how to kill the running jobs through hadoop API's as command line will not be useful for me.

---

Hi Kamboj,

I don't think you can do that directly. You can do this thing in another way. Like you can create your own API. Now you can run your commands from the shell. So you can create an API with the help of python CGI. Check if it helps you.