How I can kill the jobs using jobID running in local mode with Hadoop

0 votes
I am Running hadoop jobs in local mode i.e jobs are running in single JVM of linux machine. When I try to kill the running jobs using JobClient then the RunningJob object is returning null. Below is the code which I am using:

**String jobDetails="job_local65122045_0001";

JobID jobID = JobID.forName(jobDetails);

JobClient jobClient = new JobClient(conf);

RunningJob runningJob = jobClient.getJob(jobId);**

I am getting RunningJob object as null though the job is running.However, I am successfully able to get the RunningJob object incase of hadoop Jobs.

Could anyone please help to fix the problem.
Aug 26, 2020 in Big Data Hadoop by kamboj
• 140 points
1,143 views

Hi, @kamboj,

You can use this to kill a Hadoop job:

hadoop job -kill $jobId

 

Hi @Reshma,

I do not want to kill the job through command. My requirement is to kill job though Hadoop API's. I also used LocalJobRunner but  did not get success.

Moreover, just to check the output, I run the command mentioned by you and below is the exception I have faced.

--------------------------------------------------------------------------------------------

[linux]# hadoop job -kill job_local5123645_0008

DEPRECATED: Use of this script to execute mapred command is deprecated.

Instead use the mapred command for it.

20/08/26 06:03:24 INFO client.MapRZKBasedRMFailoverProxyProvider: Updated RM address to xen-mapr-centos7/IP8032

Exception in thread "main" java.lang.NumberFormatException: For input string: "local5123645"

at java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)

at java.lang.Long.parseLong(Long.java:589)

at java.lang.Long.parseLong(Long.java:631)

at org.apache.hadoop.mapreduce.TypeConverter.toClusterTimeStamp(TypeConverter.java:92)

at org.apache.hadoop.mapreduce.TypeConverter.toYarn(TypeConverter.java:81)

at org.apache.hadoop.mapred.ClientServiceDelegate.<init>(ClientServiceDelegate.java:120)

at org.apache.hadoop.mapred.ClientCache.getClient(ClientCache.java:68)

at org.apache.hadoop.mapred.YARNRunner.getJobStatus(YARNRunner.java:612)

at org.apache.hadoop.mapreduce.Cluster.getJob(Cluster.java:186)

at org.apache.hadoop.mapreduce.tools.CLI.getJob(CLI.java:625)

at org.apache.hadoop.mapreduce.tools.CLI.run(CLI.java:355)

at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)

at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)

at org.apache.hadoop.mapred.JobClient.main(JobClient.java:127

 -----------------------------------------------------

Could you please let me know what is wrong with this.

Hi, @Kanmboj,

I guess, Hadoop old version that supports mapred has this (hadoop job ) command.

The new version of Hadoop does not support this functionality because of YARN. You can go for this:

mapred job -kill $jobId
@Reshma,

I will try this as well. But my main concern here is to kill the job which is running locally(local mode). Could you please let me know how to kill the running jobs through hadoop API's as command line will not be useful for me.

Hi Kamboj,

I don't think you can do that directly. You can do this thing in another way. Like you can create your own API. Now you can run your commands from the shell. So you can create an API with the help of python CGI. Check if it helps you.

No answer to this question. Be the first to respond.

Your answer

Your name to display (optional):
Privacy: Your email address will only be used for sending these notifications.

Related Questions In Big Data Hadoop

0 votes
1 answer

How can I get the respective Bitcoin value for an input in USD when using c#

Simply make call to server and parse ...READ MORE

answered Mar 25, 2018 in Big Data Hadoop by charlie_brown
• 7,710 points
393 views
0 votes
2 answers

How can I list NameNode & DataNodes from any machine in the Hadoop cluster?

You can browse hadoop page from any ...READ MORE

answered Jan 23, 2020 in Big Data Hadoop by MD
• 95,360 points
9,379 views
0 votes
1 answer

How can I use my host machine’s web browser to check my HDFS services running in the VM?

The sole purpose of the virtual machine ...READ MORE

answered Apr 18, 2018 in Big Data Hadoop by Shubham
• 13,490 points
655 views
0 votes
1 answer

In Hadoop MapReduce, how can i set an Object as the Value for Map output?

Try this and see if it works: public ...READ MORE

answered Nov 21, 2018 in Big Data Hadoop by Omkar
• 69,190 points
497 views
0 votes
1 answer

How do I include all the Hadoop dependencies using Maven?

This is a dependency mismatch error. I ...READ MORE

answered Apr 10, 2018 in Big Data Hadoop by Shubham
• 13,490 points
4,707 views
0 votes
1 answer

How can I write text in HDFS using CMD?

Hadoop put & appendToFile only reads standard ...READ MORE

answered Apr 27, 2018 in Big Data Hadoop by Shubham
• 13,490 points
913 views
+1 vote
1 answer

Hadoop Mapreduce word count Program

Firstly you need to understand the concept ...READ MORE

answered Mar 16, 2018 in Data Analytics by nitinrawat895
• 11,380 points
9,094 views
0 votes
1 answer

hadoop.mapred vs hadoop.mapreduce?

org.apache.hadoop.mapred is the Old API  org.apache.hadoop.mapreduce is the ...READ MORE

answered Mar 16, 2018 in Data Analytics by nitinrawat895
• 11,380 points
1,640 views
+2 votes
11 answers

hadoop fs -put command?

Hi, You can create one directory in HDFS ...READ MORE

answered Mar 16, 2018 in Big Data Hadoop by nitinrawat895
• 11,380 points
82,970 views
–1 vote
1 answer

Hadoop dfs -ls command?

In your case there is no difference ...READ MORE

answered Mar 16, 2018 in Big Data Hadoop by kurt_cobain
• 9,390 points
3,479 views
webinar REGISTER FOR FREE WEBINAR X
Send OTP
REGISTER NOW
webinar_success Thank you for registering Join Edureka Meetup community for 100+ Free Webinars each month JOIN MEETUP GROUP