How to pause and resume hive job

0 votes
Jul 17, 2018 in Big Data Hadoop by Frankie
• 9,830 points
1,124 views

1 answer to this question.

0 votes

Practically speaking, it's difficult/impossible to pause and resume the job, because the status of the hive will be either Successful or Failed.

There is definitely not a 'debug shortcut' like in some languages that would allow you to pause processing in the middle of a step.

But we can do the following steps to get close to the solution:

1. Split  the job: Rather than writing one hive script, write 2 scripts in a way that first script would operate on the parts of the data and allows pausing the job and the 2nd script would operate on resuming the hive job.

This is a limited approach.

2. Freeze the environment : In this case, you can freeze the entire cluster and in case if you have a single node cluster, you can use Virtual Machine and freeze it.

answered Jul 17, 2018 by Neha
• 6,300 points

Related Questions In Big Data Hadoop

0 votes
1 answer

How to create a parquet table in hive and store data in it from a hive table?

Please use the code attached below for ...READ MORE

answered Jan 28, 2019 in Big Data Hadoop by Omkar
• 69,210 points
18,025 views
0 votes
1 answer

How to create a Hive Database and store it in HDFS?

You will have to exclusively mention the ...READ MORE

answered May 20, 2019 in Big Data Hadoop by Jishnu
2,131 views
0 votes
1 answer
0 votes
1 answer

How to retrieve/get complete HQL hive query from hive,spark and tez?

To get full query running for the ...READ MORE

answered Jul 11, 2019 in Big Data Hadoop by Yogi
3,713 views
0 votes
1 answer

How to get resource utilized(memory and CPU) by each hive query which is running under single application Id?

Hi@sivachandran, You can monitor the metrics of your ...READ MORE

answered Sep 18, 2020 in Big Data Hadoop by MD
• 95,440 points
2,295 views
0 votes
2 answers

How to know Hive and Hadoop versions from command prompt?

Hi, Hadoop and hive have their individual commands. ...READ MORE

answered Dec 18, 2020 in Big Data Hadoop by akhtar
• 38,230 points
8,249 views
+1 vote
2 answers

How to delete and update a record in Hive?

Hi, Earlier, there was no operation supported for ...READ MORE

answered Dec 18, 2020 in Big Data Hadoop by akhtar
• 38,230 points
5,935 views
0 votes
1 answer

How Impala is fast compared to Hive in terms of query response?

Impala provides faster response as it uses MPP(massively ...READ MORE

answered Mar 21, 2018 in Big Data Hadoop by nitinrawat895
• 11,380 points
1,852 views
0 votes
1 answer

How to convert a string to timestamp with milliseconds in Hive?

 concatenation of substrings using the following code: select ...READ MORE

answered Oct 31, 2018 in Big Data Hadoop by Neha
• 6,300 points
18,515 views
+2 votes
1 answer

How to calculate Maximum salary of the employee with the name using the Map Reduce Technique

Please try the below code and it ...READ MORE

answered Jul 25, 2018 in Big Data Hadoop by Neha
• 6,300 points
5,209 views
webinar REGISTER FOR FREE WEBINAR X
REGISTER NOW
webinar_success Thank you for registering Join Edureka Meetup community for 100+ Free Webinars each month JOIN MEETUP GROUP