How to pause and resume hive job?

0 votes
Jul 17, 2018 in Big Data Hadoop by Frankie
• 9,710 points
40 views

1 answer to this question.

Your answer

Your name to display (optional):
Privacy: Your email address will only be used for sending these notifications.
0 votes

Practically speaking, it's difficult/impossible to pause and resume the job, because the status of the hive will be either Successful or Failed.

There is definitely not a 'debug shortcut' like in some languages that would allow you to pause processing in the middle of a step.

But we can do the following steps to get close to the solution:

1. Split  the job: Rather than writing one hive script, write 2 scripts in a way that first script would operate on the parts of the data and allows pausing the job and the 2nd script would operate on resuming the hive job.

This is a limited approach.

2. Freeze the environment : In this case, you can freeze the entire cluster and in case if you have a single node cluster, you can use Virtual Machine and freeze it.

answered Jul 17, 2018 by Neha
• 6,180 points

Related Questions In Big Data Hadoop

0 votes
1 answer

How to create a parquet table in hive and store data in it from a hive table?

Please use the code attached below for ...READ MORE

answered Jan 28 in Big Data Hadoop by Omkar
• 67,000 points
437 views
0 votes
1 answer

How to create a Hive Database and store it in HDFS?

You will have to exclusively mention the ...READ MORE

answered May 20 in Big Data Hadoop by Jishnu
20 views
0 votes
1 answer

How Impala is fast compared to Hive in terms of query response?

Impala provides faster response as it uses MPP(massively ...READ MORE

answered Mar 21, 2018 in Big Data Hadoop by nitinrawat895
• 9,610 points
209 views
0 votes
1 answer

How to install and configure a multi-node Hadoop cluster?

I would recommend you to install Cent ...READ MORE

answered Mar 21, 2018 in Big Data Hadoop by Shubham
• 12,890 points
510 views
0 votes
1 answer

How to create a FileSystem object that can be used for reading from and writing to HDFS?

Read operation on HDFS In order to read ...READ MORE

answered Mar 21, 2018 in Big Data Hadoop by nitinrawat895
• 9,610 points

edited Mar 21, 2018 by nitinrawat895 179 views
0 votes
1 answer

How to learn Big Data and Ecosystem ?

First understand Big Data and challenges associated ...READ MORE

answered Mar 27, 2018 in Big Data Hadoop by kurt_cobain
• 9,240 points
26 views
0 votes
1 answer

How hadoop mapreduce job is submitted to worker nodes?

Alright, I think you are basically looking ...READ MORE

answered Mar 29, 2018 in Big Data Hadoop by Ashish
• 2,630 points
1,338 views
0 votes
1 answer

How to convert a string to timestamp with milliseconds in Hive?

 concatenation of substrings using the following code: select ...READ MORE

answered Oct 31, 2018 in Big Data Hadoop by Neha
• 6,180 points
993 views
+2 votes
1 answer

How to calculate Maximum salary of the employee with the name using the Map Reduce Technique

Please try the below code and it ...READ MORE

answered Jul 25, 2018 in Big Data Hadoop by Neha
• 6,180 points
267 views

© 2018 Brain4ce Education Solutions Pvt. Ltd. All rights Reserved.
"PMP®","PMI®", "PMI-ACP®" and "PMBOK®" are registered marks of the Project Management Institute, Inc. MongoDB®, Mongo and the leaf logo are the registered trademarks of MongoDB, Inc.