How to retrieve the list of sql Hive QL commands that has been executed in a hadoop cluster

0 votes
We have a shared 10 node hadoop cluster where we have lets say, group of ten people running different hive queries? Now, what I want is to monitor all these hive queries which is being executed through hive-odbc driver. I have checked job history server where I found only part of such queries not all of them. Basically, I was looking for a  way to monitor these queries and identified the un-optimized ones out of them. Once identified, I can notify the same to the developer for further improvement.
Apr 4, 2018 in Big Data Hadoop by nitinrawat895
• 11,380 points
13,602 views

1 answer to this question.

0 votes
Well, what you can do is use ssh to loging the node running hiveserver2 daemon. From there you can check the logs by executing following command:

cat $HIVE_HOME/hive-0.13/logs/<user_id>/hive.log

Further you can pipe a grep to the above command as:

cat $HIVE_HOME/hive-2.1/logs/<user_id>/hive.log  | grep "Executing command"

Hope it helps!
answered Apr 4, 2018 by kurt_cobain
• 9,390 points

Related Questions In Big Data Hadoop

+1 vote
1 answer

How to limit the number of rows per each item in a Hive QL?

SELECT a_id, b, c, count(*) as sumrequests FROM ...READ MORE

answered Dec 1, 2018 in Big Data Hadoop by Omkar
• 69,210 points
25,787 views
0 votes
1 answer

Hadoop Hive: How to skip the first line of csv while loading in hive table?

You can try this: CREATE TABLE temp ...READ MORE

answered Nov 8, 2018 in Big Data Hadoop by Omkar
• 69,210 points
8,395 views
0 votes
1 answer

How to check the size of a file in Hadoop HDFS?

You can use the  hadoop fs -ls command to ...READ MORE

answered Nov 21, 2018 in Big Data Hadoop by Omkar
• 69,210 points
12,591 views
0 votes
2 answers

How to change the location of a table in hive?

Changing location requires 2 steps: 1.) Change location ...READ MORE

answered Feb 12, 2020 in Big Data Hadoop by Saksham Sehrawet
8,139 views
0 votes
11 answers
0 votes
1 answer

What is the command to find the free space in HDFS?

You can use dfsadmin which runs a ...READ MORE

answered Apr 29, 2018 in Big Data Hadoop by Shubham
• 13,490 points
1,847 views
0 votes
1 answer

How to find the used cache in HDFS

hdfs dfsadmin -report This command tells fs ...READ MORE

answered May 4, 2018 in Big Data Hadoop by Shubham
• 13,490 points
2,025 views
+1 vote
1 answer

Hadoop Mapreduce word count Program

Firstly you need to understand the concept ...READ MORE

answered Mar 16, 2018 in Data Analytics by nitinrawat895
• 11,380 points
10,522 views
0 votes
1 answer

How to configure secondary namenode in Hadoop 2.x ?

bin/hadoop-daemon.sh start [namenode | secondarynamenode | datanode ...READ MORE

answered Apr 6, 2018 in Big Data Hadoop by kurt_cobain
• 9,390 points
1,524 views
webinar REGISTER FOR FREE WEBINAR X
REGISTER NOW
webinar_success Thank you for registering Join Edureka Meetup community for 100+ Free Webinars each month JOIN MEETUP GROUP