How to retrieve the list of sql (Hive QL) commands that has been executed in a hadoop cluster?

0 votes
We have a shared 10 node hadoop cluster where we have lets say, group of ten people running different hive queries? Now, what I want is to monitor all these hive queries which is being executed through hive-odbc driver. I have checked job history server where I found only part of such queries not all of them. Basically, I was looking for a  way to monitor these queries and identified the un-optimized ones out of them. Once identified, I can notify the same to the developer for further improvement.
Apr 4, 2018 in Big Data Hadoop by nitinrawat895
• 10,510 points
2,798 views

1 answer to this question.

0 votes
Well, what you can do is use ssh to loging the node running hiveserver2 daemon. From there you can check the logs by executing following command:

cat $HIVE_HOME/hive-0.13/logs/<user_id>/hive.log

Further you can pipe a grep to the above command as:

cat $HIVE_HOME/hive-2.1/logs/<user_id>/hive.log  | grep "Executing command"

Hope it helps!
answered Apr 4, 2018 by kurt_cobain
• 9,240 points

Related Questions In Big Data Hadoop

0 votes
1 answer

How to limit the number of rows per each item in a Hive QL?

SELECT a_id, b, c, count(*) as sumrequests FROM ...READ MORE

answered Nov 30, 2018 in Big Data Hadoop by Omkar
• 67,290 points
622 views
0 votes
1 answer

Hadoop Hive: How to skip the first line of csv while loading in hive table?

You can try this: CREATE TABLE temp ...READ MORE

answered Nov 8, 2018 in Big Data Hadoop by Omkar
• 67,290 points
713 views
0 votes
1 answer

How to check the size of a file in Hadoop HDFS?

You can use the  hadoop fs -ls command to ...READ MORE

answered Nov 21, 2018 in Big Data Hadoop by Omkar
• 67,290 points
674 views
0 votes
1 answer

How to change the location of a table in hive?

Hey, Basically When we create a table in hive, ...READ MORE

answered May 14 in Big Data Hadoop by Gitika
• 25,300 points
112 views
0 votes
11 answers
0 votes
1 answer

What is the command to find the free space in HDFS?

You can use dfsadmin which runs a ...READ MORE

answered Apr 29, 2018 in Big Data Hadoop by Shubham
• 13,290 points
150 views
0 votes
1 answer

How to find the used cache in HDFS

hdfs dfsadmin -report This command tells fs ...READ MORE

answered May 4, 2018 in Big Data Hadoop by Shubham
• 13,290 points
189 views
0 votes
1 answer

Hadoop Mapreduce word count Program

Firstly you need to understand the concept ...READ MORE

answered Mar 16, 2018 in Data Analytics by nitinrawat895
• 10,510 points
2,408 views
0 votes
1 answer

How to configure secondary namenode in Hadoop 2.x ?

bin/hadoop-daemon.sh start [namenode | secondarynamenode | datanode ...READ MORE

answered Apr 6, 2018 in Big Data Hadoop by kurt_cobain
• 9,240 points
347 views