How to retrieve the list of sql Hive QL commands that has been executed in a hadoop cluster

Question

We have a shared 10 node hadoop cluster where we have lets say, group of ten people running different hive queries? Now, what I want is to monitor all these hive queries which is being executed through hive-odbc driver. I have checked job history server where I found only part of such queries not all of them. Basically, I was looking for a  way to monitor these queries and identified the un-optimized ones out of them. Once identified, I can notify the same to the developer for further improvement.

Answer 1

Well, what you can do is use ssh to loging the node running hiveserver2 daemon. From there you can check the logs by executing following command:

cat $HIVE_HOME/hive-0.13/logs/<user_id>/hive.log

Further you can pipe a grep to the above command as:

cat $HIVE_HOME/hive-2.1/logs/<user_id>/hive.log  | grep "Executing command"

Hope it helps!