Difference between hive exec compress output true and mapreduce output fileoutputformat compress true

Question

What is the difference between these two properties?
hive.exec.compress.output=true;
mapreduce.output.fileoutputformat.compress=true;

and also, where can I find the explanation for all the hive properties?

Answer 1

Hey there!

The definition of these two properties are as follows:

hive.exec.compress.output - Set this property to true if you want the final output that will be stored on HDFS to be compressed.

mapreduce.output.fileoutputformat.compress- Set this property to true if you want to compress the MapReduce job output. The default value is false.

So, the difference between these two is that the first one will compress any output that will be stored on HDFS and the second one will compress only the MapReduce Job output.

Click on the below link for all hive properties. There will not be a complete explanation about each and every property in the link I have provided, but you can search them individually.

https://streever.atlassian.net/wiki/spaces/HADOOP/pages/1933314/Hive+Performance+Tips