Hadoop 3 compatibility with older versions of Hive Pig Sqoop and Spark

Question

My Company is using an 8 node Hadoop Cluster, we are currently using Hadoop 2.8.

We are thinking of shifting to Hadoop 3.0. Just wanted to know if there will be any issues with older versions of Spark, Pig, Sqoop or Hive

Answer 1

Hadoop 3 is not widely used in the industry, almost everyone is using 2.7 or 2.8.

But i think Spark will be supported by Hadoop 3 perfectly, I've used it once there were some deprecated commands, but otherwise, it was working fine,

Don't know about Hive.

It's better to wait if it's not an emergency.

Answer 2

Which Spark, Pig, Sqoop, Hive versions are you currently using in Hadoop 2.8 ?

Comments

Hi Dhiren,
I use the following configurations :
Spark 2.2
Scala 2.12
Kafka 0.11.0.0 (compatible with Scala 2.12)
Sqoop 1.4.6
Hive 1.2.2
Pig 0.12.0

Hope this helps