Apache Spark gives Failed to load native-hadoop with error

Question

I am trying to start Apache-Spark and when I do that, I am getting this error:

15/05/21 04:28:22 DEBUG NativeCodeLoader: Failed to load native-hadoop with error: java.lang.UnsatisfiedLinkError:no hadoop in java.library.path
15/05/21 04:28:22 DEBUG NativeCodeLoader: java.library.path=/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib

How can I solve this?

Answer 1

Seems like hadoop path is missing in java.library.path. Add the Hadoop library into LD_LIBRARY_PATH:

export LD_LIBRARY_PATH="$HADOOP_HOME/lib/native/:$LD_LIBRARY_PATH"