Hadoop “Unable to load native-hadoop library for your platform” warning

+10 votes

I installed Hadoop on a server running centos. Whenever I run start-dfs.sh or stop-dfs.sh, I get the following error:

WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable

My Hadoop version is - Hadoop 2.2.0.

Can anyone tell me why I m getting this warning, I am not sure what to do.

I've also added these two environment variables in hadoop-env.sh:

export HADOOP_OPTS="$HADOOP_OPTS -Djava.library.path=/usr/local/hadoop/lib/"

export HADOOP_COMMON_LIB_NATIVE_DIR="/usr/local/hadoop/lib/native/"

Any suggestions?

Mar 21, 2018 in Big Data Hadoop by coldcode
• 2,020 points
10,692 views

11 answers to this question.

+1 vote
Best answer

modify the glibc version.CentOS provides safe softwares tranditionally,it also means the version is old such as glibc,protobuf ...

ldd --version
ldd /opt/hadoop/lib/native/libhadoop.so.1.0.0

You can compare the version of current glibc with needed glibc.

Secondly: If the version of current glibc is old,you can update the glibc. DownLoad Glibc

If the version of current glibc id right,you can append word native to your HADOOP_OPTS

export HADOOP_COMMON_LIB_NATIVE_DIR=$HADOOP_HOME/lib/native
export HADOOP_OPTS="-Djava.library.path=$HADOOP_HOME/lib"
answered Sep 10, 2018 by bug_seeker
• 15,350 points

selected Jun 19 by Omkar
+3 votes

There may be two possibilities:

  1. You have installed the wrong Java JDK8 package. Please ensure to download the 64-bit JDK8 and remove your current 32-bit JDK8.
  2. It's a warning due to the Hadoop libraries being compiled for 32bits. You are probably running on a 64bit OS.

It's safe to ignore this warning.

answered Mar 22, 2018 by nitinrawat895
• 10,670 points
+3 votes

I had the same issue. It's solved by adding following lines in .bashrc:

export HADOOP_COMMON_LIB_NATIVE_DIR=$HADOOP_HOME/lib/native
export HADOOP_OPTS="-Djava.library.path=$HADOOP_HOME/lib"
answered Sep 10, 2018 by Neeraj
its not working
Hi! Did you try the solutions mentioned in other answers?
It is not working
Could you mention the steps you followed to install Hadoop?
+2 votes

In my case , after I build hadoop on my 64 bit Linux mint OS, I replaced the native library in hadoop/lib. Still the problem persist. Then I figured out the hadoop pointing to hadoop/lib not to the hadoop/lib/native. So I just moved all content from native library to its parent. And the warning just gone.

answered Sep 10, 2018 by Ram
+2 votes

This also would work:

export LD_LIBRARY_PATH=/usr/lib/hadoop/lib/native
answered Sep 10, 2018 by Gosh
+2 votes
Hello. I am using OSX and face the same problem. Any idea how to solve it?
answered Sep 10, 2018 by Sundar
Follow these steps replacing the path and Hadoop version where appropriate

wget http://www.eu.apache.org/dist/hadoop/common/hadoop-2.7.1/hadoop-2.7.1-src.tar.gz
tar xvf hadoop-2.7.1-src.tar.gz
cd hadoop-2.7.1-src
mvn package -Pdist,native -DskipTests -Dtar
mv lib /usr/local/Cellar/hadoop/2.7.1/

then update hadoop-env.sh with

export HADOOP_OPTS="$HADOOP_OPTS -Djava.net.preferIPv4Stack=true -Djava.security.krb5.realm= -Djava.security.krb5.kdc= -Djava.library.path=/usr/local/Cellar/hadoop/2.7.1/lib/native"
0 votes

Make sure you have set proper paths:

# JAVA env
#
export JAVA_HOME=/user/sys/jdk
export JRE_HOME=/user/sys/jdk/jre

export PATH=${JAVA_HOME}/bin:${JRE_HOME}/bin:${PATH}:.

# HADOOP env
#
export HADOOP_HOME=/usr/local/hadoop
export HADOOP_MAPRED_HOME=$HADOOP_HOME
export HADOOP_COMMON_HOME=$HADOOP_HOME
export HADOOP_HDFS_HOME=$HADOOP_HOME
export YARN_HOME=$HADOOP_HOME

export HADOOP_CONF_DIR=$HADOOP_HOME/etc/hadoop
export PATH=$PATH:$HADOOP_HOME/sbin:$HADOOP_HOME/bin
answered Dec 7, 2018 by Gopal
0 votes
export HADOOP_COMMON_LIB_NATIVE_DIR=$HADOOP_HOME/lib/native
export HADOOP_OPTS="-Djava.library.path=$HADOOP_HOME/lib"
answered Dec 7, 2018 by Sarthik
0 votes
Update your jdk and try again
answered Dec 7, 2018 by Parth
0 votes

Add this path:

export JAVA_LIBRARY_PATH=$HADOOP_HOME/lib/native:$JAVA_LIBRARY_PATH
source ~./bashrc
answered Dec 7, 2018 by Veeresh
+1 vote
answered Jun 18 by anonymous

reshown Jun 19 by Vardhan

Related Questions In Big Data Hadoop

0 votes
1 answer

Hadoop Warning: Unable to load Native-Hadoop Library for your platform.

Are you sure you're running Hadoop on 32bit ...READ MORE

answered May 28 in Big Data Hadoop by ravikiran
• 4,560 points
91 views
0 votes
1 answer

Warning: Unable to load Hadoop Native Library for your platform.

 I had the same issue. It's solved ...READ MORE

answered Jun 28 in Big Data Hadoop by ravikiran
• 4,560 points
60 views
0 votes
1 answer

Hadoop: Can not start namenode: Unable to load native-hadoop library for your platform

The entries in your .bashrc file looks ...READ MORE

answered Dec 5, 2018 in Big Data Hadoop by Omkar
• 67,380 points
164 views
0 votes
1 answer

Getting error when writing to HDFS. Unable to load native-hadoop library for your platform

Try this: sudo service hadoop-master restart After that try ...READ MORE

answered Dec 19, 2018 in Big Data Hadoop by Omkar
• 67,380 points
135 views
0 votes
1 answer

Error: Unable to load native-hadoop library for your platform.

Hi,  You should execute the command hdfs dfs -ls ...READ MORE

answered Jun 17 in Big Data Hadoop by Gitika
• 25,340 points
49 views
0 votes
1 answer

Hadoop dfs -ls command?

In your case there is no difference ...READ MORE

answered Mar 16, 2018 in Big Data Hadoop by kurt_cobain
• 9,240 points
972 views
0 votes
1 answer

Hadoop Mapreduce word count Program

Firstly you need to understand the concept ...READ MORE

answered Mar 16, 2018 in Data Analytics by nitinrawat895
• 10,670 points
2,651 views
0 votes
10 answers

hadoop fs -put command?

put syntax: put <localSrc> <dest> copy syntax: copyFr ...READ MORE

answered Dec 7, 2018 in Big Data Hadoop by Aditya
13,209 views
0 votes
1 answer