Not able to start hadoop dfs

Question

I am trying to install Hadoop 2.2.0 Cluster on the servers. All the servers are 64-bit. When I am running ./start-dfs.sh, I got the following error:

13/11/15 14:29:26 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
Starting namenodes on [Java HotSpot(TM) 64-Bit Server VM warning: You have loaded library /home/hchen/hadoop-2.2.0/lib/native/libhadoop.so.1.0.0 which might have disabled stack guard. The VM will try to fix the stack guard now.It's highly recommended that you fix the library with 'execstack -c <libfile>', or link it with '-z noexecstack'.namenode]
sed: -e expression #1, char 6: unknown option to `s' have: ssh: Could not resolve hostname have: Name or service not known
HotSpot(TM): ssh: Could not resolve hostname HotSpot(TM): Name or service not known
-c: Unknown cipher type 'cd'
Java: ssh: Could not resolve hostname Java: Name or service not known
The authenticity of host 'namenode (192.168.1.62)' can't be established.
RSA key fingerprint is 65:f9:aa:7c:8f:fc:74:e4:c7:a2:f5:7f:d2:cd:55:d4.
Are you sure you want to continue connecting (yes/no)? VM: ssh: Could not resolve        hostname VM: Name or service not known
You: ssh: Could not resolve hostname You: Name or service not known
warning:: ssh: Could not resolve hostname warning:: Name or service not known
library: ssh: Could not resolve hostname library: Name or service not known
have: ssh: Could not resolve hostname have: Name or service not known
64-Bit: ssh: Could not resolve hostname 64-Bit: Name or service not known
...

Comments

I am installing Hadoop. I downloaded jdk and hadoop and set path in bashrc file. I execute source .bashrc to load vairables and i get error

bash: export: `=': not a valid identifier
bash: export: `/home/user/jdk1.8.0_181/bin:/home/user/jdk1.8.0_181/bin:/home/user/jdk1.8.0_181/bin:/home/user/jdk1.8.0_181/bin:/home/user/jdk1.8.0_181/bin:/home/user/jdk1.8.0_181/bin:/home/user/jdk1.8.0_181/bin:/usr/local/bin:/usr/local/sbin:/usr/bin:/usr/sbin:/bin:/sbin:/home/user/hadoop-2.8.5/bin:/usr/lib/sqoop/bin:/home/user/.local/bin:/home/user/bin:/home/user/hadoop-2.8.5/bin:/usr/lib/sqoop/bin:/home/user/hadoop-2.8.5/bin:/home/user/hadoop-2.8.5/bin:/home/user/hadoop-2.8.5/bin:/usr/lib/sqoop/bin:/home/user/hadoop-2.8.5/bin:/usr/lib/sqoop/bin:/home/user/hadoop-2.8.5/bin:/usr/lib/sqoop/bin:/home/user/hadoop-2.8.5/bin:/usr/lib/sqoop/bin:/home/user/hadoop-2.8.5/pig/bin:/home/user/hadoop-2.8.5/bin': not a valid identifier

after this error whenever i open termnal i get same error. I reboot system but same error

---

Hey there! Could you please share the entries you made in your .bashrc file?

Answer 1

Add the following to .bashrc. Replace HADOOP_HOME with the path to your hadoop folder:

export HADOOP_COMMON_LIB_NATIVE_DIR=$HADOOP_HOME/lib/native
export HADOOP_OPTS="-Djava.library.path=$HADOOP_HOME/lib"

Then run the following commands:

ssh-keygen -t rsa -P '' -f ~/.ssh/id_rsa
cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys

This should solve your problem

Answer 2

This should be helpful:

https://gist.github.com/ruo91/7154697

Answer 3

Export variables in hadoop-env.sh like this:

vim /usr/local/hadoop/etc/hadoop/hadoop-env.sh

hadoop-env.sh can be found in /$HADOOP_HOME/etc/hadoop/

#Hadoop variables
export JAVA_HOME=/usr/lib/jvm/java-1.7.0-openjdk-amd64 # your jdk install path
export HADOOP_INSTALL=/usr/local/hadoop
export PATH=$PATH:$HADOOP_INSTALL/bin
export PATH=$PATH:$HADOOP_INSTALL/sbin
export HADOOP_MAPRED_HOME=$HADOOP_INSTALL
export HADOOP_COMMON_HOME=$HADOOP_INSTALL
export HADOOP_HDFS_HOME=$HADOOP_INSTALL
export YARN_HOME=$HADOOP_INSTALL
export HADOOP_COMMON_LIB_NATIVE_DIR=$HADOOP_INSTALL/lib/native
export HADOOP_OPTS="-Djava.library.path=$HADOOP_INSTALL/lib"

Answer 4

To solve the ssh error, you can do this:

ssh -p 22 myName@hostname

or

ssh -l myName -p 22 hostname

Answer 5

You can re-install openssh-client and openssh-server:

$ sudo apt-get autoremove openssh-client openssh-server
$ sudo apt-get install -f openssh-client openssh-server