HDFS stands for ‘Hadoop Distributed File System’. The HDFS is a sub-project of the Apache Hadoop project. This Apache Software Foundation project is designed to provide a fault-tolerant file system designed to run on commodity hardware. HDFS is accessed through a set of shell commands which will be discussed in this post.
A short note before starting: All the Hadoop Shell commands are invoked by the bin/hadoop script.
User Commands:
- Run DFS file system:
Usage: hadoop fsck – /
- Check version of Hadoop:
Usage: Hadoop version
FS Shell Commands:
The Hadoop fs command runs a generic filesystem user client that interacts with the MapR filesystem (MapR-FS).
- View file listings:
Usage: hadoop fs -ls hdfs :/
- Check memory status:
Usage: hadoop fs -df hdfs :/
- Count of Directories, Files and Bytes in specified path and file pattern:
Usage: hadoop fs -count hdfs :/
- Move file from one location to another:
Usage: -mv <src> <dst>
- Copy file from source to destination :
Usage: -cp <src> <dst>
- Delete File:
Usage: -rm <path>
- Put file from the Local file system to Hadoop Distributed File System:
Usage: -put <localsrc> … <dst>
- Copy file from Local to HDFS:
Usage: -copyFromLocal <localsrc> … <dst>
- View file in Hadoop Distributed File system:
Usage: -cat <src>
Administration Commands:
- Format the namenode:
Usage: hadoop namenode -format
- Starting Secondary namenode:
Usage: hadoop secondrynamenode
- Run namenode :
Usage: hadoop namenode
- Run data node:
Usage: hadoop datanode
- Cluster Balancing:
Usage: hadoop balancer
- Run MapReduce job tracker node:
Usage: hadoop jobtracker
- Run MapReduce task tracker node:
Usage: hadoop tasktracker
Got a question for us? Please mention them in the comments section and we will get back to you.
Related Posts:
Get started with Big Data and Hadoop