Big Data and Hadoop (170 Blogs) Become a Certified Professional

Helpful Hadoop Shell Commands

Last updated on May 22,2019 22.9K Views


HDFS stands for ‘Hadoop Distributed File System’. The HDFS is a sub-project of the Apache Hadoop project. This Apache Software Foundation project is designed to provide a fault-tolerant file system designed to run on commodity hardware. HDFS is accessed through a set of shell commands which will be discussed in this post.

A short note before starting: All the Hadoop Shell commands are invoked by the bin/hadoop script.

User Commands:

  • Run DFS file system:

Usage: hadoop fsck – /

Run-DFS-file-system (1)

  • Check version of Hadoop:

Usage: Hadoop version

Hadoop Version

FS Shell Commands:

The Hadoop fs command runs a generic filesystem user client that interacts with the MapR filesystem (MapR-FS).

  •  View file listings:

Usage: hadoop fs -ls hdfs :/

View File Listing in Hadoop

  • Check memory status:

Usage: hadoop fs -df hdfs :/

Command to check memory status

  • Count of Directories, Files and Bytes in specified path and file pattern:

Usage: hadoop fs -count hdfs :/

Command to Count Directories, Files and Bytes in specified path and file pattern

  • Move file from one location to another:

Usage: -mv <src> <dst>

Command to Move file from one location to another

  • Copy file from source to destination :

Usage: -cp <src> <dst>

Command to Copy file from source to destination

  • Delete File:

Usage: -rm <path>

Command to Delete File

  • Put file from the Local file system to Hadoop Distributed File System:

Usage: -put <localsrc> … <dst>

Command to Put file from the Local file system to HDFS

  • Copy file from Local to HDFS:

Usage: -copyFromLocal <localsrc> … <dst>

Command to Copy file from Local to HDFS

  • View file in Hadoop Distributed File system:

Usage: -cat <src>

Command to View file in HDFS

Administration Commands:

  • Format the namenode:

Usage: hadoop namenode -format

Format Namenode in Hadoop

  • Starting Secondary namenode:

Usage: hadoop secondrynamenode

Command for Starting Secondary namenode

  • Run namenode :

Usage: hadoop namenode

Run Namenode in Hadoop

  • Run data node:

Usage: hadoop datanode

Datanode in Hadoop

  • Cluster Balancing:

Usage: hadoop balancer

Cluster Balancing in Hadoop

  • Run MapReduce job tracker node:

Usage: hadoop jobtracker

Run MapReduce Job tracker node in Hadoop

  • Run MapReduce task tracker node:

Usage: hadoop tasktracker

Using Hadoop Task Tracker

Got a question for us? Please mention them in the comments section and we will get back to you.

Related Posts:

Get started with Big Data and Hadoop

Hadoop Cluster Configuration Files

Operators in Apache Pig

Comments
1 Comment

Join the discussion

Browse Categories

webinar REGISTER FOR FREE WEBINAR
REGISTER NOW
webinar_success Thank you for registering Join Edureka Meetup community for 100+ Free Webinars each month JOIN MEETUP GROUP

Subscribe to our Newsletter, and get personalized recommendations.

image not found!
image not found!

Helpful Hadoop Shell Commands

edureka.co