Big Data and Hadoop (170 Blogs) Become a Certified Professional

Helpful Hadoop Shell Commands

Last updated on May 22,2019 22.1K Views


HDFS stands for ‘Hadoop Distributed File System’. The HDFS is a sub-project of the Apache Hadoop project. This Apache Software Foundation project is designed to provide a fault-tolerant file system designed to run on commodity hardware. HDFS is accessed through a set of shell commands which will be discussed in this post.

A short note before starting: All the Hadoop Shell commands are invoked by the bin/hadoop script.

User Commands:

  • Run DFS file system:

Usage: hadoop fsck – /

Run-DFS-file-system (1)

  • Check version of Hadoop:

Usage: Hadoop version

Hadoop Version

FS Shell Commands:

The Hadoop fs command runs a generic filesystem user client that interacts with the MapR filesystem (MapR-FS).

  •  View file listings:

Usage: hadoop fs -ls hdfs :/

View File Listing in Hadoop

  • Check memory status:

Usage: hadoop fs -df hdfs :/

Command to check memory status

  • Count of Directories, Files and Bytes in specified path and file pattern:

Usage: hadoop fs -count hdfs :/

Command to Count Directories, Files and Bytes in specified path and file pattern

  • Move file from one location to another:

Usage: -mv <src> <dst>

Command to Move file from one location to another

  • Copy file from source to destination :

Usage: -cp <src> <dst>

Command to Copy file from source to destination

  • Delete File:

Usage: -rm <path>

Command to Delete File

  • Put file from the Local file system to Hadoop Distributed File System:

Usage: -put <localsrc> … <dst>

Command to Put file from the Local file system to HDFS

  • Copy file from Local to HDFS:

Usage: -copyFromLocal <localsrc> … <dst>

Command to Copy file from Local to HDFS

  • View file in Hadoop Distributed File system:

Usage: -cat <src>

Command to View file in HDFS

Administration Commands:

  • Format the namenode:

Usage: hadoop namenode -format

Format Namenode in Hadoop

  • Starting Secondary namenode:

Usage: hadoop secondrynamenode

Command for Starting Secondary namenode

  • Run namenode :

Usage: hadoop namenode

Run Namenode in Hadoop

  • Run data node:

Usage: hadoop datanode

Datanode in Hadoop

  • Cluster Balancing:

Usage: hadoop balancer

Cluster Balancing in Hadoop

  • Run MapReduce job tracker node:

Usage: hadoop jobtracker

Run MapReduce Job tracker node in Hadoop

  • Run MapReduce task tracker node:

Usage: hadoop tasktracker

Using Hadoop Task Tracker

Got a question for us? Please mention them in the comments section and we will get back to you.

Related Posts:

Get started with Big Data and Hadoop

Hadoop Cluster Configuration Files

Operators in Apache Pig

Upcoming Batches For Big Data Hadoop Certification Training Course
Course NameDate
Big Data Hadoop Certification Training Course

Class Starts on 24th December,2022

24th December

SAT&SUN (Weekend Batch)
View Details
Comments
1 Comment

Join the discussion

Browse Categories

webinar REGISTER FOR FREE WEBINAR
REGISTER NOW
webinar_success Thank you for registering Join Edureka Meetup community for 100+ Free Webinars each month JOIN MEETUP GROUP

Subscribe to our Newsletter, and get personalized recommendations.

image not found!
image not found!

Helpful Hadoop Shell Commands

edureka.co