Helpful Hadoop Shell Commands

HDFS stands for ‘Hadoop Distributed File System’. The HDFS is a sub-project of the Apache Hadoop project. This Apache Software Foundation project is designed to provide a fault-tolerant file system designed to run on commodity hardware. HDFS is accessed through a set of shell commands which will be discussed in this post.

A short note before starting: All the Hadoop Shell commands are invoked by the bin/hadoop script.

User Commands:

• Run DFS file system:

Usage: hadoop fsck – /

• Check version of Hadoop:

Usage: Hadoop version

FS Shell Commands:

The Hadoop fs command runs a generic filesystem user client that interacts with the MapR filesystem (MapR-FS).

•  View file listings:

Usage: hadoop fs -ls hdfs :/

• Check memory status:

Usage: hadoop fs -df hdfs :/

• Count of Directories, Files and Bytes in specified path and file pattern:

Usage: hadoop fs -count hdfs :/

• Move file from one location to another:

Usage: -mv <src> <dst>

• Copy file from source to destination :

Usage: -cp <src> <dst>

• Delete File:

Usage: -rm <path>

• Put file from the Local file system to Hadoop Distributed File System:

Usage: -put <localsrc> … <dst>

• Copy file from Local to HDFS:

Usage: -copyFromLocal <localsrc> … <dst>

• View file in Hadoop Distributed File system:

Usage: -cat <src>

Administration Commands:

• Format the namenode:

Usage: hadoop namenode -format

• Starting Secondary namenode:

Usage: hadoop secondrynamenode

• Run namenode :

Usage: hadoop namenode

• Run data node:

Usage: hadoop datanode

• Cluster Balancing:

Usage: hadoop balancer

• Run MapReduce job tracker node:

Usage: hadoop jobtracker

• Run MapReduce task tracker node:

Usage: hadoop tasktracker

Got a question for us? Please mention them in the comments section and we will get back to you.

Related Posts:

Get started with Big Data and Hadoop

Hadoop Cluster Configuration Files

Operators in Apache Pig