Hadoop Commands

HADOOP COMMANDS
OS : Ubuntu Environment
Author : Bottu Gurunadha Rao
Created: 31-Jan-2022
Updated: 31-Jan-2022
Release  : 1.0.1
Purpose: To learn Hadoop distributed file system commands.
History: N/A

Note :To execute hadoop commands every command need to be preceded by hadoop fs or hdfs dfs combination.
Actually hadoop is deprecated and so we recommended to use hdfs dfs combination.

1. To check your location.
id -u
if you get zero, it means you are in root, since a UID of 0 (zero) means "root".
If you get any other number, it indicates that you are not in root.

2. To check current ubuntu version
lsb_release -a

3.Creating / Adding a new user
Syntax: sudo adduser <username>
        sudo adduser guru
Note : It asks for password and confirm password provide it. Keep on pressing enter keys for rest of questions. But for last one type yes and press enter key.

4.Deleting an existing user.
Syntax : sudo deluser <username>
         sudo deluser guru
Note : Now no longer it is useful, but its name is visible in home. To remove it permanently use rm command.

5.Deleting a user using rm.
Syntax: sudo rm -r <username>
        sudo rm -r guru

6.List all the available user in home.
ls /home

7.Display contents in local filesystem.
ls

8.Display contents in distributed(hadoop/hdfs) filesystem.
hadoop fs -ls /
or
hdfs dfs -ls /

9.To check health of hadoop file system.
hadoop fsck /
or
hdfs fsck /

checking the health of a file in the directory
hadoop fsck /d1 -files  ( here d1 is the directory)

10.To know hadoop version
hadoop version or hdfs version

11.To check hadoop root
hadoop fs -ls /
or
hdfs dfs -ls /

12.creating a directory in hadoop filesystem
hdfs dfs -mkdir -p /tst    
Note : In above command tst is directory and -p is option to create a directory

13.creating an empty file in hadoop filesystem root
hdfs dfs -touchz /n1.txt

14.creating an empty file in hadoop filesystem sub directory
hdfs dfs -touchz /d2/t2.txt
Note: In the above command d2 is directory and t2.txt is an empty file.

15.copying a file from local file system to hadoop file system.
Method1: Using appendToFile command
hdfs dfs -appendToFile - /d1/gurus.txt

Note1:Once above command is issued cursor waits for user to enter some text..after entering some text save it by pressing CTRL+D.
Note2:To check weather it is created or not use the following command.
hdfs dfs -cat /d1/gurus.txt