Faculty: Ms.

Sana Shaikh Hadoop Installation Subject: Big Data Analytics

2020-2021
Hadoop Modes

1. Standalone(Local) Mode
2. Pseudo-Distributed Mode(Single node)
3. Fully Distributed Mode

1. sudo apt-get update

2. sudo apt-get install default-jdk

3. java -version

4. sudo apt-get install ssh

Faculty: Ms. Sana Shaikh Hadoop Installation Subject: Big Data Analytics
2020-2021

Press ‘Y’ and continue

5. ssh localhost

Press ‘Yes’ and continue and enter password

6.

Note-Before going further we need to exit ssh just type “exit” in same terminal. so we need to set
our ssh for password less communication. To do that execute following command in terminal.

$ ssh-keygen -t rsa -P ''

press ‘Enter’
Faculty: Ms. Sana Shaikh Hadoop Installation Subject: Big Data Analytics
2020-2021

7. cat $HOME/.ssh/id_rsa.pub >> $HOME/.ssh/authorized_keys

To check we have bypass the password we need to again execute

8. $ ssh localhost

9. $ sudo mv Desktop/hadoop-2.7.7 /usr/local/hadoop

Faculty: Ms. Sana Shaikh Hadoop Installation Subject: Big Data Analytics
2020-2021

10. $ nano ~/.bashrc

type:

#Hadoop variables
export JAVA_HOME=/usr/lib/jvm/java-1.8.0-openjdk-amd64
export HADOOP_INSTALL=/usr/local/hadoop
export PATH=$PATH:$HADOOP_INSTALL/bin
export PATH=$PATH:$HADOOP_INSTALL/sbin
export HADOOP_MAPRED_HOME=$HADOOP_INSTALL
export HADOOP_COMMON_HOME=$HADOOP_INSTALL
export HADOOP_HDFS_HOME=$HADOOP_INSTALL
export YARN_HOME=$HADOOP_INSTALL
#end of Hadoop variable declaration

ct+x... + y ... + enter

11. $ source ~/.bashrc

12. hadoop
version
Faculty: Ms. Sana Shaikh Hadoop Installation Subject: Big Data Analytics
2020-2021

13. OPEN (/usr/local/hadoop/etc/hadoop/hadoop-env.sh)

type and save and exit : export JAVA_HOME=/usr/lib/jvm/java-1.8.0-openjdk-amd64

14. To configure hadoop in Pseudo Distributed mode we need to edit following files
1)core-site.xml
2)hdfs-site.xml
3)mapred-site.xml
4)yarn-site.xml

14-1 OPEN (/usr/local/hadoop/etc/hadoop/core-site.xml)

type and save
Faculty: Ms. Sana Shaikh Hadoop Installation Subject: Big Data Analytics
2020-2021
<configuration>
<property>
<name>fs.defaultFS</name>
<value>hdfs://localhost:9000</value>
</property>
<property>
<name>hadoop.tmp.dir</name>
<value>/home/dbit/tmp</value>
</property>
</configuration>

14- 2

OPEN (/usr/local/hadoop/etc/hadoop/hdfs-site.xml)
type and save

<configuration>
<property>
<name>dfs.replication</name>
<value>1</value>
</property>
<property>
<name>dfs.namenode.name.dir</name>
<value>/home/dbit/tmp/namenode</value>
</property>
<property>
<name>dfs.datanode.data.dir</name>
<value>/home/dbit/tmp/datanode</value>
</property>
Faculty: Ms. Sana Shaikh Hadoop Installation Subject: Big Data Analytics
2020-2021
</configuration>

14-3

OPEN (/usr/local/hadoop/etc/hadoop/mapred-site.xml)
type and save

<configuration>
<property>
<name>mapreduce.framework.name</name>
<value>yarn</value>
</property>
</configuration>
Faculty: Ms. Sana Shaikh Hadoop Installation Subject: Big Data Analytics
2020-2021

14-4 OPEN

(/usr/local/hadoop/etc/hadoop/yarn-site.xml)
type and save

<configuration>
<property>
<name>yarn.nodemanager.aux-services</name>
<value>mapreduce_shuffle</value>
</property>
</configuration>

15.

$ hdfs namenode -format

Faculty: Ms. Sana Shaikh Hadoop Installation Subject: Big Data Analytics
2020-2021

Now to start hadoop you can use two command

$ start-dfs.sh
$ start-yarn.sh

or you can also use deprecated command as

$ start-all.sh

To check the which components are working you can use bellow command
$ jps

you will get output as

type in browser:
https://localhost:50070/
Faculty: Ms. Sana Shaikh Hadoop Installation Subject: Big Data Analytics
2020-2021