Hadoop 2.6.

0 Single Node Setup on Fedora

Object 1

Dear viewers,

This post will help you to single node setup of hadoop 2.6.0 on Fedora or similar systems.

Steps:

1. Install Java if not exists. (Version 1.6 +)

use sudo yum install java-package-name
After installation, cross check it by below commands

[pavan@Pavan ~]$ java -version

java version "1.7.0_b147-icedtea"
OpenJDK Runtime Environment (fedora-2.1.fc17.6-x86_64)
OpenJDK 64-Bit Server VM (build 22.0-b10, mixed mode)

[pavan@Pavan ~]$ which java

/usr/bin/java

[pavan@Pavan ~]$ whereis java

java: /bin/java /usr/bin/java /etc/java /lib64/java /usr/lib64/java /usr/share/java
/usr/share/man/man1/java.1.gz

2. set up ssh certificate

[pavan@Pavan ~]$ ssh-keygen -t rsa
Generating public/private rsa key pair.
Enter file in which to save the key (/home/pavan/.ssh/id_rsa):
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in /home/pavan/.ssh/id_rsa.
Your public key has been saved in /home/pavan/.ssh/id_rsa.pub.
The key fingerprint is:
51:79:b2:85:c8:20:28:fd:3c:b6:77:23:35:86:94:3c pavan@Pavan
The key's randomart image is:
+--[ RSA 2048]----+
| . ....+ ..o |
|. o .E o.+ o |
| . o . o. = |
| = . +.. |
| . o oS. |
| .oo |
| .o. |
| |
| |
+-----------------+
[pavan@Pavan ~]$ cat ~/.ssh/id_rsa >> ~/.ssh/authorized_keys
[pavan@Pavan ~]$ ssh localhost
pavan@localhost's password:

3. Download Hadoop-2.6.0 and extract it

You may download it from http://hadoop.apache.org/releases.html
Extract it and put it in /usr/local directory. (You can decide destination directory of your choice).
In my case I extracted it at /usr/local directory

[pavan@Pavan ~]$ ls /usr/local/

bin games hadoop-2.6.0 lib libexec share
etc include lib64 sbin src

4. Set up envorionment variable

Go to home directory and open .bashrc file in text editor. If you can not see .bashrc file in home
directory, then press ctrl+h, you will see it.
Add following lines to .bashrc

[pavan@Pavan ~]$ vi .bashrc

#Hadoop variables
# Modify JAVA_HOME and HADOOP_INSTALL as per your requirements

export JAVA_HOME=/usr/lib/jvm/java-1.7.0-openjdk-1.7.0.3.x86_64
export HADOOP_INSTALL=/usr/local/hadoop-2.6.0
export PATH=$PATH:$HADOOP_INSTALL/bin
export PATH=$PATH:$HADOOP_INSTALL/sbin
export HADOOP_MAPRED_HOME=$HADOOP_INSTALL
export HADOOP_COMMON_HOME=$HADOOP_INSTALL
export HADOOP_HDFS_HOME=$HADOOP_INSTALL
export YARN_HOME=$HADOOP_INSTALL
#end of Hadoop variable declaration

To apply changes to .bashrc run following command

[pavan@Pavan ~]$ source .bashrc

5. Open /usr/local/hadoop-2.6.0/etc/hadoop/hadoop-env.sh in text editor find below line

export JAVA_HOME=${JAVA_HOME}
and replace it with
export JAVA_HOME=/usr/lib/jvm/java-1.7.0-openjdk-1.7.0.3.x86_64

6. Check hadoop version

You shall get similar to this

[pavan@Pavan ~]$ hadoop version

Hadoop 2.6.0
Subversion https://git-wip-us.apache.org/repos/asf/hadoop.git -r
e3496499ecb8d220fba99dc5ed4c99c8f9e33bb1
Compiled by jenkins on 2014-11-13T21:10Z
Compiled with protoc 2.5.0
From source with checksum 18e43357c8f927c0695f1e9522859d6a
This command was run using /usr/local/hadoop-2.6.0/share/hadoop/common/hadoop-common-
2.6.0.jar

7. Configure xml files hadoop-2.6.0

Four files at directory /usr/local/hadoop-2.6.0/etc/hadoop need to be modified.
A. core-site.xml
B. mapred-site.xml
C. yarn-site.xml
D. hdfs-site.xml

A. Open file core-site.xml and paste following under <configuration> </configuration>

<property>
<name>fs.default.name</name>
<value>hdfs://localhost:9000</value>
</property>
<property>
<name>hadoop.tmp.dir</name>
<value>/usr/local/hadoop-2.6.0/tmp</value>
</property>

B. Open file mapred-site.xml and paste following under <configuration> </configuration>

<property>
<name>mapreduce.framework.name</name>
<value>yarn</value>
</property>
C. Open file yarn-site.xml and paste following under <configuration> </configuration>
<property>
<name>yarn.resourcemanager.resource-tracker.address</name>
<value>localhost:9025</value>
</property>

D. Set up default directories and for namenode and datanode and then edit hdfs-site.xml
fileNote that these directories must be in the users home directory, incase of fedora, otherwise the
combined space in HDFS doesn't show full access to provided storage to the system.

[pavan@Pavan ~]$ cd ~
[pavan@Pavan ~]$ pwd
/home/pavan
[pavan@Pavan ~]$ mkdir -p hadoopdata/hdfs/namenode
[pavan@Pavan ~]$ mkdir -p hadoopdata/hdfs/datanode
[pavan@Pavan ~]$ vi /usr/local/hadoop-2.6.0/etc/hadoop/hdfs-site.xml

# add following in <configuration> </configuration

#Default replication value is 3, but you can set it as per your choice.

<property>
<name>dfs.replication</name>
<value>1</value>
</property>

8. Format namenode
# As this is fresh installation, formatting name node will not harm your data, because it do not
contains any data. Formatting namenode will erase hadoop data. So just be sure when you format
hdfs next time.

[pavan@Pavan ~]$ hdfs namenode -format

15/07/31 17:39:42 INFO namenode.NameNode: STARTUP_MSG:
/************************************************************
STARTUP_MSG: Starting NameNode
STARTUP_MSG: host = Pavan/192.168.5.239
STARTUP_MSG: args = [-format]
STARTUP_MSG: version = 2.6.0
STARTUP_MSG: classpath = /usr/local/hadoop-2.6.0/etc/hadoop:/usr/local/hadoop-
2.6.0/share/hadoop/common/lib/hadoop-annotations-2.6.0.jar:/usr/local/hadoop-
2.6.0/share/hadoop/common/lib/jackson-core-asl-1.9.13.jar:/usr/local/hadoop-
2.6.0/share/hadoop/common/lib/jsr305-1.3.9.jar:/usr/local/hadoop-
2.6.0/share/hadoop/common/lib/jersey-server-1.9.jar:/usr/local/hadoop-
2.6.0/share/hadoop/common/lib/jackson-jaxrs-1.9.13.jar:/usr/local/hadoop-
2.6.0/share/hadoop/common/lib/jets3t-0.9.0.jar:/usr/local/hadoop-
2.6.0/share/hadoop/common/lib/apacheds-kerberos-codec-2.0.0-M15.jar:/usr/local/hadoop-
2.6.0/share/hadoop/common/lib/commons-cli-1.2.jar:/usr/local/hadoop-
2.6.0/share/hadoop/common/lib/jersey-core-1.9.jar:/usr/local/hadoop-
2.6.0/share/hadoop/common/lib/jasper-runtime-5.5.23.jar:/usr/local/hadoop-
2.6.0/share/hadoop/common/lib/jersey-json-1.9.jar:/usr/local/hadoop-
2.6.0/share/hadoop/common/lib/zookeeper-3.4.6.jar:/usr/local/hadoop-
2.6.0/share/hadoop/common/lib/httpcore-4.2.5.jar:/usr/local/hadoop-
2.6.0/share/hadoop/common/lib/mockito-all-1.8.5.jar:/usr/local/hadoop-
2.6.0/share/hadoop/common/lib/commons-compress-1.4.1.jar:/usr/local/hadoop-
2.6.0/share/hadoop/common/lib/jackson-xc-1.9.13.jar:/usr/local/hadoop-
2.6.0/share/hadoop/common/lib/jaxb-impl-2.2.3-1.jar:/usr/local/hadoop-
2.6.0/share/hadoop/common/lib/jaxb-api-2.2.2.jar:/usr/local/hadoop-
2.6.0/share/hadoop/common/lib/netty-3.6.2.Final.jar:/usr/local/hadoop-
2.6.0/share/hadoop/common/lib/jasper-compiler-5.5.23.jar:/usr/local/hadoop-
2.6.0/share/hadoop/common/lib/commons-lang-2.6.jar:/usr/local/hadoop-
2.6.0/share/hadoop/common/lib/paranamer-2.3.jar:/usr/local/hadoop-
2.6.0/share/hadoop/common/lib/api-asn1-api-1.0.0-M20.jar:/usr/local/hadoop-
2.6.0/share/hadoop/common/lib/activation-1.1.jar:/usr/local/hadoop-
2.6.0/share/hadoop/common/lib/log4j-1.2.17.jar:/usr/local/hadoop-
2.6.0/share/hadoop/common/lib/gson-2.2.4.jar:/usr/local/hadoop-
2.6.0/share/hadoop/common/lib/hadoop-auth-2.6.0.jar:/usr/local/hadoop-
2.6.0/share/hadoop/common/lib/apacheds-i18n-2.0.0-M15.jar:/usr/local/hadoop-
2.6.0/share/hadoop/common/lib/commons-beanutils-1.7.0.jar:/usr/local/hadoop-
2.6.0/share/hadoop/common/lib/htrace-core-3.0.4.jar:/usr/local/hadoop-
2.6.0/share/hadoop/common/lib/httpclient-4.2.5.jar:/usr/local/hadoop-
2.6.0/share/hadoop/common/lib/protobuf-java-2.5.0.jar:/usr/local/hadoop-
2.6.0/share/hadoop/common/lib/hamcrest-core-1.3.jar:/usr/local/hadoop-
2.6.0/share/hadoop/common/lib/slf4j-api-1.7.5.jar:/usr/local/hadoop-
2.6.0/share/hadoop/common/lib/curator-framework-2.6.0.jar:/usr/local/hadoop-
2.6.0/share/hadoop/common/lib/commons-el-1.0.jar:/usr/local/hadoop-
2.6.0/share/hadoop/common/lib/commons-net-3.1.jar:/usr/local/hadoop-
2.6.0/share/hadoop/common/lib/commons-digester-1.8.jar:/usr/local/hadoop-
2.6.0/share/hadoop/common/lib/xmlenc-0.52.jar:/usr/local/hadoop-
2.6.0/share/hadoop/common/lib/commons-configuration-1.6.jar:/usr/local/hadoop-
2.6.0/share/hadoop/common/lib/asm-3.2.jar:/usr/local/hadoop-2.6.0/share/hadoop/common/lib/jsp-
api-2.1.jar:/usr/local/hadoop-2.6.0/share/hadoop/common/lib/commons-collections-
3.2.1.jar:/usr/local/hadoop-2.6.0/share/hadoop/common/lib/jackson-mapper-asl-
1.9.13.jar:/usr/local/hadoop-2.6.0/share/hadoop/common/lib/api-util-1.0.0-
M20.jar:/usr/local/hadoop-2.6.0/share/hadoop/common/lib/avro-1.7.4.jar:/usr/local/hadoop-
2.6.0/share/hadoop/common/lib/jettison-1.1.jar:/usr/local/hadoop-
2.6.0/share/hadoop/common/lib/stax-api-1.0-2.jar:/usr/local/hadoop-
2.6.0/share/hadoop/common/lib/jetty-util-6.1.26.jar:/usr/local/hadoop-
2.6.0/share/hadoop/common/lib/java-xmlbuilder-0.4.jar:/usr/local/hadoop-
2.6.0/share/hadoop/common/lib/snappy-java-1.0.4.1.jar:/usr/local/hadoop-
2.6.0/share/hadoop/common/lib/slf4j-log4j12-1.7.5.jar:/usr/local/hadoop-
2.6.0/share/hadoop/common/lib/commons-codec-1.4.jar:/usr/local/hadoop-
2.6.0/share/hadoop/common/lib/servlet-api-2.5.jar:/usr/local/hadoop-
2.6.0/share/hadoop/common/lib/jsch-0.1.42.jar:/usr/local/hadoop-
2.6.0/share/hadoop/common/lib/commons-math3-3.1.1.jar:/usr/local/hadoop-
2.6.0/share/hadoop/common/lib/junit-4.11.jar:/usr/local/hadoop-
2.6.0/share/hadoop/common/lib/commons-httpclient-3.1.jar:/usr/local/hadoop-
2.6.0/share/hadoop/common/lib/curator-client-2.6.0.jar:/usr/local/hadoop-
2.6.0/share/hadoop/common/lib/jetty-6.1.26.jar:/usr/local/hadoop-
2.6.0/share/hadoop/common/lib/commons-logging-1.1.3.jar:/usr/local/hadoop-
2.6.0/share/hadoop/common/lib/commons-io-2.4.jar:/usr/local/hadoop-
2.6.0/share/hadoop/common/lib/curator-recipes-2.6.0.jar:/usr/local/hadoop-
2.6.0/share/hadoop/common/lib/xz-1.0.jar:/usr/local/hadoop-
2.6.0/share/hadoop/common/lib/commons-beanutils-core-1.8.0.jar:/usr/local/hadoop-
2.6.0/share/hadoop/common/lib/guava-11.0.2.jar:/usr/local/hadoop-
2.6.0/share/hadoop/common/hadoop-nfs-2.6.0.jar:/usr/local/hadoop-
2.6.0/share/hadoop/common/hadoop-common-2.6.0.jar:/usr/local/hadoop-
2.6.0/share/hadoop/common/hadoop-common-2.6.0-tests.jar:/usr/local/hadoop-
2.6.0/share/hadoop/hdfs:/usr/local/hadoop-2.6.0/share/hadoop/hdfs/lib/jackson-core-asl-
1.9.13.jar:/usr/local/hadoop-2.6.0/share/hadoop/hdfs/lib/xml-apis-1.3.04.jar:/usr/local/hadoop-
2.6.0/share/hadoop/hdfs/lib/jsr305-1.3.9.jar:/usr/local/hadoop-2.6.0/share/hadoop/hdfs/lib/jersey-
server-1.9.jar:/usr/local/hadoop-2.6.0/share/hadoop/hdfs/lib/commons-cli-1.2.jar:/usr/local/hadoop-
2.6.0/share/hadoop/hdfs/lib/jersey-core-1.9.jar:/usr/local/hadoop-2.6.0/share/hadoop/hdfs/lib/jasper-
runtime-5.5.23.jar:/usr/local/hadoop-2.6.0/share/hadoop/hdfs/lib/netty-
3.6.2.Final.jar:/usr/local/hadoop-2.6.0/share/hadoop/hdfs/lib/commons-lang-
2.6.jar:/usr/local/hadoop-2.6.0/share/hadoop/hdfs/lib/log4j-1.2.17.jar:/usr/local/hadoop-
2.6.0/share/hadoop/hdfs/lib/htrace-core-3.0.4.jar:/usr/local/hadoop-
2.6.0/share/hadoop/hdfs/lib/protobuf-java-2.5.0.jar:/usr/local/hadoop-
2.6.0/share/hadoop/hdfs/lib/commons-el-1.0.jar:/usr/local/hadoop-
2.6.0/share/hadoop/hdfs/lib/xmlenc-0.52.jar:/usr/local/hadoop-2.6.0/share/hadoop/hdfs/lib/asm-
3.2.jar:/usr/local/hadoop-2.6.0/share/hadoop/hdfs/lib/jsp-api-2.1.jar:/usr/local/hadoop-
2.6.0/share/hadoop/hdfs/lib/jackson-mapper-asl-1.9.13.jar:/usr/local/hadoop-
2.6.0/share/hadoop/hdfs/lib/jetty-util-6.1.26.jar:/usr/local/hadoop-
2.6.0/share/hadoop/hdfs/lib/commons-codec-1.4.jar:/usr/local/hadoop-
2.6.0/share/hadoop/hdfs/lib/servlet-api-2.5.jar:/usr/local/hadoop-
2.6.0/share/hadoop/hdfs/lib/commons-daemon-1.0.13.jar:/usr/local/hadoop-
2.6.0/share/hadoop/hdfs/lib/jetty-6.1.26.jar:/usr/local/hadoop-2.6.0/share/hadoop/hdfs/lib/commons-
logging-1.1.3.jar:/usr/local/hadoop-2.6.0/share/hadoop/hdfs/lib/commons-io-
2.4.jar:/usr/local/hadoop-2.6.0/share/hadoop/hdfs/lib/xercesImpl-2.9.1.jar:/usr/local/hadoop-
2.6.0/share/hadoop/hdfs/lib/guava-11.0.2.jar:/usr/local/hadoop-2.6.0/share/hadoop/hdfs/hadoop-
hdfs-2.6.0.jar:/usr/local/hadoop-2.6.0/share/hadoop/hdfs/hadoop-hdfs-2.6.0-
tests.jar:/usr/local/hadoop-2.6.0/share/hadoop/hdfs/hadoop-hdfs-nfs-2.6.0.jar:/usr/local/hadoop-
2.6.0/share/hadoop/yarn/lib/jackson-core-asl-1.9.13.jar:/usr/local/hadoop-
2.6.0/share/hadoop/yarn/lib/jsr305-1.3.9.jar:/usr/local/hadoop-2.6.0/share/hadoop/yarn/lib/jersey-
server-1.9.jar:/usr/local/hadoop-2.6.0/share/hadoop/yarn/lib/jackson-jaxrs-
1.9.13.jar:/usr/local/hadoop-2.6.0/share/hadoop/yarn/lib/commons-cli-1.2.jar:/usr/local/hadoop-
2.6.0/share/hadoop/yarn/lib/jersey-core-1.9.jar:/usr/local/hadoop-2.6.0/share/hadoop/yarn/lib/jersey-
json-1.9.jar:/usr/local/hadoop-2.6.0/share/hadoop/yarn/lib/zookeeper-3.4.6.jar:/usr/local/hadoop-
2.6.0/share/hadoop/yarn/lib/jline-0.9.94.jar:/usr/local/hadoop-
2.6.0/share/hadoop/yarn/lib/commons-compress-1.4.1.jar:/usr/local/hadoop-
2.6.0/share/hadoop/yarn/lib/jackson-xc-1.9.13.jar:/usr/local/hadoop-
2.6.0/share/hadoop/yarn/lib/jaxb-impl-2.2.3-1.jar:/usr/local/hadoop-
2.6.0/share/hadoop/yarn/lib/jaxb-api-2.2.2.jar:/usr/local/hadoop-2.6.0/share/hadoop/yarn/lib/netty-
3.6.2.Final.jar:/usr/local/hadoop-2.6.0/share/hadoop/yarn/lib/commons-lang-
2.6.jar:/usr/local/hadoop-2.6.0/share/hadoop/yarn/lib/activation-1.1.jar:/usr/local/hadoop-
2.6.0/share/hadoop/yarn/lib/log4j-1.2.17.jar:/usr/local/hadoop-2.6.0/share/hadoop/yarn/lib/protobuf-
java-2.5.0.jar:/usr/local/hadoop-2.6.0/share/hadoop/yarn/lib/jersey-client-1.9.jar:/usr/local/hadoop-
2.6.0/share/hadoop/yarn/lib/leveldbjni-all-1.8.jar:/usr/local/hadoop-
2.6.0/share/hadoop/yarn/lib/guice-servlet-3.0.jar:/usr/local/hadoop-2.6.0/share/hadoop/yarn/lib/asm-
3.2.jar:/usr/local/hadoop-2.6.0/share/hadoop/yarn/lib/commons-collections-
3.2.1.jar:/usr/local/hadoop-2.6.0/share/hadoop/yarn/lib/aopalliance-1.0.jar:/usr/local/hadoop-
2.6.0/share/hadoop/yarn/lib/jackson-mapper-asl-1.9.13.jar:/usr/local/hadoop-
2.6.0/share/hadoop/yarn/lib/jersey-guice-1.9.jar:/usr/local/hadoop-
2.6.0/share/hadoop/yarn/lib/jettison-1.1.jar:/usr/local/hadoop-2.6.0/share/hadoop/yarn/lib/guice-
3.0.jar:/usr/local/hadoop-2.6.0/share/hadoop/yarn/lib/stax-api-1.0-2.jar:/usr/local/hadoop-
2.6.0/share/hadoop/yarn/lib/jetty-util-6.1.26.jar:/usr/local/hadoop-
2.6.0/share/hadoop/yarn/lib/commons-codec-1.4.jar:/usr/local/hadoop-
2.6.0/share/hadoop/yarn/lib/servlet-api-2.5.jar:/usr/local/hadoop-
2.6.0/share/hadoop/yarn/lib/javax.inject-1.jar:/usr/local/hadoop-
2.6.0/share/hadoop/yarn/lib/commons-httpclient-3.1.jar:/usr/local/hadoop-
2.6.0/share/hadoop/yarn/lib/jetty-6.1.26.jar:/usr/local/hadoop-
2.6.0/share/hadoop/yarn/lib/commons-logging-1.1.3.jar:/usr/local/hadoop-
2.6.0/share/hadoop/yarn/lib/commons-io-2.4.jar:/usr/local/hadoop-2.6.0/share/hadoop/yarn/lib/xz-
1.0.jar:/usr/local/hadoop-2.6.0/share/hadoop/yarn/lib/guava-11.0.2.jar:/usr/local/hadoop-
2.6.0/share/hadoop/yarn/hadoop-yarn-server-resourcemanager-2.6.0.jar:/usr/local/hadoop-
2.6.0/share/hadoop/yarn/hadoop-yarn-server-tests-2.6.0.jar:/usr/local/hadoop-
2.6.0/share/hadoop/yarn/hadoop-yarn-common-2.6.0.jar:/usr/local/hadoop-
2.6.0/share/hadoop/yarn/hadoop-yarn-api-2.6.0.jar:/usr/local/hadoop-
2.6.0/share/hadoop/yarn/hadoop-yarn-registry-2.6.0.jar:/usr/local/hadoop-
2.6.0/share/hadoop/yarn/hadoop-yarn-server-nodemanager-2.6.0.jar:/usr/local/hadoop-
2.6.0/share/hadoop/yarn/hadoop-yarn-server-common-2.6.0.jar:/usr/local/hadoop-
2.6.0/share/hadoop/yarn/hadoop-yarn-applications-distributedshell-2.6.0.jar:/usr/local/hadoop-
2.6.0/share/hadoop/yarn/hadoop-yarn-server-web-proxy-2.6.0.jar:/usr/local/hadoop-
2.6.0/share/hadoop/yarn/hadoop-yarn-applications-unmanaged-am-launcher-
2.6.0.jar:/usr/local/hadoop-2.6.0/share/hadoop/yarn/hadoop-yarn-server-applicationhistoryservice-
2.6.0.jar:/usr/local/hadoop-2.6.0/share/hadoop/yarn/hadoop-yarn-client-2.6.0.jar:/usr/local/hadoop-
2.6.0/share/hadoop/mapreduce/lib/hadoop-annotations-2.6.0.jar:/usr/local/hadoop-
2.6.0/share/hadoop/mapreduce/lib/jackson-core-asl-1.9.13.jar:/usr/local/hadoop-
2.6.0/share/hadoop/mapreduce/lib/jersey-server-1.9.jar:/usr/local/hadoop-
2.6.0/share/hadoop/mapreduce/lib/jersey-core-1.9.jar:/usr/local/hadoop-
2.6.0/share/hadoop/mapreduce/lib/commons-compress-1.4.1.jar:/usr/local/hadoop-
2.6.0/share/hadoop/mapreduce/lib/netty-3.6.2.Final.jar:/usr/local/hadoop-
2.6.0/share/hadoop/mapreduce/lib/paranamer-2.3.jar:/usr/local/hadoop-
2.6.0/share/hadoop/mapreduce/lib/log4j-1.2.17.jar:/usr/local/hadoop-
2.6.0/share/hadoop/mapreduce/lib/protobuf-java-2.5.0.jar:/usr/local/hadoop-
2.6.0/share/hadoop/mapreduce/lib/hamcrest-core-1.3.jar:/usr/local/hadoop-
2.6.0/share/hadoop/mapreduce/lib/leveldbjni-all-1.8.jar:/usr/local/hadoop-
2.6.0/share/hadoop/mapreduce/lib/guice-servlet-3.0.jar:/usr/local/hadoop-
2.6.0/share/hadoop/mapreduce/lib/asm-3.2.jar:/usr/local/hadoop-
2.6.0/share/hadoop/mapreduce/lib/aopalliance-1.0.jar:/usr/local/hadoop-
2.6.0/share/hadoop/mapreduce/lib/jackson-mapper-asl-1.9.13.jar:/usr/local/hadoop-
2.6.0/share/hadoop/mapreduce/lib/avro-1.7.4.jar:/usr/local/hadoop-
2.6.0/share/hadoop/mapreduce/lib/jersey-guice-1.9.jar:/usr/local/hadoop-
2.6.0/share/hadoop/mapreduce/lib/guice-3.0.jar:/usr/local/hadoop-
2.6.0/share/hadoop/mapreduce/lib/snappy-java-1.0.4.1.jar:/usr/local/hadoop-
2.6.0/share/hadoop/mapreduce/lib/javax.inject-1.jar:/usr/local/hadoop-
2.6.0/share/hadoop/mapreduce/lib/junit-4.11.jar:/usr/local/hadoop-
2.6.0/share/hadoop/mapreduce/lib/commons-io-2.4.jar:/usr/local/hadoop-
2.6.0/share/hadoop/mapreduce/lib/xz-1.0.jar:/usr/local/hadoop-
2.6.0/share/hadoop/mapreduce/hadoop-mapreduce-client-hs-plugins-2.6.0.jar:/usr/local/hadoop-
2.6.0/share/hadoop/mapreduce/hadoop-mapreduce-client-hs-2.6.0.jar:/usr/local/hadoop-
2.6.0/share/hadoop/mapreduce/hadoop-mapreduce-client-core-2.6.0.jar:/usr/local/hadoop-
2.6.0/share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-2.6.0-tests.jar:/usr/local/hadoop-
2.6.0/share/hadoop/mapreduce/hadoop-mapreduce-client-shuffle-2.6.0.jar:/usr/local/hadoop-
2.6.0/share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-2.6.0.jar:/usr/local/hadoop-
2.6.0/share/hadoop/mapreduce/hadoop-mapreduce-examples-2.6.0.jar:/usr/local/hadoop-
2.6.0/share/hadoop/mapreduce/hadoop-mapreduce-client-common-2.6.0.jar:/usr/local/hadoop-
2.6.0/share/hadoop/mapreduce/hadoop-mapreduce-client-app-2.6.0.jar:/contrib/capacity-
scheduler/*.jar
STARTUP_MSG: build = https://git-wip-us.apache.org/repos/asf/hadoop.git -r
e3496499ecb8d220fba99dc5ed4c99c8f9e33bb1; compiled by 'jenkins' on 2014-11-13T21:10Z
STARTUP_MSG: java = 1.7.0_b147-icedtea
************************************************************/
15/07/31 17:39:42 INFO namenode.NameNode: registered UNIX signal handlers for [TERM, HUP,
INT]
15/07/31 17:39:42 INFO namenode.NameNode: createNameNode [-format]
Formatting using clusterid: CID-baa43796-9698-4563-8c45-c3c53740104d
15/07/31 17:39:44 INFO namenode.FSNamesystem: No KeyProvider found.
15/07/31 17:39:44 INFO namenode.FSNamesystem: fsLock is fair:true
15/07/31 17:39:44 INFO blockmanagement.DatanodeManager: dfs.block.invalidate.limit=1000
15/07/31 17:39:44 INFO blockmanagement.DatanodeManager:
dfs.namenode.datanode.registration.ip-hostname-check=true
15/07/31 17:39:44 INFO blockmanagement.BlockManager:
dfs.namenode.startup.delay.block.deletion.sec is set to 000:00:00:00.000
15/07/31 17:39:44 INFO blockmanagement.BlockManager: The block deletion will start around
2015 Jul 31 17:39:44
15/07/31 17:39:44 INFO util.GSet: Computing capacity for map BlocksMap
15/07/31 17:39:44 INFO util.GSet: VM type = 64-bit
15/07/31 17:39:44 INFO util.GSet: 2.0% max memory 888.9 MB = 17.8 MB
15/07/31 17:39:44 INFO util.GSet: capacity = 2^21 = 2097152 entries
15/07/31 17:39:44 INFO blockmanagement.BlockManager: dfs.block.access.token.enable=false
15/07/31 17:39:44 INFO blockmanagement.BlockManager: defaultReplication =1
15/07/31 17:39:44 INFO blockmanagement.BlockManager: maxReplication = 512
15/07/31 17:39:44 INFO blockmanagement.BlockManager: minReplication =1
15/07/31 17:39:44 INFO blockmanagement.BlockManager: maxReplicationStreams =2
15/07/31 17:39:44 INFO blockmanagement.BlockManager: shouldCheckForEnoughRacks = false
15/07/31 17:39:44 INFO blockmanagement.BlockManager: replicationRecheckInterval = 3000
15/07/31 17:39:44 INFO blockmanagement.BlockManager: encryptDataTransfer = false
15/07/31 17:39:44 INFO blockmanagement.BlockManager: maxNumBlocksToLog = 1000
15/07/31 17:39:44 INFO namenode.FSNamesystem: fsOwner = pavan (auth:SIMPLE)
15/07/31 17:39:44 INFO namenode.FSNamesystem: supergroup = supergroup
15/07/31 17:39:44 INFO namenode.FSNamesystem: isPermissionEnabled = true
15/07/31 17:39:44 INFO namenode.FSNamesystem: HA Enabled: false
15/07/31 17:39:44 INFO namenode.FSNamesystem: Append Enabled: true
15/07/31 17:39:44 INFO util.GSet: Computing capacity for map INodeMap
15/07/31 17:39:44 INFO util.GSet: VM type = 64-bit
15/07/31 17:39:44 INFO util.GSet: 1.0% max memory 888.9 MB = 8.9 MB
15/07/31 17:39:44 INFO util.GSet: capacity = 2^20 = 1048576 entries
15/07/31 17:39:44 INFO namenode.NameNode: Caching file names occuring more than 10 times
15/07/31 17:39:44 INFO util.GSet: Computing capacity for map cachedBlocks
15/07/31 17:39:44 INFO util.GSet: VM type = 64-bit
15/07/31 17:39:44 INFO util.GSet: 0.25% max memory 888.9 MB = 2.2 MB
15/07/31 17:39:44 INFO util.GSet: capacity = 2^18 = 262144 entries
15/07/31 17:39:44 INFO namenode.FSNamesystem: dfs.namenode.safemode.threshold-pct =
0.9990000128746033
15/07/31 17:39:44 INFO namenode.FSNamesystem: dfs.namenode.safemode.min.datanodes = 0
15/07/31 17:39:44 INFO namenode.FSNamesystem: dfs.namenode.safemode.extension = 30000
15/07/31 17:39:44 INFO namenode.FSNamesystem: Retry cache on namenode is enabled
15/07/31 17:39:44 INFO namenode.FSNamesystem: Retry cache will use 0.03 of total heap and
retry cache entry expiry time is 600000 millis
15/07/31 17:39:44 INFO util.GSet: Computing capacity for map NameNodeRetryCache
15/07/31 17:39:44 INFO util.GSet: VM type = 64-bit
15/07/31 17:39:44 INFO util.GSet: 0.029999999329447746% max memory 888.9 MB = 273.1 KB
15/07/31 17:39:44 INFO util.GSet: capacity = 2^15 = 32768 entries
15/07/31 17:39:44 INFO namenode.NNConf: ACLs enabled? false
15/07/31 17:39:44 INFO namenode.NNConf: XAttrs enabled? true
15/07/31 17:39:44 INFO namenode.NNConf: Maximum size of an xattr: 16384
15/07/31 17:39:44 INFO namenode.FSImage: Allocated new BlockPoolId: BP-837338973-
192.168.5.239-1438344584385
15/07/31 17:39:44 INFO common.Storage: Storage directory /usr/local/hadoop-2.6.0/data/dfs/name
has been successfully formatted.
15/07/31 17:39:44 INFO namenode.NNStorageRetentionManager: Going to retain 1 images with
txid >= 0
15/07/31 17:39:44 INFO util.ExitUtil: Exiting with status 0
15/07/31 17:39:44 INFO namenode.NameNode: SHUTDOWN_MSG:
/************************************************************
SHUTDOWN_MSG: Shutting down NameNode at Pavan/192.168.5.239
************************************************************/

9. Start hadoop service.

Either you can start individual service or you can start all. These daemons are available at
/usr/local/hadoop-2.6.0/sbin

[pavan@Pavan ~]$ cd /usr/local/hadoop-2.6.0/sbin/

[pavan@Pavan sbin]$ start-all.sh

This script is Deprecated. Instead use start-dfs.sh and start-yarn.sh

Starting namenodes on [localhost]
pavan@localhost's password:
localhost: starting namenode, logging to /usr/local/hadoop-2.6.0/logs/hadoop-pavan-namenode-
Pavan.out
pavan@localhost's password:
localhost: starting datanode, logging to /usr/local/hadoop-2.6.0/logs/hadoop-pavan-datanode-
Pavan.out
Starting secondary namenodes [0.0.0.0]
The authenticity of host '0.0.0.0 (0.0.0.0)' can't be established.
RSA key fingerprint is fb:23:08:a5:a8:4c:97:23:82:b5:49:08:e8:ad:ee:b4.
Are you sure you want to continue connecting (yes/no)? yes
0.0.0.0: Warning: Permanently added '0.0.0.0' (RSA) to the list of known hosts.
pavan@0.0.0.0's password:
0.0.0.0: starting secondarynamenode, logging to /usr/local/hadoop-2.6.0/logs/hadoop-pavan-
secondarynamenode-Pavan.out
starting yarn daemons
starting resourcemanager, logging to /usr/local/hadoop-2.6.0/logs/yarn-pavan-resourcemanager-
Pavan.out
pavan@localhost's password:
localhost: starting nodemanager, logging to /usr/local/hadoop-2.6.0/logs/yarn-pavan-nodemanager-
Pavan.out

10. Hadoop configuration is done. To check the java process status run following

[pavan@Pavan sbin]$ jps

7347 DataNode
7654 ResourceManager
7947 NodeManager
8044 Jps
7508 SecondaryNameNode
7220 NameNode

11. To see Web UI of hadoop overview, put following in browser

localhost:50070

12. To stop all processes run following[pavan@Pavan sbin]$ stop-all.sh

Congratulation !!! you have successfully setup hadoop 2.6.0 single node cluster.

Thanks.
References:
1. http://www.michael-noll.com/tutorials/running-hadoop-on-ubuntu-linux-single-node-cluster/
2. https://accretioninfinity.wordpress.com/2013/06/11/installing-hadoop-for-fedora-oracle-
linuxsingle-node-cluster/

Move folder hadoop-2.6.0 to /usr/local

mv hadoop-2.6.0 /usr/local

mkdir -p /usr/local/hadoop

To remove Hadoop folder from /user/local

[root@localhost hadoop]# rm -r -f /usr/local/hadoop/

[root@localhost hadoop]# rm -r -f /usr/local/hadoop-2.6.0