Hadoop 1.03 and Hive on Ubuntu 12.

04
Download VMWare Workstation
https:!".v!ware.#o!webv!waredetails$produ#t%d&241'download(roup&W)*+,-04,./
Download Ubuntu Desktop 12.04 0MD
http:releases.ubuntu.#o!12.04ubuntu,12.04,desktop,a!d14.iso
2reate the i!a3e 400( disk4 2 pro#essors4 5brid3ed6 network
.aun#h a +er!inal via Dash Ho!e
7 sudo apt,3et update
Wait 8or Update Mana3er to auto,laun#h9 2li#k 5%nstall Updates6
:eboot
.aun#h +er!inal
7 sudo nano et#sudoers
2op" 8ro! (uest is 2trl,*hi8t,2
;aste to (uest is 2trl,*hi8t,V
0ppend to end o8 et#sudoers
0.. & <0..= >?;0**WD: 0..
**H %nstru#tions
http:#loud8ront.blo3spot.#o!20120@how,to,setup,and,#on8i3ure,ssh,on,ubuntu.ht!l
7 ssh,ke"3en ,t rsa ,; AA
%nstall ?ra#le Bava
http:#loud8ront.blo3spot.in20120@how,to,install,sunora#le,Cava,on,ubuntu.ht!l
7 sudo add,apt,repositor" ppa:webupd-tea!Cava
7 sudo apt,3et update
7 sudo apt,3et install ora#le,Cava@,installer
7 Cava ,version
%nstall Hadoop
http:#loud8ront.blo3spot.#o!20120@how,to,#on8i3ure,hadoop.ht!l
7 w3et ,# http:!irror.!etro#ast.netapa#hehadoop#o!!onhadoop,1.0.3hadoop,1.0.3,bin.tar.3D
7 tar ,DEv8 hadoop,1.0.3,bin.tar.3D
7 nano .bashr#
0ppend to .bashr#
eEport H0D??;FH?MG&ho!e!"userna!ehadoop,1.0.3
2lose the +er!inal and laun#h a new one to pi#k up the new environ!ent variable
7 eEit
*et B0V0FH?MG in hadoop env
7 #d hadoop,1.0.3#on8
7 nano hadoop,env.sh
0ppend neEt to #o!!ented B0V0FH?MG
eEport B0V0FH?MG&usrlibCv!Cava,@,ora#le
2reate hd8s tar3et dire#tories
7 !kdir Hhd8s
7 !kdir Hhd8sna!e
7 !kdir Hhd8sdata
7 !kdir Hhd8st!p
7 sudo #h!od ,: @II Hhd8s
Modi8" the #on8i3 8iles as des#ribed in: http:#loud8ront.blo3spot.#o!20120@how,to,#on8i3ure,
hadoop.ht!l
7 sudo nano Hhadoop,1.0.3#on8#ore,site.E!l
7 sudo nano Hhadoop,1.0.3#on8hd8s,site.E!l
7 sudo nano Hhadoop,1.0.3#on8!apred,site.E!l
Jor!at the na!enode and start hadoop servi#es
7 Hhadoop,1.0.3binhadoop na!enode ,8or!at
7 Hhadoop,1.0.3binstart,all.sh
2on8ir! servi#es are started
7 Cps
Hadoop status
http:lo#alhost:I00@0
Map :edu#e status
http:lo#alhost:I0030
%nstall Hive https:#wiki.apa#he.or3#on8luen#edispla"Hive(ettin3*tartedK(ettin3*tarted,
%nstallationand2on8i3uration
7 w3et ,# http:apa#he.#laD.or3hivehive,0.L.0hive,0.L.0,bin.tar.3D
7 tar ,EDv8 hive,0.L.0,bin.tar.3D
0dd these lines to H.bashr# and restart "our ter!inal
eEport H0D??;FH?MG&ho!e!"userna!ehadoop,1.0.3
eEport H%VGFH?MG&ho!e!"userna!ehive,0.L.0,bin
eEport ;0+H&7H%VGFH?MGbin:7;0+H
eEport ;0+H&7H0D??;FH?MGbin:7;0+H
2reate hive dire#tories within hd8s and set per!issions 8or table #reate
7 hadoop 8s ,!kdir userhivewarehouse
7 hadoop 8s ,!kdir t!p
7 hadoop 8s ,#h!od 3Mw userhivewarehouse
7 hadoop 8s ,#h!od 3Mw t!p
.aun#h hive and #reate sa!ple tables
7 hive
hiveN 2:G0+G +0O.G shakespeare <8reP %>+4 word *+:%>(= :?W J?:M0+ DG.%M%+GD
J%G.D* +G:M%>0+GD OQ RStR *+?:GD 0* +G/+J%.G9
hiveN 2:G0+G +0O.G kCv <8reP %>+4 word *+:%>(= :?W J?:M0+ DG.%M%+GD J%G.D*
+G:M%>0+GD OQ RStR *+?:GD 0* +G/+J%.G9
hiveN eEit9
Download sa!ple data 8ro! 2loudera
7 w3et ,? shakespeare.tar.3D https:3ithub.#o!#loudera#loudera,
trainin3blob!asterdatashakespeare.tar.3D$raw&true
7 w3et ,? bible.tar.3D https:3ithub.#o!#loudera#loudera,trainin3blob!asterdatabible.tar.3D$
raw&true
7 tar ,DvE8 bible.tar.3D
7 tar ,DvE8 shakespeare.tar.3D
;ut the *hakespeare sa!ple data into hd8s
7 hadoop 8s ,!kdir shakespeare,input
7 hadoop 8s ,put Hinputall,shakespeare user!"userna!eshakespeare,input
7 hadoop 8s ,ls shakespeare,input
:un the 53rep6 sa!ple a3ainst the hd8s dire#tor" 5shakespeare,input6 and pla#e results in
5shakespeareF8reP6
7 hadoop Car Hhadoop,1.0.3hadoop,eEa!ples,1.0.3.Car 3rep shakespeare,input shakespeareF8reP
RSwMR
7 hadoop 8s ,ls shakespeareF8reP
;ut the bible sa!ple data into hd8s
7 hadoop 8s ,!kdir bible,input
7 hadoop 8s ,put Hbibleall,bible user!"userna!ebible,input
7 hadoop 8s ,ls bible,input
:un the 53rep6 sa!ple a3ainst the hd8s dire#tor" 5bible,input6 and pla#e results in 5bibleF8reP6
7 hadoop Car Hhadoop,1.0.3hadoop,eEa!ples,1.0.3.Car 3rep bible,input bibleF8reP RSwMR
7 hadoop 8s ,ls bibleF8reP
2leanup the lo3s
7 hadoop 8s ,r!r bibleF8rePFlo3s
7 hadoop 8s ,r!r shakespeareF8rePFlo3s
?pen Hive
7 hive
hiveN load data inpath AshakespeareF8rePA into table shakespeare9
hiveN sele#t T 8ro! shakespeare li!it 109
hiveN sele#t T 8ro! shakespeare where 8reP N 20 sort b" 8reP as# li!it 109
hiveN sele#t 8reP4 #ount<1= as 82 8ro! shakespeare 3roup b" 8reP sort b" 82 des# li!it 109
hiveN eEplain sele#t 8reP4 #ount<1= as 82 8ro! shakespeare 3roup b" 8reP sort b" 82 des# li!it 109
hiveN load data inpath 5bibleF8reP6 into table kCv9
hiveN #reate table !er3ed <word strin34 shakeF8 int4 kCvF8 int=9
hiveN insert overwrite table !er3ed sele#t s.word4 s.8reP4 k.8reP 8ro! shakespeare s Coin kCv k on
<s.word & k.word= where s.8reP N& 1 and k.8reP N& 19
hiveN sele#t T 8ro! !er3ed li!it 209
hiveN sele#t word4 shakeF84 kCvF84 <shakeF8 M kCvF8= as ss 8ro! !er3ed sort b" ss des# li!it 209
>ow "ou know9 and knowin3 is hal8 the battle
+otal Map:edu#e 2;U +i!e *pent: 1 se#onds 140 !se#
?)
the 2I-4- 123L4 --242
and 1L1@1 3-L-I I-1I1
o8 11@00 341I4 I13I4
% 23031 --I4 31--I
to 1-03- 13I21 31I14
in 10@L@ 1244I 23242
a 141@0 -0I@ 2222@
that --1L 12103 214@2
0nd @-00 12-41 20141
is ---2 1--4 1I@11
!" 112L@ 413I 1I432
"ou 12@02 2@20 1I422
he I@20 L1@2 1I3L2
his 1-1@ -3-I 1I202
not -40L 1IL1 1I000
be 1@@3 1L13 131-1
8or 130L @2@0 13I@L
with @2-4 10I@ 13341
it @1@- IL1@ 130LI
shall 32L3 L@14 130I@
+i!e taken: 1@.@11 se#onds