Name

class Roll No
racHcal No
Name aE PeHtalNauigatting in Hadaop enuiranmel
Smae

1 versian
Command Name Versim
(om man USage Verslam_
example
hadoo p versim
DesconpHim Shouws the vesim af hadbop installed

mkdi
Command Namemkd
Command Usage mkdiy Kpah
example hd fs d£s-mkslirDser ldalaflair ldir L
Desciphm Iha (ommand akes the <path as anqrqumeat
and Ctae the decfay

3 1
Command Name Is
Command Usage
els
ls<path
example hdfs-dfs-1sluserdabaalr
DescompHm. This Command displayS the Cantalotof
the drechyspecified by Path>. It shows. the
Namepemiasiam_Oune SDe 4 moditratisn daka of
each enhy
Secmd example hdEs ds-Is-Roser
DescmpHm Thus Cammand behal/es 1}ke s bot displays
eies in all the Sub-dectenies yecOYs ely
LPuk
Command Namepot
Commadd Usaaeput lacalsrcyKdest2
example hdEs dfs-pothome Sample.txE/usecldutarfeidh
DesamphHm Thu's Command Copies the l e inthe local fesyst
tothe Hle lo D£S

SCopy fom LocoL

Command NamePAT Copy bomloca
CoPyhomLeco
Command Usagept lacalsc)Kdest
example hdfs dfs-Copyfpmlacallhame/Sample /usey
debePlairldar
DescipHmThis Command 5 Similar to puE Commane
BoEtheSoUYe shauld refer to local ile

6get
Command Nlame:qek
Command Usas ege STC <localdes+2
eXxample- hdfs dfs -qetluserldateLlairldlhome
pescoipim This Hadeop shell Cammand Copies the eleln HDES
identlEtea by <Src) to Aile In_local Hle systemiden-lEled
by<lotaldesE>
2nd ekampele hdfs dEs-4elmeng e luserJdalacla-ldirl| Sample.Lkt
uSerldaa £lareldor 2 Sample +xt lhomeSampleLtkt
DescmpHio: Thik HDfS Commane reeucs all eiles inthe
SoUNCe path entemtd by the User lo HDES And merges
them into one single file Coceated nthe loca file
Systemidentiled by local destioatiom.
3d ekample hadoor fs-getfacll userldada flair|dirl
kample. hadoap fs-9elacl-Rlusealdaka£larr ldr_
pescmpHm-Thi_éaede Hadoop (ommand shows the HCess
Conhol lisks af le + Þmecharies

CpyD Loca
Command Name Copyto loCa
Compand Usage Copy TDLoCal ksre> <localdesb
eg hadE dfs-copy To Local loser ldakaplarldllhume
Desaiphia:lt Is slmilcr to gek Cammand.6nly the diE"-
is that io this the desinaHin_af _Copitd Hleshoalal
efeno loc fle

8 Cat
Command Name'Cat
Command saqe CatkHle-name
eg hdEs d£s-catloserldataElair ldir l1s ample-tx
DesThis Hadoap shell Command display he Comtenks of File
on CanSole oyStdut

m
Command Name'mv
Command usag e mV <Src <dest>
eg hadfs dEs-mv|user dalaElair dl|Sample-txeluser
Ldatallalr ldi
pes.cmipHan-Th's hadoop shell Command movosthe file fom
he speciLed Soure des jnaHam wthin HoES
 ommand Name CP
Lommand Usagecpisre Kdest
eg had fs dEs-celuseaLdata flaLA Sample. #-ser
ldataflarldrL
Desi This hadoop Shell Command (opjes the fle adircay
mm Men SDUYe to dest Lo tho HDES
 tauchCareoute a Nen file or Lpdaute its timestamp
Syntakitouch LopTLON- LEILEl
eq eale_empy Eles Called filel 4 ile2
Stouch Ale Ele

CcutCon Catenate pile 4 pnnt fo stdout

Suntax!Cat [OPTION]- ELLE
eCreute ilelwith enttred Cctent
$Cat file l
Hallo
D
Cp opy fles
6yntax! Ce LoPT1ONISourre desHnahicn
eg Copies he ontents om Hlelto fle_dad caotents
of Hlel s eleuined
$ Ce fle fle2

HmV! move fle_a re name fles

syata u loPtloNJ SouYe destinsHian
eq Coeote emptyfle Caled Filel 4 ile
mv file file2
sm:Remoue Eiles 4 Digrectanes
Syntax Ym LofTIo N]--- EILEL
e belete flel_
m filel

6 mkdir Make d nectay

Suntaxmkd L6PT|oN] diecay
e orcate_dmectany Called dil
mkdr dr

rm dir Remove adireckny

syntax_rmdir oPTION]d recteny
eg (cate empt Hles clled Elel 4 File
$Ymar dr

8 Cchange dîrecteny
Suntax cd LoFT loN]dÎirckaxy
eg Change wrklnq deckenntodr
$cd d

9PLDd Pnot the present_aakln9 direcheny

SuataxPwd LofTloN]
Pnat dir f a Cment oarkin dlrckeny is dir

1o Pudte e! Used display he (ontens ofq dltctoy as

anlodented ree
$Pul tmee.
]head The head Command in linnkpnnts the 15tN
lines ofa gen Gle Cocntent
shead S a txE
2ailThe tail Command in linok pnints he last N
lin e off a qen le Comtent
tai atkE
13) we word Co ont
$we tkE

14 awk:
+ is a uey powen folioterpreted pybgrammins lanquage
ohich 3 speclaly designed fen tet_pacessing houg h
this we Can Serch COtmanlpolate fexe. tcan be
Used as q Eield ektachara baSiC Calculatar as a pate
mateher 4 E a louus fhe oser o Use Uomables n bmcm
funcHons
Vi at
Lo PUne mh
pR pimpn mb
3 03 sangli mb
awk'smnt 4-3 att
awk le'a-tx
SpliEing a line Into Helds
auk leh print 5L$4 at
TO findthe lengh of the longesline present in
the hle
ak 'iE (length/so)>max) mak- length ($o)? END

To (ounE he line in a ile

awk'END Pnnt NRS atkxE

Sedis Shoeam elterA sheam editer is 0se fo

pecteom basic tcxE fanstarmaHan an an lP Stream
+Can pecform los af OpemHm_a0 fhle
like Searchihs
Lmd 4 eplace-insetiam deleian 4 Can also be
USed fo pemfamm
Complex modiiCcion to shrems of
daLta
Replacth ar_SobsEitutnq stong sed snl/ri a tx
RRe placina the nth occurande of a pattem inaline sed snlp
otxt
3 Replacin al the CorenceAffhe
patfenninaline
Sed's/nlplg'a tkt
+ replares_all he dccLmence of In wth'p

|&l COT Command

Command-line ool tr Cottng data hom_each line ct
Hles umtinghe resulE o Sandangd oP
Name
Class Rollolo
PracHce No
Name to pratoa Pi4 (ommand

BasiC Pig Commands

fs This will list af the file in the HDfs
q t Es -ls
clean This wil clear the inieractive Gunt ShellL
gmuntcler
3 HiSten
This Coamand shows the Commands Cxecuted soter
gunE hiktey
Reading pata Assuming the dale resides in HDFS,4-we
need to Yead data to Pia
guhE College Studenks= LoAD-hdts :llocalhosE:g0oolPig-Dos
Callee-dak tkt USING PigStorge)
asld inE Exshnamc charay lastnamCICharaY phone
Charay City chararal
Pigstorageis the funchm thak loads 4 stere debe,
CLS StMlctlmal texE Ees.

S Shein Dada: Stare operakar k Usedl to Stoninq tne Proressel

oaded daje
grunt STORE College -studernt INto' hdfs!]1localhast! Joo
Pig-00E POE / sIntg Pigstarage ' ) _

6 Dump operabar! This (ammend s Osed to display the

YesoE On Scmeen + USLally helps iodebogging
 gyunt DumP College-Stodernks

7Descmbe openater 14 helps the parammer to uleo

he schema af the relatan
gunt~descahe College-stodenks,
l8-Explain This (o mmand helps fo reujeuo the ogical thysta
and map-redure ekecutian_plansS
qunt explajn Callege-Stodenti

-Tll ostate operala This qies Step-by Step eketuaHan

OE Stahemenks in Pig Commends
gent illoshate Collese-Stodens
Piq Commands
.GMp Ths lommand LLcks toods qupingdabqcort
Hhe Same key
amunt)9 bup-olaa =GROUP Colleac-StodenE by 1stname

2.CoU-RoUP!+_warks similony to the gmop operalk-The

main d HE Le Gbop4 (ogmop Opetaler s the
whule
gmue penalrUSually Used oth on re laHan
Logmu l's Used h mneLhan one relaHim

3jotoThls 1S 0Sed o Comblhe two_ome yelatians

ea ilnardeoc to pekam Selffoln le's say relaHian
CUstomer is l0aded from HDES tP Pig Commands 1otuoo
e laHm CUStomes 4 Lustoom@S
3jainThi's I's used to (ombine twoar mare relaHians
eg ln amder Fo perfanm selE-jain,letls Sayrelatian
"Costomen s loaded fom Hofs tp Pig (ommandsin
too relatms customerl 4 CUshomer
gnkCustamer8E JoINOUstomer Bid tostomev By id
jain ould be self oin, Inner-jaln o0fe-join

4Cross: This P/4 Command Calculates +he CrOsS Pmduct

ot fWo r mee relaHans
gunE Coss-daka:CROss CUStan@s, Cdens

SUnion t merqes two laHmThe CondiHm fer

meging1s that both Hhe velaHm lolumns 4 domaths
must be ldenHcal
gYunE studenk- UNLO stodentLStudent2

Ad van Ced Commands

lte This helps in £ilteing Oot the tuples ODtaf
xelatim based 60 CCatalo Condli-iong
GlLHer daba: F1lTER college-stulent By cit/-chenna

Distioct!Thk helps iothe remaleal 6fredudant

toples fom the velaHm.
unt distinck -data:DSTNCT College-Stodent

3 fereach This helesin qeneraHog daka translam.atn

based O0 ColomD dala
grunt Feveach daka: FoREACH Sthdenk-delails GENRATE B,
agecit
e:Oxdr by: This command display the esolk 1n.a Sated
d e r bas.ed onColumndale..
Ynt).dxdenbyDae.a.ORDERCall.cg.eStudent Byaje
DESC

s.LimtThis Commane geks limited na-aF tuplks hom

thevelaHm
qxunt) Limit-dataLImI student-delauls L
 Name
class Rollno!-
Prachical No!3
Name af PracHecdHiye Command Hadoop envimment

funcHia0 Hive Uey Or Commands

selecHngq_ dalabase lse dale base,

LSHaqdaabase SHo DATAB ASES
histing tahle in dB SHouD TABLES
pesmbing fmatof kable DEsRTBE (EORMATEDJEXIEANDED)
cOTeati q dalabase CREATE DATA BASE db-name
ppnq a data base_ DRoP DATABASE JL-name CCAScARE
TO yehme UR ioamaHal Seleok from-Columns fomkuble conl
TD Select all lalues SELECT Afomtabks
tD SelecE pcaticulanr Cabeg y selec-kfam.-table whenc erame
- TD Selec formultiele (mtemg seleckfom table where reeVele
ANP ec?="Vale'
fer Selecorg SpeciHc Col Selec Co-name from tablej
TD YemeuL_unizue_olP eCads Selec DisiNCE Col-name bam tabe!
fne Sciaq Selec-cal1, Col? -foom table rder ColP
SoHg backwad select colL, Colp bom taleadar by
ColR DEsc
 funcHan Hive qutY

ferCovatioa oLOS omthehableseleck Count G)tom tabe

fa4muping.alonq.wthfounlfngSelech duwner,tauniCH]too
tale Gmup By Auwntor
foselectiag max Values selec Ouwner lounLh]fombabe
Goup By Owrer
Selkc HHngom molHade taue seleck petname Comment
Jolaing fomRet Jain euent an
Leekname eVenOame)
T ron the quey hieeselecka Colbom fabla
TDYUD 9AeMy_losilenkmode hive-S-c'seect a.tolfemtabl a'
TD Select hae (mEg anisdle hive-eselech atolfum tablo
biveebe hNe not-l0gaer =0EBG
TD DSethe ioitialDaionhive-i atkialiaat Sq
TO YUn thenon-lathaie hive-LalscnipkSql
o run Hhe li'sEcommand dfs -IsDser
TO un s (bashCommand
TD setconEauaticn Uanablea Se_maprd reduretask=a?
Ta Quto CompleHam_ set hive. <TAB_
Tb display all Vonable-
stating with hiu se
TD rvert all onables reset
 Name
clas s Roll No:
PracHica No'
Practical Name Map -reduce io Hadoo enu

Map Pmqram
lmprjaa in loEXcepHiam
mg iava. Otil ShmnqTokeniner
impnt arg apache. hddoap. lo:intLontable
impa g apache- hddoop lo-longtwnkable
impat ng qpache-hadooP lo Text
mpart rq apache hadooP mapred MapRedure BaseL
imt a apache hadooe maPYtd mappec
mp rg apache hadoop maprd OuHputCollecken
Lmp Org-apache hadoap mapred Repcnter
public class oc-mapger extends Map RedureBase implements
maperk lonqlova ablejTextText Intonteahle
povate Hnal ShaHc Lnt mtahle one hew Intwntable
povate Text word =neo text(O
pLublic uold map(Langontabekey Text Vale,olp colleckor
Text InHumtahle>olp Repote reparter]thrmws
ToExcepHan 2 Shon TDkennerr tokennev neu Shina7bken/20
ine)
Lwhile (oken2eY hasmome Tokensc)S
wd Sek (tokenizer nexToken)_
olP Colltc (ud One)
 Reduce fcng am

loptJaua.lo 1toExceptHam
imptJaua DHl Shong Tkenizenj
lmpm Cmg 9pache hadaopjo-lot mtable
impn Java DHil Lteaaler
Lmpn drg qpache hadaof la:1ekt
Impar- rgpache hadoap-mapred_mapReaduce Base
imp Yg 9pache hado0p mapyredO0tpuuE Collecta
Lmpn g-apache.hadoap-mapred Reduce
lmpn org qpache. hadoop mapyd Repate
eublicclass C-Reducenr extends map Redur ense plenat
Reducenr KTeXt, Lotunatable,Tekt,lntumtable >1
leublic uaid Yeduce (Tex- key Heratr TIntumtahle >Lalves
olp Colleckerr<TekE latumkable?dlP, Repatey ePeater
fonus ToExtepHom
iot SomE0/
Lohile Values .has Nex+ O)2
Somt= Values hext).get O

output olleck (key new Intumtable(Sum))

3
wOTd Count pmaram_using map- Reduce
mpa java lo TOExceptHan
10pne arg apache. hadoop fspathj
imp_org apache. hadoop. io:Text
10aYE Ar9 apache. hadoop. mapyed Ele nputfarmat
impnt-nrg apache hadoop mamred Eileoutput fnmal
impant arq apache. hadoop mapyed Job client
impatg qpaChe.hadoofmqpred.Tex-Input rmat
impt ag apache .hadoop mapred.Text output frmat
public class wC-Ron aY
public staHc uoid main{Stnng l1 args) thnes ToExephkm
Joh Conf CmE neLw jeblamE (wc-Ronner Classl
Con F. setlabalame uvrd CountD
Conf setout eut keyclass CTex Class)
Conb seE mapperclass (LwC-mapper Class)
Conf Se-Combiner (lass(wc-Reduce class)_
LmEseElnPDE eomat(Tektoutput fnmat class)
Loof Set0LtPUHEmeLt(Tektouteutfmat class)

Loput Hle - dale tx

olP
HOfS
HadoopY
map redue

Is
6E
PYOCessi1g
Staraq e
todl
untE
Name
clasS Roll No
PYachcal No:
Name af PYrathicalmee.menhaHm df Mahaut in Hadoop
Envimen

Shepl: Crake Dala madlel ohjec

The Conshucin of Peasoo loreakianSimilanty Cass
ruirsadaka model. ajeeE.which hald aPle ha
Contalnes the USensfems a Prferencs defails Ata
pmducE Heve 1s the Sampk daa madel1E
The paa model abiee raulrs the fle objcetmhich camiaune
the path o ilp fle Crcate toe DakaMadel nbieck as sheen
Daamodel daamodel =neuy leOnkamodel(new Fleiptde)_

Shep rate seneSimilan ahjtc

LOeateUsersimimk obiecf using PeaSanlonelatian
Similm class Shauwn
Dser Similaty sim1l.ambyDcu Pensan Canrlaiin Simikaty
Cdalamad el
Step3: (eate vserNei4bar hodd dajco
Th's obiect Lomeutes neighhchaad of userlike.agien_
lser ThfNTtare toa pes ak nei hhar hoadi
Necres-NUSer Neighbachood-This class Compztes_aneighb.auh
Consiting of Hhe neareSE 0Dser fo gien Use
 step coeale Re.commender obje
Cotala LserhasedRecammender.objeckeass.all the akaue
Cxtate0bjechtoitsConshiuchoe a3 Shou
USexBased Recommende eCommender nCu GenemcL5r
Based Recommcnider (madel, neig.h baxhaud 1Similady

Shep Recommend tems a 0ser

Recommend produces to a Use using the
YCCo.mmadC) mehad afRecommende Interfale Thk
methodregure two Paramete
LisE&Recommendltem 2Ye CommMendaHan3 rlommender
Ye co.nmend (2,3)
Eay (Reco.m.men.dlHe m e co.mm.endaHams Yctomm.edatims
System0Dt pnatln(etam.menda.Hamal))
3

example progranm
impac ava lo LLle
mp JavqUHl: List.
impn Org apathe. mahoukCE tashe impl. madel
m.ad eil
eleAlepae
Imgot arg apache. Mahout.CE taste. impl
Ther shald 0See Aleihbohaod neighbaxhoo.d
mpax rq apache.
maboutcftasFe: reCommenderr USer
Nighbahonl
impa Oxg apache mahouE cf+aste
Siamilaty USersimkns
 public class Recommender2
public Static Void maia CSing argsLI13
hy
Daamalel datamodel-new fle Daa mod el(new GleldaleD
LIS.erSimllanay Usersimilnty Deuw peasan ComekHan Simiatdddhonas
Usersimlny Usexnêighbeed-ntu ThershaH OSer Neighkorhaod
(3.0Usersimilerty Ldatamade.
use-Bas ed Recammender reco.mmender she genemc dser
Bcsed Recom.mender (dalamod el usexheghbarhood, useasimtliat
sKRecom.mended ltem> retomm.endaHms=
remmender Ycro mmed CR3)_
far Recommendedlfem ecommendaHm etommed ahin
Sysem Qut ponbln (YeCommendahm

3cakch ExcepHm ell?

- Complie the pyoqram using the llouinq Cammands

javacRecommenden- ava.
java Reommendenr
E shoDld Produce he follouoin dlp
Recommend.ed lfem Litem :9 ValueS