Training Document

1st Week

Learning higher level Hadoop Architecture that includes Hadoop Common, Hadoop YARN, Hadoop
Distributed File System, Hadoop MapReduce.
Learning MapReduce concepts with weather dataset application .
1.Anatomy of a MapReduce Job Run
2. Shuffle and Sort(Map and Reduce tasks)
3. MapReduce Types and Formats.
ClimateAnalyseApp: Implemented a Map program which takes raw weather data as input
and process it in to a key value pair of year and temperatures and Reduce program to
iterate over the temperature of each year and find the max value over it and DB output
program to model the data and provides a getter and setter methods to store the
enhanced result from the reduce task as year and its max temperature. And finally a job
program to configure database and run map and reduce tasks.
Learning HDFS(Hadoop Distributed Filesystem) concepts.
I. Blocks, Namenodes and Datanodes, Caching, HDFS High Availability.
II. Basic Filesystem Operations, Reading and writing Data Using the Filesystem API.
Learning YARN(Cluster Resource management system)
I. YARN Compared to MapReduce 1,Anatomy of a YARN Application Run.
II. Schedulers in YARN: the FIFO, Capacity, and Fair Schedulers.