YARN

 “I’m sure, the highest

capacity of storage device,
will not enough to record
all our stories; because,
everytime with you is very
valuable data.” – Aditia
Rinaldi – Famous Author
YARN
YARN
 Next generation
architecture of Hadoop (2.0)
 Operating System for
Hadoop
 Supports batch processing,
interactive queries, Stream
Processing and others
YARN
Hadoop 1.0
MapReduce

HDFS

Hadoop 2.0
Tez (interactive Strom (Stream
MapReduce Others
queries) Processing)

YARN

HDFS
YARN – Key Components
 Resource Manager
 Scheduler
 Applications Manager (AsM)
 Application Master (AM)
 Node Manager
 Containers
YARN – Resource Manager
 RM – Manages assignment of
resources to applications (jobs)
 Consists of Scheduler – a service that
manages the resource scheduling in
the cluster
 Applications Manager (AsM) –
Manages running applications
masters in the cluster. Responsible for
starting, monitoring and restarting on
different nodes in case of failures
YARN – Application Master
 AM – Manages the lifecycle
of a job (application).
Requests for resources from
RM
 Works with NMs to execute
and monitor tasks
YARN – Node Manager
 NM is the node framework
agent (per node)
 Manages the containers,
monitors their resource
usage
 Reports to the Resource
Manager.
YARN – Node Manager
YARN – Containers
 Container – It’s a resource bundle
allocated by RM to a datanode
 Consists of memory, CPU and
network
 Provides privilege to use resources
in a machine to perform a task
 Each node can have multiple
containers based on resource
allocation by RM
YARN – Node Manager
 NM is the node framework
agent (per node)
 Manages the containers,
monitors their resource
usage
 Reports to the Resource
Manager.