Nom et Prénom : EL AMINE MEHDI

Apache Hadoop is an open source, Scalable, and Fault tolerant

framework written in Java. It efficiently processes large volumes of data on
a cluster of commodity hardware. Hadoop is not only a storage system but
it is a platform for large data storage as well as processing.
In this lecture, we get a look on how Apache Hadoop works uder the
hood. So when Apache Hadoop is getting fed a huge file, the framework
divides that chunk of big data into smaller pieces and stores them across
multiple machines to be processed in parallel, so that’s why Hadoop
interconnects an army of widely-available and relatively inexpensive
machines that form a Hadoop cluster, and no matter what the size of the
file that the user feeds to Hadoop, each one of its clusters accommodates
three functional layers, Hadoop distributed file systems for data storage,
Hadoop MapReduce for processing, and Hadoop Yarn for resource
management.
Then we get a brief introduction to HDFS, a distributed file systems that
follows master/slave architecture. It consists of a single namenode and
many datanodes. In the HDFS architecture, a file is divided into one or
more blocks of 128 Mb (the size can be changed in the configurations)
and stored in separate datanodes. Datanodes are responsible for
operations such as block creation, deletion and replication according to
namenode instructions. Apart from that, they are responsible to perform
read-write operations on file systems.
Namenode acts as the master server and the central controller for HDFS. It
holds the file system metadata and maintains the file system namespace.
Namenode oversees the condition of the datanode and coordinates
access to data.