Big Data Course (First Term 2023) Final Project

❖ Every group must choose one project, every group consists of (5 : 8) students
❖ Every Project is available for 9 groups
❖ Project grads 10 points.

Notes : register more than one idea ,if first choice is not available , you will execute the available idea
for you

Project 1:

Re-write SQL Join algorithms (inner - outer joins) using map-reduce and compare with Hive execution
time.

Project 2:

Develop marketing web-based application (CRUD operations) that uses hive as Back-end Database
and do full text search.

Project 3:

Create Desktop GUI for HADOOP frame work enabling user to:
▪ Start and Stop Hadoop.
▪ Load MapReduce JAR File.
▪ Input data files.
▪ Determine Output Data files.
▪ Copy result data into local file.

Project 4:

Create Desktop GUI for HIVE frame work enabling user to:
▪ Create Schema DDL
▪ Load schema Data tables DDL
▪ Queering schema by SQL queries DML
▪ Store query result into local file system

Project 5:

Use Hadoop map reduce for matrix some operation :

▪ write map reduce program to load two matrices as input each matrix size 1000*1000
▪ calculate matrix multiplication
▪ calculate matrix addition
▪ calculate matrix subtraction
▪ calculate matrix division
▪ calculate matrix mod division
▪ copy output into local file system
▪ do multiplication process at every node concurrently
Project 6:

- run word count sample within Hadoop samples

-re-write word count to sort file words alphabetically
- find the word that has large frequency
- find the word that has small frequency
- find the words that have the same frequency

Project 7:

Using what you learn in big data to develop a distributed search engine to crawel 100,000 web pages
f from the news websites and index them using PageRank technique .all work must run in a
distributed fashion on Spark/Hadoop cluster.
Finally , provide search services on the craweled pages.

Project 8:

- You have number of documents , do inverted index on them and search engine
- do OR , AND operation for examples
Hello appears in : Doc1 , Doc3
World appears in : Doc1 , Doc2 ,
Hello And World : result is : Doc1
Hello OR World : result is : Doc1 , Doc2 ,Doc3