UNIT:01

INTRODUCTION TO BIG DATA

ANALYTICS

Dr. M B Patil
Dr. M B Patil Dept of CSE N K Orchid College Solapur 1
Content
◦ Why Big Data and where did it come from?
◦ Characteristics of Big Data.
◦ Application of Big Data.
◦ Enabling Technologies for Big Data
◦ Big Data Stack
◦ Big data distribution packages

Dr. M B Patil Dept of CSE N K Orchid College Solapur 2

 Why Big Data and where did it come from?
◦ What is Data?
The quantities, characters, or symbols on which operations are performed by a computer, which may be
stored and transmitted in the form of electrical signals and recorded on magnetic, optical, or mechanical
recording media.
◦ What is Big Data?
Big Data is also data but with a huge size. Big Data is a term used to describe a collection of data that is huge in
volume and yet growing exponentially with time. In short such data is so large and complex that none of the
traditional data management tools are able to store it or process it efficiently

Dr. M B Patil Dept of CSE N K Orchid College Solapur 3

 Examples Of Big Data
The New York Stock Exchange generates about one terabyte of new trade data per day

Dr. M B Patil Dept of CSE N K Orchid College Solapur 4

 Social Media
The statistic shows that 500+terabytes of new data get ingested into the databases of social media site
Facebook, every day. This data is mainly generated in terms of photo and video uploads, message
exchanges, putting comments etc

Dr. M B Patil Dept of CSE N K Orchid College Solapur 5

Tabular Representation of various Memory Sizes

Dr. M B Patil Dept of CSE N K Orchid College Solapur 6

Types of Data
1. Structured.

2. Unstructured.

3. Semi structured.

Dr. M B Patil Dept of CSE N K Orchid College Solapur 7

Semi-structured
• Semi-structured data can contain both the forms of data.
• We can see semi-structured data as a structured in form but it is actually not defined with e.g. a
table definition in relational DBMS.
• Example of semi-structured data is a data represented in an XML file.
• Examples Of Semi-structured Data Personal data stored in an XML file- Prashant RaoMale35
Seema R.Female41 Satish ManeMale29 Subrato RoyMale26 Jeremiah J.Male35</rec

Dr. M B Patil Dept of CSE N K Orchid College Solapur 8

Characteristics Of Big Data
◦ The following are known as “Big Data Characteristics”.

◦ 1. Volume
◦ Volume means “How much Data is generated”. Now-a-days,

◦ 2. Velocity
◦ Velocity means “How fast produce Data”.

◦ 3. Variety

◦ Variety means “Different forms of Data”.

◦ 4. Veracity

◦ Veracity means “The Quality or Correctness or Accuracy of Captured Data”

Dr. M B Patil Dept of CSE N K Orchid College Solapur 9

Big Data challenges
◦ Insufficient understanding and acceptance of Big data.

◦ Confusion while Big data tool selection.

◦ Big loads of Money.

◦ Data Integration.

◦ Data security.

◦ Other Challenges.

Dr. M B Patil Dept of CSE N K Orchid College Solapur 10

Applications of Big Data
◦ Business Intelligence and Decision Making
◦ Healthcare
◦ Finance
◦ Marketing and Customer Insights
◦ Supply Chain Optimization.
◦ Energy Management.
◦ Smart Cities and Urban Planning
◦ Manufacturing and Industry 4.0
◦ Agriculture
◦ Transportation and Logistics
◦ Media and Entertainment
◦ Environmental Monitoring
◦ Research and Development
◦ Government and Public Services

Dr. M B Patil Dept of CSE N K Orchid College Solapur 11

Enabling Technology of Big Data:
◦ The Key technology that enable Big data are
1. Apache Hadoop
2. No SQL.
3. Data Warehousing Solutions
4. Machine Learning and AI
5. Data Streaming and Real-Time Analytics :Apache Kafka and Apache Flink
6. Data Visualization Tools
7. Cloud Computing
8. Data Pre-processing Tools
9. Data Security and Privacy Tools

Dr. M B Patil Dept of CSE N K Orchid College Solapur 12