BIG DATA

What is big data and analytics

this is a collection of data that is growing exponentially with time

Big data is data that contains greater variety arriving in increasing volumes
and with ever-higher velocity.

Big data analytics examines large and different types of data to uncover
hidden patterns correlations and other insights
Types of big data

Structured unstructured
data that can be stored accessed data with unknown form an example of
and processed in form of fixed unstructured data is a data source containing
format eg a database a combination of simple text images and
videos .

Semi structured
contains both structured and unstructured .eg
data represented in XML
 Characteristics of big data
Volume
the size of data plays a crucial role in
determining the value of data. For
data to be termed as big data it
depends on the volume of the data
variety
this refers to the nature of data both
structured and unstructured. Data is
in form of emails, photos, videos,
monitoring, devices, pdfs, audio etc.
velocity
refers to the speed of generation of
data .how fast the data is generated
and processed to meet the demands
determines the real potential in the
data.
variability
refers to the inconsistency which can
be shown by the data at times thus
hampering the process Despite being
red, Mars is actually a cold place
 Big data use cases
1.Product development-Companies like Netflix and Procter & Gamble use big data to anticipate customer demand. They build
predictive models for new products and services by classifying key attributes of past and current products or services and
modeling the relationship between those attributes and the commercial success of the offerings.

2.Predictive maintenance-Factors that can predict mechanical failures may be deeply buried in structured data, such as the
year, make, and model of equipment, as well as in unstructured data that covers millions of log entries, sensor data, error
messages, and engine temperature. By analyzing these indications of potential issues before the problems happen,
organizations can deploy maintenance more cost effectively and maximize parts and equipment uptime.

3.Customer experience- Big data enables you to gather data from social media, web visits, call logs, and other sources to
improve the interaction experience and maximize the value delivered thus delivering personalized offers.

4.Fraud and compliance-Big data helps you identify patterns in data that indicate fraud and aggregate large volumes of
information to make regulatory reporting much faster.

5.Machine learning-We are now able to teach machines instead of program them. The availability of big data to train machine
learning models makes that possible.
 Challenges facing big data

Big data Time consuming Changing rapidly

Organizations still struggle Data scientists spend 50 Keeping up with big
to keep pace with their to 80 percent of their data technology is an
data and find ways to time curating and ongoing
effectively store it.data preparing data before it challenge.Apache
volumes are doubling in can actually be used. Hadoop and Apache
size about every two spark
years
Technologies used

Frameworks that are being used to manage big data are :

1.Apache Hadoop

2.Apache spark

3.Apache Flink

4.Apache storm

5.Samza