Big Data Analytics

Introduction to Big Data Analytics
2
EMC PROVEN PROFESSIONAL
Copyright © 2012 EMC Corporation. All Rights Reserved. Module 1: Introduction to BDA 1
Introduction to Big Data Analytics
Your Thoughts?
What is Big Data?
What makes data, “Big” Data?
2
Big Data Defined
• “Big Data” is data whose scale, distribution, diversity,
and/or timeliness require the use of new technical
architectures and analytics to enable insights that unlock
new sources of business value.
 Requires new data architectures, analytic sandboxes
 New tools
 New analytical methods
 Integrating multiple skills into new role of data scientist
• Organizations are deriving business benefit from analyzing

ever larger and more complex data sets that increasingly
require real-time or near-real time capabilities
Source: McKinsey May 2011 article Big Data: The next frontier for innovation, competition, and productivity
2
Copyright © 2011
2012 EMC Corporation. All Rights Reserved. Module 1: Introduction to BDA
Key Characteristics of Big Data
1. Data Volume
 44x increase from 2010 to 2020
(1.2zettabytes to 35.2zb)
2. Processing Complexity
 Changing data structures
 Use cases warranting additional transformations and
analytical techniques
3. Data Structure
 Greater variety of data structures to mine and analyze
2
Copyright © 2011
2012 EMC Corporation. All Rights Reserved. Module 1: Introduction to BDA
Big Data Characteristics:
S Data Structures
Data Growth e is Increasingly Unstructured
“ m
Q i • Data containing a defined data type, format, structure
u -
• Example: Transaction data and OLAP
a S
s t • Textual data files with a discernable pattern,
i r
More Structured
enabling parsing
” u • Example: XML data files that are self

U c describing and defined by an xml schema
n S t • Textual data with erratic data formats, can

be formatted with effort, tools, and time
s t u
t r r • Example: Web clickstream data that
may contain some inconsistencies in data
r u e values and formats
• Data that has no inherent
u c d structure and is usually stored
as different types of files.
c t
t u • Example: Text documents,
PDFs, images and video
u r
r ePROFESSIONAL
2
EMC PROVEN
e2012 EMC Corporation. All Rights Reserved.

Copyright © Module 1: Introduction to BDA 5
Four Main Types of Data Structures
Structured Data Quasi-Structured Data
Semi-Structured Data
View  Source
http://www.google.com/
#hl=en&sugexp=kjrmc&cp=8&gs_id=2m&xhr=t&q=data+scientist&pq=big+data&pf=p&sclien
t=psyb&source=hp&pbx=1&oq=data+sci&aq=0&aqi=g4&aql=f&gs_sm=&gs_upl=&bav=on.2,
or.r_gc.r_pw.,cf.osb&fp=d566e0fbd09c8604&biw=1382&bih=651
Unstructured Data
The Red Wheelbarrow, by
William Carlos Williams
2
Data Repositories, An Analyst Perspective
Data Islands Data Warehouses Analytic Sandbox
“Spreadmarts”
Centralized data containers Data assets gathered from multiple
Isolated data marts in a purpose-built space sources and technologies for analysis
• Spreadsheets and low- • Supports BI and reporting, but • Enables high performance analytics
volume DB‘s for restricts robust analyses using in-db processing
recordkeeping • Analyst dependent on IT & • Reduces costs associated with data
• Analyst dependent on DBAs for data access and replication into "shadow" file
data extracts schema changes systems
• Analysts must spend significant • “Analyst-owned” rather than “DBA
time to get extracts from owned”
multiple sources
2
Introduction to Big Data Analytics: Mini-Case Study
Yoyodyne Bank Scenario
• Evolving from small community bank to a global bank
• Needs to move away from its legacy mainframes to an environment that
supports more robust analytics
• Growing through mergers and acquisitions
• Subject to many new regulatory requirements
• Increasing customer base and increased product offerings
Your Thoughts?
Discussion Questions
1. Discuss how the bank’s data would change under these circumstances.
2. How are their needs changing with these business changes?
3. What do you need to consider from an analyst point of view? What are
some things to consider implementing as the bank grows?
2
Business Drivers for Analytics
Current Business Problems Provide Opportunities for Organizations to
Become More Analytical & Data Driven
Driver Examples
1
Desire to optimize business
Sales, pricing, profitability, efficiency
operations
2
Desire to identify business risk Customer churn, fraud, default
3
Predict new business opportunities Upsell, cross-sell, best new customer prospects
4
Comply with laws or regulatory
Anti-Money Laundering, Fair Lending, Basel II
requirements
2
Copyright © 2011
2012 EMC Corporation. All Rights Reserved. Module 1: Introduction to BDA 10
Analytical Approaches for Meeting Business Drivers
Business Intelligence vs. Data Science
Predictive Analytics & Data Mining
(Data Science)
Typical • Optimization, predictive modeling,
Techniques & forecasting, statistical analysis
Data Types • Structured/unstructured data, many types
of sources, very large data sets
High Common • What if…..?

Questions • What’s the optimal scenario for our
business ?
• What will happen next? What if these
trends continue? Why is this happening?
Data
Science Business Intelligence
BUSINESS Typical • Standard and ad hoc reporting,
Techniques & dashboards, alerts, queries, details on
VALUE Data Types demand
Business • Structured data, traditional sources,
Intelligence manageable data sets
Common • What happened last quarter?
Questions • How many did we sell?
• Where is the problem? In which
situations?
Low
Past TIME Future
2
A Typical Analytical Architecture
1 Data
Sources
Non-Agile Models
2 Departmental
“Spread
Marts”
Warehouse
Enterprise 4
Departmental Applications
Warehouse
3 Prioritized
Operational
Processes
Static schemas
accrete over time Reporting Siloed
Analytics
Non-Prioritized Data Provisioning
Errant data & marts
2
Copyright © 2011
Implications of Typical Architecture for Data Science
• High-value data is hard to reach and leverage

• Predictive analytics & data mining activities are last
in line for data
 Queued after prioritized operational processes
• Data is moving in batches from EDW to local Slow
“time-to-insight”
analytical tools
&
 In-memory analytics (such as R, SAS, SPSS, Excel)
reduced
 Sampling can skew model accuracy business impact
• Isolated, ad hoc analytic projects, rather than
centrally-managed harnessing of analytics
 Non-standardized initiatives
 Frequently, not aligned with corporate business goals
2
Opportunities for a New Approach to Analytics
New Applications Driving Data Volume
MEASURED IN MEASURED IN WILL BE MEASURED IN

LARGE TERABYTES PETABYTES EXABYTES
1TB = 1,000GB 1PB = 1,000TB 1EB = 1,000PB
VOLUME OF INFORMATION
SMALL
1990’s 2000’s 2010’s

(RDBMS & DATA (CONTENT & DIGITAL ASSET (NO-SQL & KEY/VALUE)
WAREHOUSE) MANAGEMENT)
2
Opportunities for a New Approach to Analytics
Big Data Ecosystem
1
Data
Devices
Individual
Analytic Medical Information

Services Brokers Advertising Marketers Employers
Law
Enforcement
Government Internet
2
Data
Websites
3
Collectors Data
Aggregators
Data
Users/Buyers
Catalog
4 Co-Ops
Phone/TV Retail
Media
Private
Media Credit List Investigators
Archives Bureaus Financial Brokers Delivery /Lawyers
Banks Service
Government
2
Copyright © 2011
Considerations for Big Data Analytics
Criteria for Big Data Projects New Analytic Architecture
Analytic Sandbox
Data assets gathered from multiple sources
1. Speed of decision making and technologies for analysis
2. Throughput
• Enables high performance analytics
using in-db processing
3. Analysis flexibility • Reduces costs associated with data
replication into "shadow" file
systems
• “Analyst-owned” rather than “DBA
owned”
2
State of the Practice in Analytics: Mini-Case Study
Big Data Enabled Loan Processing at Yoyodyne
Traditional Big Data Enabled

Underwriting Underwriting Your Thoughts?
Risk Level Risk Level
Underwriting Risk
e t al
om on en y ing ais
c
In ati ym o r c or tory pr
ic plo is
t S is Ap
e rif Em H edit d H
V Cr An
TRADITIONAL DATA LEVERAGED BIG DATA LEVERAGED
2
Copyright © 2011
Three Key Roles of the New Data Ecosystem
Data Scientists Role Role Description

Projected U.S. People with advanced training in
talent gap: Deep Analytical quantitative disciplines, such as
140,000 to Talent mathematics, statistics, and machine
190,000 learning.
People with a basic knowledge of statistics
Data Savvy
and/or machine learning, who can define
Analysts & Data Professionals
key questions that can be answered using
Savvy Managers advanced analytics
Projected U.S. People providing technical expertise to
talent gap: 1.5 Technology & Data support analytical projects. Skills sets
million Enablers including computer programming and
database administration
Note: Figures above reflect a projected talent gap in US in 2018, as shown in McKinsey May 2011 article Big Data: The next frontier for innovation,
competition, and productivity
2
Copyright © 2011
Roles Needed for Analytical Projects
Data Scientist Key Activities
Data Scientists
Key Activities Data Data Bl LOB
Engineers Analyst Analyst
• Reframe business User
challenges as analytics
challenges Analytic Productivity Platform
• Design, implement and

Tools & Services
deploy statistical models
and data mining Data
Data Access & Query Platform
techniques on big data Admin
• Create insights that lead

to actionable
recommendations Cloud Infrastructure
2
Copyright © 2011
2012 EMC Corporation. All Rights Reserved. Module 1: Intro duction to BDA 22
Profile of a Data Scientist
Quantitative
Curious &
Technical
Creative
Skeptical Communicative
& Collaborative
2
Big Data Analytics: Industry Examples
1
Health Care
• Reducing Cost of Care Medical
2 Public Services Government Internet
• Preventing Pandemics
3 Life Sciences Data
Collectors
• Genomic Mapping
4 IT Infrastructure
• Unstructured Data Analysis
Phone/TV Retail
5 Online Services
Financial
• Social Media for Professionals
2
1
Big Data Analytics: Healthcare
• Poor police response and problems with medical care, triggered

Situation by shooting of a Rutgers student
• The event drove local doctor to map crime data and examine
local health care
• Dr. Jeffrey Brenner generated his own crime maps from medical
Use of Big Data billing records of 3 hospitals
• City hospitals & ER’s provided expensive care, low quality care
• Reduced hospital costs by 56% by realizing that 80% of city’s
Key medical costs came from 13% of its residents, mainly low-
Outcomes income or elderly
• Now offers preventative care over the phone or through home
visits
2
2
Big Data Analytics: Public Services
• Threat of global pandemics has increased exponentially

Situation
• Pandemics spreads at faster rates, more resistant to antibiotics
• Created a network of viral listening posts

• Combines data from viral discovery in the field, research in
Use of Big Data disease hotspots, and social media trends
• Using Big Data to make accurate predications on spread of new
pandemics
• Identified a fifth form of human malaria, including its origin
Key • Identified why efforts failed to control swine flu

Outcomes
• Proposing more proactive approaches to preventing outbreaks
2
3
Big Data Analytics: Life Sciences
Situation • Broad Institute (MIT & Harvard) mapping the Human Genome
• In 13 yrs, mapped 3 billion genetic base pairs; 8 petabytes
Use of Big Data

• Developed 30+ software packages, now shared publicly, along
with the genomic data
• Using genetic mappings to identify cellular mutations causing

Key cancer and other serious diseases
Outcomes
• Innovating how genomic research informs new pharmaceutical
drugs
2
4
Big Data Analytics: IT Infrastructure
• Explosion of unstructured data required new technology to

Situation
analyze quickly, and efficiently
• Doug Cutting created Hadoop to divide large processing tasks

into smaller tasks across many computers
Use of Big Data
• Analyzes social media data generated by hundreds of
thousands of users
• New York Times used Hadoop to transform its entire public

Key archive, from 1851 to 1922, into 11 million PDF files in 24 hrs
Outcomes
• Applications range from social media, sentiment analysis,
wartime chatter, natural language processing
2
5
Big Data Analytics: Online Services
Situation • Opportunity to create social media space for professionals
• Collects and analyzes data from over 100 million users

Use of Big Data
• Adding 1 million new users per week
• LinkedIn Skills, InMaps, Job Recommendations, Recruiting

Key
Outcomes • Established a diverse data scientist group, as founder believes
this is the start of Big Data revolution
2

Big Data Analytics

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Big Data Analytics

Uploaded by

Copyright:

Available Formats

Introduction to Big Data Analytics

What is Big Data?

What makes data, “Big” Data?

• Organizations are deriving business benefit from analyzing

” u • Example: XML data files that are self

n S t • Textual data with erratic data formats, can

e2012 EMC Corporation. All Rights Reserved.

High Common • What if…..?

Past TIME Future

Non-Prioritized Data Provisioning

Errant data & marts

• High-value data is hard to reach and leverage

MEASURED IN MEASURED IN WILL BE MEASURED IN

1990’s 2000’s 2010’s

Analytic Medical Information

Traditional Big Data Enabled

TRADITIONAL DATA LEVERAGED BIG DATA LEVERAGED

Data Scientists Role Role Description

• Design, implement and

• Create insights that lead

2 Public Services Government Internet

• Poor police response and problems with medical care, triggered

• Threat of global pandemics has increased exponentially

• Created a network of viral listening posts

Key • Identified why efforts failed to control swine flu

• In 13 yrs, mapped 3 billion genetic base pairs; 8 petabytes

Use of Big Data

• Using genetic mappings to identify cellular mutations causing

• Explosion of unstructured data required new technology to

• Doug Cutting created Hadoop to divide large processing tasks

• New York Times used Hadoop to transform its entire public

Situation • Opportunity to create social media space for professionals

• Collects and analyzes data from over 100 million users

• LinkedIn Skills, InMaps, Job Recommendations, Recruiting

You might also like