Big Data Analytics

We are Big Data
The future of the information society
prof. dr. Sander Klous

Big Data Ecosystems in Business and Society
University of Amsterdam
Big Data Analytics
KPMG Advisory
klous.sander@kpmg.nl
@sanderklous
http://nl.linkedin.com/in/sanderklous
Extreme expectations
https://www.youtube.com/watch?v=2vXyx_qG6mQ
2014 KPMG Advisory N.V., registered with the trade register in the Netherlands under number 33263682, is a subsidiary of KPMG Europe LLP and a member firm of the KPMG network of
independent member firms affiliated with KPMG International Cooperative (KPMG International), a Swiss entity. All rights reserved. Printed in the Netherlands. The KPMG name, logo and
cutting through complexity are registered trademarks or trademarks of KPMG International.
Big, bigger, biggest
Whhy
W
y ddoo p
a
p
a
h
haavvee m rrttiicclleess
maasss?
s?
60 TB/s
01
010
1101
11001
110110
0111000
01001110
110111001
1111010001
010110000101
W 300 MB/s
0 Z
Nobel prize 2013 to Higgs and Englert

9 pages with authors
(3 authors @ KPMG)
The new information society
Shared value with Big Data
http://www.visaeurope.com/en/newsroom/news
/articles/2010/validsoft_fraud_solution.aspx
Torrents in the music industry,

Bitcoins in the financial sector
When is a trend going to

disrupt your business?
10
Jobless recovery in a developing information society
Accountants: 95%
http://www.futuretech.ox.ac.uk/sites/futuretech.ox.ac.uk/files/The_Future_of_Employment_OMS_Working_Paper_1.pdf
11
The dark side of Big Data
Big Brother is watching you
13
Big Brother is watching you
14
The Target case: life changing events
Andrew Pole had just started

working as a statistician for
Target in 2002
http://www.nytimes.com/2012/02/19/mag
azine/shopping-habits.html
15
Lack of transparency: LG and TP-Link (Philips) example
http://doctorbeet.blogspot.nl/2013/11/lg-smart-tvs-logging-usb-filenames-and.html
http://www.cbpweb.nl/downloads_pb/pb_20130822-persoonsgegevens-smart-tv.pdf
16
Big Brother?
Thats us!
If something is for free,

youre not the client,
youre the product!
Fundamental questions about

Big Data
The profile of the data scientist

Apples and Pears
Suppose we have two jars with apples and pears.
Our prior knowledge is that:
Jar A contains 10 apples and 30 pears
Jar B contains 20 of each
Fred picks a jar, without further evidence there is a 50% chance

this is jar A (or B).
Jar A
Jar B
Fred pulls out a pear. The new probability that Fred picked bowl A
is 0.75 x 0.5 / ( 0.75 x 0.5 + 0.5 x 0.5 ) = 0.6
P(Hn|E) =
P(E|Hn)P(Hn)
Sum1N (P(E|Hn))
Simpsons paradox
Although in this simple example Bayesian statistics appears to be straight forward, many subtleties
arise in analysis. One of the more famous pitfalls is called Simpsons paradox:
http://en.wikipedia.org/wiki/Simpson's_paradox
Question:
Do you have a higher chance of survival when falling overboard with or without a life jacket?
Correlation or Causality?
19
Mechanism Design (Nobel prize 2007)

Sandy
Pentland
(MIT)
Living lab
Trento
Tipping points
Understanding large scale

human mobility (UvA project)
20
Rules and regulations in a do it yourself society

Our behavior is determined
by the systems we use
21
Big Data and Society, an ethical perspective
22
A Practical Guide to Big Data
Boundary conditions
24
KPMG Analytics & Visualization Environment

(KAVE)
The Overview
Horizontally Scalable
Open Source
Configurable
Modular
Secure
The Implementation
Remote (or local) hosting
Dedicated hardware
Virtualized system
Secure internal network
25
Sharing insights
Two countries /
clients / departments
need to combine and
share insights
securely
No single attractive target for hackers

26
Governance in a data driven organization

D&A Board
Operational units
Executive
Committee
The D&A board is

accountable for the success
of D&A projects. The board
consists of multiple
delegates from the
operational and D&A units.
The board is headed by an
executive that reports
directly to the CEO.
From ideas to projects
HR
Finance
Marketing
Business
D&A Board
Plan of
approach
Project
Groups
Processing
Vision
Data
Acquisition
As the D&A units mature, the

division of data-oriented
functionalities into
operational silos decreases.
From data to ideas

The D&A services are
managed by the board. Ideas
are collected from all sources
within the organization (not
only from within the D&A
Services groups).
The D&A units facilitate the

operational units to capitalize
on data through data driven
business models.
Becoming data driven
Real time analysis
The board is responsible for

the vision, plan of approach
and formation of project
groups. Project groups
consist of talent from within
the organization to work out
their own initiatives and
ideas. Participation in these
projects is by invitation only
and considered a career
accelerator. Project
development occurs in short
cycles.
Aggregating data
Idea generation
Understanding D&A
services
The D&A units provide a
variety of solutions to the
operational units.
Operational units will
continue to run and be
responsible for their own
activities and associated
data.
D&A units
27
Agile project management
Client
Client
cases
cases
Preparation for and creating client cases using a
Big Data opportunity event
Proof of Concept for selected cases and

hand over
Transformation of the solution

and business model
Continuous interaction to orchestrate Big

Data strategy process and if necessary
adjust business case directions.
2 weeks
10 weeks
to be decided
Coordinating
Coordinating the
the project
project using
using aa phased
phased and
and agile
agile approach
approach
Envisioning
Preparing
Developing
Reviewing
uncovering client
cases using a Big
Data opportunities
selected client cases

for Proof of Concepts
the Proof of Concepts

of the selected client
cases
evaluating the results

and determining the
way forward
Part
Part of
of this
this
proposal
proposal
Additional
Additional
activities
activities
28
Applying Big Data

Start Small
Behavioral patterns:
Dynamic online profiling
Background
Background
Conference venue
Knowing
Knowing your
your online
online web
web site
site visitors
visitors is
is advantageous
advantageous since
since itit enables
enables
increasing
the
effectiveness
of
online
offerings:
by
targeting
increasing the effectiveness of online offerings: by targeting more
more relevant
relevant
content,
content, advertisements
advertisements and
and e-commerce
e-commerce products
products to
to audiences
audiences with
with interests
interests
that
match,
conversion
rates
and
thus
revenue
will
increase.
Data-driven
that match, conversion rates and thus revenue will increase. Data-driven
insights
insights based
based on
on web
web access
access logs
logs help
help to
to create
create this
this understanding
understanding in
in
customer
customer behavior.
behavior.
Requirements
Requirements // Challenges
Challenges
Processing
Processing and
and combining
combining large
large volumes
volumes of
of data
data from
from two
two separate
separate silos
silos
with
with little,
little, but
but valuable,
valuable, overlap:
overlap: identified
identified customer
customer (subscription)
(subscription) data
data
versus
versus unidentified
unidentified customer
customer (online)
(online) data
data
Automatized
Automatized identification
identification of
of meaningful
meaningful clusters
clusters of
of interests
interests in
in data
data
Resources
Resources
Data
Data sources:
sources: web
web access
access logs
logs and
and online
online subscription
subscription data
data
Duration
Duration (develop
(develop phase):
phase): 88 16
16 weeks
weeks
Solution
Solution
KPMG
KPMG Analytics
Analytics && Visualization
Visualization Environment
Environment (KAVE)
(KAVE)cluster
cluster used
used as
as aa
horizontally
horizontally scalable
scalable storage
storage and
and computing
computing resource
resource
User
User profile
profile A
A
Searched
Searched
:: BMW
BMW
Read:
Read:
car.com
car.com
User
User profile
profile B
B
Searche
d:
Searche
d:
Nike
Read:
Read:
olympics
olympics
Nike
Avg. CTR:
0.063
Analysis
Analysis algorithms
algorithms and
and machine
machine learning
learning tools
tools were
were developed
developed to
to apply
apply aa
dynamic
segmentation
model
and
identify
clusters
of
customers
interests
dynamic segmentation model and identify clusters of customers interests
CTR SWAP: 0.079

(+25%)
Benefits
Benefits
AApilot
pilot testing
testing such
such Decision
Decision Engine
Engineshows
shows aa >25%
>25% increase
increase in
in conversion
conversion
rate
rate (CTR)
(CTR) which
which corresponds
corresponds to
to 0,3
0,3 1,7m
1,7m euros
euros of
of potential
potential gain
gain in
in online
online
advertisement
revenue
for
this
specific
case.
advertisement revenue for this specific case.
30
Crowd & Mobility Management at a large Dutch Telco
Background
Background
Conference venue
An
An infrastructure
infrastructure company
company wants
wants to
to use
use aa sensor
sensor network
network that
that covers
covers the
the whole
whole
of
the
Netherlands.
The
network
generates
terabytes
of
data
that
can
provide
of the Netherlands. The network generates terabytes of data that can provide
more
more insight
insight in
in the
the transport
transport flows
flows in
in addition
addition to
to their
their own
own transport
transport data.
data. Based
Based
on
on these
these combined
combined data,
data, the
the infrastructure
infrastructure company
company can
can make
make more
more efficient
efficient
logistics
logistics planning
planning and
and adequately
adequately respond
respond to
to disturbances.
disturbances.
Requirements
Challenges
Linking
Linking external
external sensor
sensor network
network data
data to
to internal
internal data
data sources
sources
Recognizing
Recognizing trends
trends in
in historical
historical data
data && real-time
real-time monitoring
monitoring of
of network
network data
data
Clear
Clear and
and simple
simple overview
overview of
of large
large volumes
volumes of
of complex
complex data
data
Resources
Resources
Data
Data sources:
sources: infrastructure
infrastructure and
and network
network data,
data, sensor
sensor network
network data
data
Duration
Duration (develop
(develop phase):
phase): 66 12
12 weeks
weeks
Solution
Solution
Combining
Combining different
different data
data sources
sources into
into aa data
data institute
institute
Data
Data storage
storage by
by Teradata
Teradata unified
unified data
data architecture
architecture including
including Hadoop
Hadoop cluster
cluster
Distributed
Distributed analysis
analysis using
using Aster
Aster && MapReduce
MapReduce algorithms
algorithms
Storm
Storm implementation
implementation for
for real-time
real-time data
data processing
processing
Benefits
Benefits
Reduction
Reduction of
of IT
IT costs
costs by
by outsourcing
outsourcing to
to specialized
specialized data
data institute
institute
Cost
Cost savings
savings by
by acting
acting on
on trends
trends and
and disruptions
disruptions adequately
adequately
Enriched
Enriched data
data as
as aa product
product for
for other
other parties
parties in
in the
the transport
transport ecosystem
ecosystem
31
Shopping behavior at a large international furniture retailer
Background
Background
Conference venue
Tracking
Tracking the
the customer
customer behaviour
behaviour is
is important
important information
information which
which can
can be
be
gathered
using
several
types
of
sources.
Wi-Fi
signals
can
for
instance
gathered using several types of sources. Wi-Fi signals can for instance be
be used
used
to
detect
shopping
patterns
from
customers,
but
one
can
think
of
many
different
to detect shopping patterns from customers, but one can think of many different
data
data sources
sources and
and techniques,
techniques, like
like security
security cameras,
cameras, iBeacon
iBeacon or
or Bluetooth.
Bluetooth.
Requirements
Challenges
Real-time
Real-time collection
collection and
and analysis
analysis Wi-Fi
Wi-Fi device
device signals.
signals.
Analysing
Analysing the
the customer
customer behaviour
behaviour and
and plotting
plotting patterns,
patterns, graphs
graphs and
and trend
trend
lines.
lines.
Storing
Storing sales
sales data,
data, data
data from
from RFID
RFID chips
chips or
or motion
motion sensors
sensors for
for extra
extra
information
about
certain
products.
information about certain products.
Storing
Storing other
other relevant
relevant (e.g.
(e.g. Social
Social Media)
Media) information.
information.
Resources
Resources
Data
Data sources:
sources: Wi-Fi
Wi-Fi device
device signals.
signals. Additionally:
Additionally: sales
sales data,
data, RFID
RFID data,
data,
social
social media
media data,
data, camera
camera footage,
footage, iBeacon.
iBeacon.
Duration
Duration (develop
(develop phase):
phase): 88 16
16 weeks
weeks
Solution
Solution
The
The analysis
analysis of
of any
any of
of these
these separate
separate data
data streams
streams will
will provide
provide vital
vital information
information
about
the
shopping
behaviour
of
customers.
Combining
all
the
data
creates
about the shopping behaviour of customers. Combining all the data creates
even
even richer
richer information.
information.
Benefits
Benefits
The
The goal
goal is
is to
to define
define which
which variables
variables affect
affect aa purchase
purchase and
and make
make changes
changes
accordingly.
accordingly. For
For example,
example, aa store
store might
might deploy
deploy more
more salespeople,
salespeople, alter
alter displays
displays
or
put
out
red
blouses
instead
of
blue,
according
to
a
story
on
Bloomberg.
or put out red blouses instead of blue, according to a story on Bloomberg.
32
Asset management and planning at a University Medical Center

Monte
Monte Carlo
Carlo simulation
simulation to
to analyze
analyze operational
operational efficiency
efficiency of
of Operation
Operation Room
Room
planning
planning at
at aa large
large academic
academic hospital.
hospital. AAnew
new blue
blue print
print for
for OR
OR planning
planning was
was
created
created and
and needed
needed to
to be
be evaluated
evaluated on
on performance,
performance, expected
expected production
production
numbers
numbers and
and where
where issues
issues might
might arise.
arise.
Requirements
Challenges
Reality
Background
Background
Handling
Handling poor
poor data
data quality
quality
Translate
Translate highly
highly flexible
flexible human
human decision-making
decision-making to
to rules
rules in
in model
model
Resources
Resources
Data
Data sources:
sources: blueprint,
blueprint, historical
historical OR
OR usage
usage data
data
Duration
Duration (develop
(develop phase):
phase): 10
10 16
16 weeks
weeks
Simulation
Client
Client involvement
involvement (expert
(expert knowledge)
knowledge) was
was needed
needed throughout
throughout the
the process
process
in
in order
order to
to create
create an
an expert
expert system
system for
for planning
planning
Solution
Solution
Built
Built aa Monte
Monte Carlo
Carlo simulation
simulation model
model to
to evaluate
evaluate the
the expected
expected performance
performance
of
the
blueprint
in
various
scenarios
and
to
quantify
these
results.
of the blueprint in various scenarios and to quantify these results.
2012
2014
Benefits
Benefits
With
With our
our results,
results, the
the client
client gained
gained insight
insight into
into the
the expected
expected performance
performance of
of the
the
new
blueprint,
including
the
expected
performance
for
several
alternative
new blueprint, including the expected performance for several alternative
scenarios.
scenarios.
33
Mortgage Risk Portfolio Management at a large Dutch bank

Background
Background
AADutch
Dutch retail
retail bank
bank started
started aa project
project combining
combining internal
internal and
and external
external data
data
sources.
The
goal
is
to
model
risk
for
a
collection
of
mortgages
sources. The goal is to model risk for a collection of mortgages based
based on
on lifelifechanging
changing events
events in
in order
order to
to reduce
reduce costs
costs on
on financially
financially unhealthy
unhealthy clients.
clients. By
By
combining
combining public
public information
information on
on an
an online
online house-selling
house-selling website
website with
with internal
internal
bank
data,
a
model
has
been
built
to
describe
and
predict
the
length
bank data, a model has been built to describe and predict the length of
of
mortgages
mortgages and
and to
to analyze
analyze the
the difference
difference in
in house-selling
house-selling price
price and
and the
the
mortgage.
mortgage.
Requirements
Challenges
Company A
Privacy
Privacy challenge:
challenge: sensitive
sensitive personal
personal information
information needs
needs to
to be
be anonymized
anonymized
Duration
Duration (develop
(develop phase):
phase): 66 10
10 weeks
weeks
Solution
Solution
Customer
3
encryption
Resources
Resources
Data
Data sources:
sources: internal
internal bank
bank data,
data, open
open data
data crawled
crawled from
from Funda.nl
Funda.nl
Analytics
Institute
Data Hub
Company B
encryption
More
More than
than 200,000
200,000 mortgages
mortgages && over
over 250,000
250,000 homes
homes for
for sale
sale on
on Funda.nl,
Funda.nl,
while
limited
or
no
understanding
of
customers
selling
houses
while limited or no understanding of customers selling houses
An
An application
application crawls
crawls the
the Funda.nl
Funda.nl pages
pages on
on aa daily
daily basis
basis
The
The results
results are
are loaded
loaded into
into aa Teradata
Teradata Aster
Aster Cluster
Cluster and
and combined
combined with
with
bank
information
on
customers
and
products
bank information on customers and products
Mortgages
Mortgages and
and Funda.nl
Funda.nl results
results are
are linked
linked based
based on
on postal
postal code
code and
and
house
house number
number
Red line:
clients from
Fin. Health
Dep.
Blue line:
financially
healthy clients
Benefits
Benefits
Daily
Daily insight
insight into
into the
the housing
housing market:
market: prices,
prices, locations
locations and
and time
time and
and
understanding
customers
that
offer
a
home
or
apartment
for
sale
understanding customers that offer a home or apartment for sale
34
Financial health monitoring at a large Dutch bank

In
In aa short
short Big
Big Data
Data pilot,
pilot, aa bank
bank wants
wants to
to research
research the
the possibility
possibility to
to find
find
indicators
indicators that
that consumers
consumers (customers)
(customers) will
will get
get into
into financial
financial trouble
trouble before
before itit
actually
actually happens.
happens. This
This can
can be
be input
input for
for proactively
proactively helping
helping customers
customers that
that
need
financial
aids,
enlarging
customer
trust,
cutting
costs
on
special
needs
need financial aids, enlarging customer trust, cutting costs on special needs
customers
customers and
and decrease
decrease losses
losses on
on defaults.
defaults.
Requirements
Challenges
In
In order
order to
to get
get hands-on
hands-on experience
experience the
the bank
bank wants
wants to
to use
use Big
Big Data
Data
technologies
(Teradata
cluster)
technologies (Teradata cluster)
Resources
Resources
How often does a customer spend an amount under 1000 euros?
Fraction of all spending
Background
Background
Conservative
spending
Data
Data sources:
sources: bank
bank transactions
transactions data
data and
and customer
customer data
data
Conservativity
index
Duration
Duration (develop
(develop phase):
phase): 66 16
16 weeks
weeks
Customer
needs
special
attention
Solution
Solution
Pattern
Pattern recognition
recognition of
of income
income and
and expenditure
expenditure
Modelling
Modelling the
the customer
customer dynamically,
dynamically, as
as aa response-system
response-system
Describe
Describe the
the transaction
transaction behavior
behavior as
as aa coupled
coupled second-order
second-order linear
linear
differential
differential equation
equation
Month
Customers heading for a financially
unhealthy situation
Financially healthy customers
Monte
Monte Carlo
Carlo simulation
simulation on
on the
the customer
customer behavior
behavior in
in order
order to
to do
do aa
prediction
prediction of
of financial
financial health.
health.
Benefits
Benefits
Insights
Insights in
in typical
typical spending
spending behavior
behavior of
of financially
financially healthy
healthy and
and unhealthy
unhealthy
customers
customers
Triangles hold financially

sustainable behavior
35
Conclusions and Reflections
Conclusions
Think big, start small, scale fast!

Start at the data. Value of data cannot be determined
upfront. Work iteratively toward value.
Privacy rules and regulations change when moving
from small to Big Data. Big Data really is different.
Allow the team to experiment and be creative.
Embrace a fail-fast distributed risk approach.
Support a possible use-case with a business case so
The Golden Case:

1. Adds value for society and clients.
everybody is clear on what to expect as a result.
2. Shows the potential of Big Data
Big data is about ideas, skills but most of all: data.
3. Starts small but is meaningful.
Team up to get access to complementary sources.
4. Has the potential to scale fast.
37
Maybe trust is overrated
Trust to not loose money / data

versus
Trust to do the right thing
38
2014 KPMG Advisory N.V., registered with the trade register in the
Netherlands under number 33263682, is a subsidiary of KPMG Europe LLP
and a member firm of the KPMG network of independent member firms
affiliated with KPMG International Cooperative (KPMG International),
a Swiss entity. All rights reserved. Printed in the Netherlands.
The KPMG name, logo and cutting through complexity are registered
trademarks or trademarks of KPMG International.
Sander Klous
KPMG Management Consulting
Laan van Langerhuize 1
1186 DS, Amstelveen
Tel: +31 20 656 7186
klous.sander@kpmg.nl

Big Data Analytics

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Big Data Analytics

Uploaded by

Copyright:

Available Formats

We are Big Data

The future of the information society

prof. dr. Sander Klous

Big, bigger, biggest

Nobel prize 2013 to Higgs and Englert

The new information society

Shared value with Big Data

Torrents in the music industry,

When is a trend going to

Jobless recovery in a developing information society

The dark side of Big Data

Big Brother is watching you

Big Brother is watching you

The Target case: life changing events

Andrew Pole had just started

Lack of transparency: LG and TP-Link (Philips) example

If something is for free,

Fundamental questions about

The profile of the data scientist

Jar A contains 10 apples and 30 pears

Jar B contains 20 of each

Fred picks a jar, without further evidence there is a 50% chance

Mechanism Design (Nobel prize 2007)

Understanding large scale

Rules and regulations in a do it yourself society

Big Data and Society, an ethical perspective

A Practical Guide to Big Data

KPMG Analytics & Visualization Environment

No single attractive target for hackers

Governance in a data driven organization

The D&A board is

As the D&A units mature, the

From data to ideas

The D&A units facilitate the

Real time analysis

The board is responsible for

Agile project management

Proof of Concept for selected cases and

Transformation of the solution

Continuous interaction to orchestrate Big

selected client cases

the Proof of Concepts

evaluating the results

Applying Big Data

CTR SWAP: 0.079

Asset management and planning at a University Medical Center

Mortgage Risk Portfolio Management at a large Dutch bank

Financial health monitoring at a large Dutch bank

How often does a customer spend an amount under 1000 euros?

Fraction of all spending

Financially healthy customers

Triangles hold financially

Conclusions and Reflections

Think big, start small, scale fast!

The Golden Case:

everybody is clear on what to expect as a result.

2. Shows the potential of Big Data

Big data is about ideas, skills but most of all: data.

3. Starts small but is meaningful.

Team up to get access to complementary sources.

4. Has the potential to scale fast.

Maybe trust is overrated

Trust to not loose money / data