You are on page 1of 81

Big Data Architecture

Industry 4.0 2017-2018

Prof. Dr. Óscar Gallego Castilla

oscar.gallegocastilla@ge.com
ogallego@iese.edu
ogallego@comillas.edu
oscgal123@gmail.com
Topics

1. What is Big Data

2. Big Data Architecture

Deusto Industry 4.0 2017-2018 Prof. Dr. Óscar Gallego


Some definitions on Big Data
• Big data refers to massive • “The effective use of big data
amounts of business data has the potential to transform
from a wide variety of economies, delivering a new
sources, much of which is wave of productivity growth
available in real time, and and consumer surplus.
much of which is uncertain or Using big data will become a
unpredictable. IBM calls key basis of competition for
these characteristics existing companies, and will
volume, variety, velocity, create new competitors who
and veracity. are able to attract employees
that have the critical skills for
a big data world.” - McKinsey
Global Institute, 2011

Deusto Industry 4.0 2017-2018 Prof. Dr. Óscar Gallego


Some definitions on Big Data
• IDC definition of Big Data • “Gartner definition Big data is
(conservative and strict high-volume, high-velocity
approach) : "A new and high-variety information
generation of technologies assets that demand cost-
and architectures designed effective, innovative forms of
to economically extract value information processing for
from very large volumes of a enhanced insight and
wide variety of data by decision making.
enabling high-velocity
capture, discovery, and/or
analysis“

Deusto Industry 4.0 2017-2018 Prof. Dr. Óscar Gallego


Some definitions on Big Data
• Big Data: a massive volume • “Data that exceeds the
of both structured and processing capacity of
unstructured data that is so conventional database
large that it's difficult to systems. The data is too big,
process using traditional moves too fast, or doesn’t fit
database and software the structures of your
techniques. – From “The Big database architectures. To
Data Long Tail” blog post by gain value from this data,
Jason Bloomberg (Jan 17, you must choose an
2013). alternative way to process it.”
– Ed Dumbill, program chair
for the O’Reilly Strata
Conference

Deusto Industry 4.0 2017-2018 Prof. Dr. Óscar Gallego


Some definitions on Big Data

Deusto Industry 4.0 2017-2018 Prof. Dr. Óscar Gallego


Some definitions on Big Data
• “Generic Big Data Properties
• Volume
• Variety
• Velocity

• Acquired Properties (after


entering system).
• Value
• Veracity
• Variability

Deusto Industry 4.0 2017-2018 Prof. Dr. Óscar Gallego


Some definitions on Big Data
• (1) Big Data Properties: 5V – Volume, Variety, Velocity, Value, Veracity –
Additionally: Data Dynamicity (Variability)
• (2) New Data Models – Data Lifecycle and Variability – Data linking,
provenance and referral integrity
• (3) New Analytics – Real-time/streaming analytics, interactive and machine
learning analytics
• (4) New Infrastructure and Tools – High performance Computing, Storage,
Network – Heterogeneous multi-provider services integration – New Data
Centric (multi-stakeholder) service models – New Data Centric security models
for trusted infrastructure and data processing and storage
• (5) Source and Target – High velocity/speed data capture from variety of
sensors and data sources – Data delivery to different visualisation and
actionable systems and consumers – Full digitised input and output,
(ubiquitous) sensor networks, full digital control

Deusto Industry 4.0 2017-2018 Prof. Dr. Óscar Gallego


Some definitions on Big Data

Deusto Industry 4.0 2017-2018 Prof. Dr. Óscar Gallego


Key definitions regarding Big Data
• Data: numerical or textual • Database - a collection of • Metric - a unit of
facts and figures that are related files containing measurement that provides a
collected through some type records on people, places, or way to objectively quantify
of measurement process. things. performance.
Evans (2016) • A database file is usually • Discrete metric - one that is
• Data set - a collection of data. organized in a two- derived from counting
dimensional table, where something.
• Examples: Marketing
the columns correspond
survey responses, a table • delivery on time or not;
to each individual
of historical stock prices, order complete or
element of data (called
and a collection of incomplete; invoice
fields, or attributes), and
measurements of errors; proportion of on-
the rows represent
dimensions of a time or off-time
records of related data
manufactured item. deliveries, etc..
elements.
• Data sources: marketing, • Information: result of
economic trends, audits, analyzing data; that is,
annual reports, ops, HR, extracting meaning from data
web (time in web, origin, to support evaluation and
destination, products decision making.
searched, etc.)

Deusto Industry 4.0 2017-2018 Prof. Dr. Óscar Gallego


Key definitions regarding Big Data
• Continuous metrics are • Categorical (nominal) data • Reliability - data are
based on a continuous scale - sorted into categories accurate and consistent.
of measurement. according to specified
• Validity - data correctly
• Any metrics involving characteristics.
measures what it is
dollars, length, time, • Ordinal data - can be supposed to measure.
volume, or weight, for ordered or ranked according
example, are continuous. • Examples:
to some relationship to one
• A tire pressure gage that consistently
• Measurement - the act of another. reads several pounds of pressure
below the true value is not reliable,
obtaining data associated • Interval data - ordinal but although it is valid because it does
with a metric. have constant differences measure tire pressure.
• The number of calls to a customer
• Measures - numerical between observations and service desk might be counted
values associated with a have arbitrary zero points. correctly each day (and thus is a
reliable measure) but not valid if it is
metric. • Ratio data - continuous and used to assess customer
dissatisfaction, as many calls may be
have a natural zero simple queries.
• A survey question that asks a
customer to rate the quality of the food
in a restaurant may be neither reliable
(because different customers may
have conflicting perceptions) nor valid
(if the intent is to measure customer
satisfaction, as satisfaction generally
includes other elements of service
besides food).

Deusto Industry 4.0 2017-2018 Prof. Dr. Óscar Gallego


Example of a DB

Records

Entities Fields or Attributes

Deusto Industry 4.0 2017-2018 Prof. Dr. Óscar Gallego


Example of data elements classification

Deusto Industry 4.0 2017-2018 Prof. Dr. Óscar Gallego


How is data generated - Machine
• Example - Planes: • Processing: • Shit:
• .5-1 TB per mid-range flight • Before - RDMBS • Real-time computation
enables real-tim actions
• Data moved elsewhere to and a different design
be processed
• Real-time design enables
real-time operations and
impact in: customer
relations, fraud detection,
system and monitoring,
Smart metering, machine
health, etc.
• Produced by • All enabled by the cloud:
• Now: in-situ, bringing increased used of
• Accelerometers
(turbulence) computation to data, resulting scalable computing
in real-time computation at power, SCADA
• Sensors (temperature,
+40K feet to solve a problem (Supervisory Control and
pressure, malfunctions)
in air Data Acquisition)
• Helping identifying
trends, reducing waste,
increasing efficiency, etc..

Deusto Industry 4.0 2017-2018 Prof. Dr. Óscar Gallego


How is data generated - Organization
• Example - UPS: • Spend on Big Data: • Netnet:
• 16 m shipments per day • Higher sales
• 40 m tracking requests • Improved
• 16 PBs of data safety
• Efficient
operations
• Results: • Customer
satisfaction
• Improved
profitability…
• Reduction of 1 mile route /
driver reduces costs by 50
m USD, achieved by large
operational data and
optimization algorithms

Deusto Industry 4.0 2017-2018 Prof. Dr. Óscar Gallego


How is data generated - People
• Example - Facebook: • Issue: heavily un-structured data (80-90%)
• In 1 day it generates more
data than all the US
Academic Libraries 30+ vs 2
PBs
• Others:
Company Data ProcessedDaily
eBay 100 Petabytes(PB)
Google 100 PB
Facebook 30+ PB
Twitter 100 Terabytes(=.1PB)
Spotify 64 Terabytes

• Even more un-structured:

Deusto Industry 4.0 2017-2018 Prof. Dr. Óscar Gallego


How is data generated - People
• Velocity of data generated being an issue (per minute):

• Real issue is:


• Integration
• Data acquisition
• Storage
• Retrival
• Cleaning
• Processing
• Visualizing
• Delivering…

AND THE VALUE!!!

Deusto Industry 4.0 2017-2018 Prof. Dr. Óscar Gallego


The expansion of data
• Size matters, useful data too:

• Real issue is:


• Value of data at a
fraction of the total
size
• Cost of Data is
huge

Deusto Industry 4.0 2017-2018 Prof. Dr. Óscar Gallego


Where is Big Data going?

19 Deusto Industry 4.0 2017-2018 Prof. Dr. Óscar Gallego


Where is Big Data going?

20 Deusto Industry 4.0 2017-2018 Prof. Dr. Óscar Gallego


Topics

1. What is Big Data

2. Big Data Architecture

Deusto Industry 4.0 2017-2018 Prof. Dr. Óscar Gallego


Big Data Architecture
Architecture, Infrastructure and Systems
• The key idea is that documenting the right foundation of
architecture, infrastructure and applications ultimately allows the
business to more effectively use big data more on an everyday
basis.
• While big data architecture can seem like a highly technical topic, it’s
important to realize that big data innovations and insights are not
possible without a well-conceived, clearly defined and
thoughtfully designed architecture.
• If your business has big plans for big data, a strong big data
architecture is required to executing those plans

22 Deusto Industry 4.0 2017-2018 Prof. Dr. Óscar Gallego


Big Data Architecture
Asking the right questions
• How do big data strategies – basically, the business problems to be solved,
operations to be improved and objectives to be achieved by using big data –
shape architecture needs?
• Which of our existing data sources and systems can be “plugged into” an
integrated architecture for big data?
• How do we account for new data sets (like sensor data or data from the
Internet of Things)?
• How can our approach to big data architecture help move data-driven and
analytics-enabled thinking into the center of our business?
• What are the required components to “operationalize” or scale big data and
analytics program beyond pilot phases?

23 Deusto Industry 4.0 2017-2018 Prof. Dr. Óscar Gallego


What is Big Data Architecture
• Think of blueprints for a house or building: a big data architecture is
a conceptual or graphical model of how big data and other
information assets will be captured, stored, managed and made
accessible to various user groups and applications.
• Typically, big data architectures outline the hardware and software
components that are necessary to a full big data solution. Big data
architecture documents may also describe protocols for data
sharing, application integrations and information security.
• The more you’re investing in big data solutions, the more you need a
big data architecture to make sure you get the ROI you desire - >
big data architectures help ensure data flows as planned so the
right users can access it via the right tools.

24 Deusto Industry 4.0 2017-2018 Prof. Dr. Óscar Gallego


What is Big Data Architecture
• Business/IT must not mistake the blueprint for the solution itself.
• A big data architecture is the design and documentation describing
how big data works once all the components, data sources and
applications are connected and integrated in a unified whole.
• Big Data Architecture doesn't necessarily generate business value
on its own, but it sets up a foundation for success.
• The value comes from defining the right big data processes and
structure, deploying advanced big data analytics and having the
right people and teams in place to interact with and interrogate the
data.

25 Deusto Industry 4.0 2017-2018 Prof. Dr. Óscar Gallego


Architectural Patterns and Styles
• Reuse is critical in software development: reinventing the wheel is
unnecessary, time-consuming, and is ultimately very expensive.
• SW architects and engineers embrace patterns to apply the same,
or very similar solutions to commonly occurring problems.
• Design patterns refer to reusable patterns applied in software code,
whereas architectural patterns are reusable patterns used to design
complete software, big data, IoT, and/or analytics-based solutions.
Usually categorized as: creational, structural, behavioral,
or concurrent, and there are many well-defined patterns for each
category.

26 Deusto Industry 4.0 2017-2018 Prof. Dr. Óscar Gallego


Architectural Patterns and Styles
• Architectural patterns are often categorized and grouped by
an architectural style. Some architectural styles: structure, shared
memory, messaging, adaptive systems, and distributed systems.
• Cloud solutions consist of clients, servers, and data storage.
• Clients typically include desktop, web, and/or native mobile
applications.
• Servers usually host databases, web servers, services,
microservices.

27 Deusto Industry 4.0 2017-2018 Prof. Dr. Óscar Gallego


Architectural Patterns and Styles
• Network Communication and Information Transfer
• How information is transferred between different computers, software,
and services?
• Ultimately, everything transmitted on local networks and the
Internet are a form of data message and are typically encoded
according to a specific protocol. T
• The network protocols found in the OSI model layers are the most
common, and include HTTP/HTTPS, TCP/IP, UDP, IPv4/IPv6, etc.
• All requests made from a web browser to a server, and the
subsequent responses, leverage the HTTP and TCP/IP protocols. Two
primary messaging patterns are request-response (e.g., HTTP)
and one-way (e.g., UDP).

28 Deusto Industry 4.0 2017-2018 Prof. Dr. Óscar Gallego


Architectural Patterns and Styles
• Client-server, Layers and Tiers
• In client-server, a client makes requests to a server responsible for
providing an appropriate response. Both the client and the server can
be considered independent applications that interact with each other
through well-defined interfaces.
• The code in both is often separated into layers that each address
different concerns.
• Common layers include a presentation layer, application (or service)
layer, business (or domain) logic layer, and a data access (or
persistence) layer. Actual data storage and databases can be
considered the database layer.

29 Deusto Industry 4.0 2017-2018 Prof. Dr. Óscar Gallego


Architectural Patterns and Styles
• Client-server, Layers and Tiers
• Layers are used to implement separation of concerns in the code
that’s running on a single computing machine. Architecturally, software
solutions are often separated into tiers, where a tier describes a part of
the solution running on a different computing instance
(physical or virtual).
• Typical tiers include the presentation tier, domain logic tier (e.g., API),
and data storage tier. This is a typical 3-tier architecture, and n-tier is
used to describe an architecture that includes more than one tier.

30 Deusto Industry 4.0 2017-2018 Prof. Dr. Óscar Gallego


Architectural Patterns and Styles
• Services, Service Oriented Architecture (SOA) and
Microservices
• SW applications can be separated into services, or units of software functionality that are deployed
remotely and can be accessed via networking communication protocols.
• Services are leveraged for many uses, with information retrieval and execution of operations being the
most common.
• SOA is an architectural pattern that describes a solution that’s broken into discrete components that
are each responsible for a certain part of the solution’s functionality, and provide that functionality as a
service to other solution components. It has 4 key properties:
• It logically represents a business activity with a specified outcome
• It is self-contained
• It is a black box for its consumers
• It may consist of other underlying services
• SOA is an separation of concerns, modularity, and loose coupling as applied at the application or
solution level. The most typical implementations of SOA include web services, messaging services,
and RESTful services (e.g., API).
• A popular and more specialized version of SOA is know as microservices. Microservices are organized
around individual application capabilities, as compared to the services of SOA being organized around
a specific business activity.
31 Deusto Industry 4.0 2017-2018 Prof. Dr. Óscar Gallego
Architectural Patterns and Styles
• Services, Service Oriented Architecture (SOA) and
Microservices
• Microservices are very similar to the services of SOA, but are characterized as being:
• More granular (smaller) and finer grained interfaces
• More lightweight
• More modular and loosely coupled
• More easily replaced
• More language and technology agnostic
• More specialized around a specific application capability
• Better suited for containerization and deployment
• Better suited for agility and continuous delivery

32 Deusto Industry 4.0 2017-2018 Prof. Dr. Óscar Gallego


Architectural Patterns and Styles
• Services, Service Oriented Architecture (SOA) and
Microservices
• Microservices also introduce many potential drawbacks as well, which should be considered carefully.
Some of these are related to the fallacies of distributed computing, and include:
• Significant increase in:
• Application testing, configuration, deployment, and management complexity
• Internal team communications and coordination costs
• HTTP request/response call chain complexity
• Potential networking issues such as network failure and high latency
• Computational and performance overhead due to network-based messaging communication and transfer
protocols
• Requires DevOps adoption (e.g., scalability) and monitoring practices
• Potentially distributed transaction handling
• Some also note that the true benefits of microservices are not realized unless each microservice has
sole ownership of their data, i.e., having their own data store

33 Deusto Industry 4.0 2017-2018 Prof. Dr. Óscar Gallego


Architectural Patterns and Styles
Example of Microservices

34 Deusto Industry 4.0 2017-2018 Prof. Dr. Óscar Gallego


Solution Architectures and Patterns
• Structure
• Separation of concerns (SOC) is a way of separating
different concerns of a code base or complete solution in order to
improve code reuse, reduce coupling, improve testability and
maintainability, and so on.
• In the extreme case where concerns are barely separated, if at all, an
application and its code base is referred to as being monolithic. This
type of application has very tightly coupled and mostly non-reusable
code.
• Software solutions that embrace and emphasize SOC are usually
referred to as being layered and/or tiered as previously discussed, but
also can be component-based, modular, service oriented, and so on.

35 Deusto Industry 4.0 2017-2018 Prof. Dr. Óscar Gallego


Solution Architectures and Patterns
• Structure
• Components are usually higher level, larger abstractions of
encapsulated concerns of a complete solution. Examples include web
services, software packages, or software binaries.
• Modules on the other hand are usually separate and encapsulated
concerns within an application, service, package, or library’s code
base. These items when composed of many interchangeable, yet
integrated modules is considered to be highly modular and
therefore loosely coupled.

36 Deusto Industry 4.0 2017-2018 Prof. Dr. Óscar Gallego


Solution Architectures and Patterns
• Shared memory
• The main pattern to note here is the blackboard pattern and
associated blackboard system. The common terms involved with this
pattern include blackboard, knowledge sources, and control
component (or control shell).
• The pattern is analogous to an actual blackboard (or whiteboard)
session, in which multiple people (knowledge sources) write various
approaches or elements to solve a problem on a board (blackboard),
and then a control component is able to selectively piece together (i.e.,
execute) knowledge sources as needed to produce a solution.

37 Deusto Industry 4.0 2017-2018 Prof. Dr. Óscar Gallego


Solution Architectures and Patterns
• Messaging
• A very important architecture is called event-driven or message-
driven architecture (EDA). This architecture deals with producing,
detecting, processing, and responding to events, which as previously
mentioned, take the form of messages. This architectural pattern
promotes loose coupling, scalability, high performance, efficiency, and
potentially asynchronous, non-blockingoperations.
• Events are created by a so-called event emitter or agent, which are
received by event consumersor sinks. The communication and pairing
between emitters and consumers is facilitated by event channels.
• The specific event channel and event handling framework is typically
implemented using a system or pattern such as message-oriented
middleware (MOM), publish-subscribe (pub/sub), and message
queues.
38 Deusto Industry 4.0 2017-2018 Prof. Dr. Óscar Gallego
Solution Architectures and Patterns
• Messaging
• Other related concepts and architectures are the enterprise service
bus (ESB) and enterprise messaging system (EMS). An ESB acts as a
highly flexible and loosely coupled communication system, in the form
of a virtual software bus, that sits between distributed services and/or
software applications.
• Message-based communication via an ESB is based on an enterprise
messaging system (EMS), which defines and outlines various
standards, best practices, and implementations of the messaging
system itself. This includes message formats, queuing, transfer
protocols, application protocols, security, and more.

39 Deusto Industry 4.0 2017-2018 Prof. Dr. Óscar Gallego


Solution Architectures and Patterns
• Adaptive systems
• The most predominant architectural patterns in this category are plug-
ins, integrations, and the microkernel pattern. Plug-ins and integrations
are increasingly being referred to as apps or extensions.
• Plug-ins and integrations can be considered as part of a broader
category known as add-ons, which allow users to add functionality and
features as needed. Software applications that allow and support any
form of add-ons are considered to be customizable.
• The main difference between plug-ins, extensions, and integrations
involves the type and form of functionality being added.

40 Deusto Industry 4.0 2017-2018 Prof. Dr. Óscar Gallego


Solution Architectures and Patterns
• Adaptive systems
• Plug-ins tend to be software packages that host applications leverage (via a plug-in
manager) to provide additional non-native functionality by specifying an interface and API for
which the plug-in adapts to. Plug-ins tend to be dedicated software packages created
specifically for the host application. Examples include Salesforce Apps and Adobe Flash for
web browsers.
• Integrations, on the other hand, are ‘apps’ that can extend a host application’s functionality
like a plug-in, but typically represent a stand alone software application or service that exists
as a product or service in its own right. In this case, the host recognizes another app or
service’s usefulness and identifies ways in which it can be integrated into a user’s workflow
within the host application.
• Examples include Google Calendar, Trello, and Jira integrations with Slack. Google
Chromeoffers many app integrations and extensions as well.

41 Deusto Industry 4.0 2017-2018 Prof. Dr. Óscar Gallego


Solution Architectures and Patterns
• Distributed systems
• Distributed systems use more than one physical or virtual compute
instance to provide functionality, e.g., a software application, service,
or database.
• Some examples include the tiered and service-oriented architectures,
as discussed. Other notable architectures in this category
include peer-to-peer, space-based, and the many different
architectural patterns associated with cloud computing and cloud
services (e.g., AWS) in general.
• Peer-to-peer computing contrasts with client-server computing in that
peers (aka nodes) work together to accomplish tasks and share
resources, as opposed to being characterized by strict separations
of consumer/provider or request/response relationships. Peers can be
both consumers and providers within their network.
42 Deusto Industry 4.0 2017-2018 Prof. Dr. Óscar Gallego
Solution Architectures and Patterns
• Distributed systems
• Space-based architecture is a pattern driven primarily by scalability
and performance goals, particularly when involving large numbers of
concurrent and often unpredictable loads on the system, usually in the
form of requests.
• This pattern is a bit complex relative to some others, and involves
horizontally scalable computing or processing units that each
leverage in-memory data for maximum performance. There are also
caching and asynchronous data persistence components.
• Lastly, space-based architecture involves a virtualized
middleware component, which consists of a messaging grid, data
grid, processing grid, and deployment manager. This middleware
component is a controller that performs most management and
coordination tasks associated with this architecture.
43 Deusto Industry 4.0 2017-2018 Prof. Dr. Óscar Gallego
Other Architectural Patterns
• Proxy server
• Caching
• MV*__ (e.g., _MVC, MVP, MVVM, …)
• Single page application (SPA) vs multi-page
• Static site (HTML, CSS, JavaScript, and assets only)
• Job Scheduler, aka scheduling
• Inversion of control (IOC)

44 Deusto Industry 4.0 2017-2018 Prof. Dr. Óscar Gallego


Other Architectural Patterns
## APIs and SDKs
• One of the most important and widely used patterns is
the application programming interface(API) and often
complementary software development kit (SDK).
• Web applications are usually split between client and server
components, where the client typically runs in a web browser or
mobile device, and the server typically runs on a physical or virtual
machine.
• A very popular and widespread trend is to shift an increasing amount
of application logic to the front-end of web and/or native mobile-
based applications. Given this, many software applications
communicate primarily through matched SDKs on the client,
and RESTful APIs on the server.
45 Deusto Industry 4.0 2017-2018 Prof. Dr. Óscar Gallego
Other Architectural Patterns
## APIs and SDKs
• REST stands for representational state transfer and is a communication and
information transfer pattern between clients and servers. Servers
expose RESTful API endpoints so that client applications can communicate with
them over common Internet protocols (e.g., HTTP and HTTPS), and in order to
perform CRUD operations on application data, or to execute specific server-side
tasks and operations.
• SDKs are packages that client applications (web and mobile) can use to interact
with related APIs, and are designed to simplify and abstract away much of the
complexity of the client/server communication and data transfer. SDKs are
usually implemented as JavaScript libraries or packages for web applications,
and also often come in native mobile (iOS and Android) versions as well.

46 Deusto Industry 4.0 2017-2018 Prof. Dr. Óscar Gallego


What is an Architectural Framework
• Existing attempts address architecture issues in a traditional way: ODCA, TMF,
NIST –
http://bigdatawg.nist.gov/_uploadfiles/BD_Vol5_RefArchSurvey_V1Draft_Prerel
ease.pdf
• Architecture vs Ecosystem
• Big Data undergo a number of transformations during their lifecycle
• Big Data fuel the whole transformation chain: Data sources and data consumers,
target data usage
• Multi-dimensional relations between: Data models and data driven processes,
Infrastructure components and data centric services
• Architecture vs Architecture Framework
• Separates concerns and factors: Control and Management functions, orthogonal
factors
• Architecture Framework components are inter-related

47 Deusto Industry 4.0 2017-2018 Prof. Dr. Óscar Gallego


What is an Architectural Framework
Las clases que yo impartiría son:
- Diseño de Big Data Arquitectura
- Caso de Éxito: la fabricación 3D más grande el mundo (GE AM)
- Aplicaciones de Producción (MES), inteligencia y control
- Plataformas Colaborativas Industriales
- Caso de Éxito: la mayor transformación digital del mundo (GE)
- Brilliant Manufacturing (Digital)
- Caso de Éxito: Industria Pesada Digital

48 Deusto Industry 4.0 2017-2018 Prof. Dr. Óscar Gallego


What is an Architectural Framework
Key Components
• (1) Data Models, Structures, Types – Data formats, non/relational, file systems,
etc.
• (2) Big Data Management – Big Data Lifecycle (Management) Model • Big Data
transformation/staging – Provenance, Curation, Archiving
• (3) Big Data Analytics and Tools – Big Data Applications • Target use,
presentation, visualisation
• (4) Big Data Infrastructure (BDI) – Storage, Compute, (High Performance
Computing,) Network – Sensor network, target/actionable devices – Big Data
Operational support
• (5) Big Data Security – Data security in-rest, in-move, trusted processing
environments

49 Deusto Industry 4.0 2017-2018 Prof. Dr. Óscar Gallego


What is an Architectural Framework

50 Deusto Industry 4.0 2017-2018 Prof. Dr. Óscar Gallego


What is an Architectural Framework

51 Deusto Industry 4.0 2017-2018 Prof. Dr. Óscar Gallego


Data Models

52 Deusto Industry 4.0 2017-2018 Prof. Dr. Óscar Gallego


Data Transformation Models

53 Deusto Industry 4.0 2017-2018 Prof. Dr. Óscar Gallego


Big Data Architecture and Patterns

54 Deusto Industry 4.0 2017-2018 Prof. Dr. Óscar Gallego


Using Hadoop

55 Deusto Industry 4.0 2017-2018 Prof. Dr. Óscar Gallego


Big Data Architecture and Patterns

56 Deusto Industry 4.0 2017-2018 Prof. Dr. Óscar Gallego


Big Data Architecture and Patterns

57 Deusto Industry 4.0 2017-2018 Prof. Dr. Óscar Gallego


Big Data Architecture and Patterns

58 Deusto Industry 4.0 2017-2018 Prof. Dr. Óscar Gallego


Solution Architectures and Patterns

59 Deusto Industry 4.0 2017-2018 Prof. Dr. Óscar Gallego


Solution Architectures and Patterns

60 Deusto Industry 4.0 2017-2018 Prof. Dr. Óscar Gallego


Solution Architectures and Patterns

61 Deusto Industry 4.0 2017-2018 Prof. Dr. Óscar Gallego


Solution Architectures and Patterns

62 Deusto Industry 4.0 2017-2018 Prof. Dr. Óscar Gallego


Solution Architectures and Patterns

63 Deusto Industry 4.0 2017-2018 Prof. Dr. Óscar Gallego


Solution Architectures and Patterns

64 Deusto Industry 4.0 2017-2018 Prof. Dr. Óscar Gallego


Solution Architectures and Patterns

65 Deusto Industry 4.0 2017-2018 Prof. Dr. Óscar Gallego


Solution Architectures and Patterns

66 Deusto Industry 4.0 2017-2018 Prof. Dr. Óscar Gallego


Solution Architectures and Patterns

67 Deusto Industry 4.0 2017-2018 Prof. Dr. Óscar Gallego


Solution Architectures and Patterns

68 Deusto Industry 4.0 2017-2018 Prof. Dr. Óscar Gallego


Solution Architectures and Patterns

69 Deusto Industry 4.0 2017-2018 Prof. Dr. Óscar Gallego


Solution Architectures and Patterns

70 Deusto Industry 4.0 2017-2018 Prof. Dr. Óscar Gallego


Solution Architectures and Patterns

71 Deusto Industry 4.0 2017-2018 Prof. Dr. Óscar Gallego


Solution Architectures and Patterns

72 Deusto Industry 4.0 2017-2018 Prof. Dr. Óscar Gallego


Solution Architectures and Patterns

73 Deusto Industry 4.0 2017-2018 Prof. Dr. Óscar Gallego


Solution Architectures and Patterns

74 Deusto Industry 4.0 2017-2018 Prof. Dr. Óscar Gallego


Solution Architectures and Patterns

75 Deusto Industry 4.0 2017-2018 Prof. Dr. Óscar Gallego


Solution Architectures and Patterns

76 Deusto Industry 4.0 2017-2018 Prof. Dr. Óscar Gallego


Solution Architectures and Patterns

77 Deusto Industry 4.0 2017-2018 Prof. Dr. Óscar Gallego


Solution Architectures and Patterns

78 Deusto Industry 4.0 2017-2018 Prof. Dr. Óscar Gallego


Solution Architectures and Patterns

79 Deusto Industry 4.0 2017-2018 Prof. Dr. Óscar Gallego


Solution Architectures and Patterns

80 Deusto Industry 4.0 2017-2018 Prof. Dr. Óscar Gallego


Solution Architectures and Patterns

81 Deusto Industry 4.0 2017-2018 Prof. Dr. Óscar Gallego

You might also like