INFORMATION MANAGEMENT (2013 REGULATION)

UNIT III INFORMATION GOVERNANCE

Master Data Management (MDM) – Overview, Need for

MDM,

Privacy, regulatory requirements and compliance.

Data Governance – Synchronization and data quality

management.

1
UNIT - I WEB
INFORMATION MANAGEMENT (2013 SERVERS
REGULATION) , SERVLETS
Data Information
Data is raw & unorganized form. (After processing the data)
Information is organized, structured (or) presented in a given
context so as to make it useful.

Data is of various types:

1. Unstructured – Heterogeneous (Does not have any format)

2. Structured - structured data include numbers, dates, and groups of words and numbers
called strings

3. Transactional – Invoices, Claims, Sales and deliveries

4. Metadata – Data about the data ( Reports, log, configuration files)

5. Hierarchical – Data consist of relationships among different entities.

UNIT - I WEB
INFORMATION MANAGEMENT (2013 SERVERS
REGULATION) , SERVLETS
Information Governance: [IG]
“Is the management of information at an organization”
 Information governance balances the use and security of information.

 Information governance helps with legal compliance, operational transparency, and reducing
expenditures associated with legal discovery.

 An organization can establish a consistent and logical framework for employees to handle data through
their information governance policies and procedures.

 These policies guide proper behavior regarding how organizations and their employees handle
electronically stored information.

 Information governance encompasses more than traditional records management.

 It incorporates information security and protection, compliance, data governance, electronic discovery, risk
management, privacy, data storage and archiving, knowledge management, business operations and
management, audit, analytics, IT management, master data management, enterprise architecture, business
intelligence, big data, data science, and finance.
UNIT - I WEB
INFORMATION MANAGEMENT (2013 SERVERS
REGULATION) , SERVLETS
Need for the Master Data Management:
Big Data:
 Collection of data that is huge in size and growing exponentially with time.
Challenges:
 Data exists in different types, and it has to be stored in different forms.
 The old data is changed and updated, which is no longer valid and needs to be discarded. (The correct data
version should be available and accessible)
 The enterprise find it very difficult and challenging to create, find and manage data that is complete,
accurate, readily and uniformly available to its user. (Single version of data, good quality)
 Data in an enterprise is shared and used by several applications. Large amount of inconsistency, poor quality,
leads to unacceptable and unexpected outcome.
Therefore,
 Master data is usually non-transactional in nature, but in some cases gray areas exist where
transactional processes and operations may be considered master data by an organization.
 Master data management is needed to resolve the uncertainty of data and to make a single version of truth
across the enterprise, various entities can be identified as suitable candidates for master data.
Ex: master data may contain information about customer data, Product data, location data, employer data etc.
 Identifying candidates for mater data items and managing them is done by Master Data Management.
UNIT - I WEB
INFORMATION MANAGEMENT (2013 SERVERS
REGULATION) , SERVLETS
Definition of Master Data Management:

Alex berson and Larry dubov defined MDM:

“The framework of processes and technologies aimed at creating and maintaining an authoritative, reliable,
sustainable, accurate and secure data environment that represents a single and holistic version of the truth for
master data and its relationships as well as an accepted benchmark used within an enterprise as well as across
enterprises and spanning a diverse set of application system lines of business channels and user communities.”

Ex: Technology solution ERP  Enterprise Resource Planning

 Procurement  logistics Process
 Production
 Distribution
 Accounting
 Human resource
 Corporate performance and Governance
 Customer Service
 Sales
 Business Intelligence
 Enterprise asset management
 E-Commerce
UNIT - I WEB
INFORMATION MANAGEMENT (2013 SERVERS
REGULATION) , SERVLETS
Master Data :
UNIT - I WEB
INFORMATION MANAGEMENT (2013 SERVERS
REGULATION) , SERVLETS
Master Data Management:
 INFORMATION MANAGEMENT (2013 REGULATION)

Key benefits of Master Data Management:

 Lower total cost of ownership and speed time-to-value.
 Increase operational efficiency and confidence in data.

 Improve accuracy and decision making.

Characteristics and benefits of Master Data Management:

1. It provide single version of truth.
2. It provide an increased consistency by reducing redundancy and data discrepancies.
3. It facilitates analysis across departments.
4. It facilitates data governance and data stewardships.
5. It facilitates support for multiple domains.
6. It manages the relationship between domains efficiently.
7. It supports easy configuration and administration of master data entities.
8. It separates master data from individual applications.
9. It acts as a central application independent resource.
10. In simplifies ongoing integration tasks and reduce the development time for new application.
11. It ensures consistent master information across transactional and analytical systems, that is, all systems have
the same information about any entity.
12. It provides safeguards and regulatory compliance.
 INFORMATION MANAGEMENT (2013 REGULATION)

Master Data Management vs Data Warehousing:

Master Data Management:

 Master Data Management (MDM) refers to the process of creating and managing data that an organization

must have as a single master copy, called the master data.

 Usually, master data can include customers, vendors, employees, and products, but can differ by different

industries and even different companies within the same industry.

 MDM is important because it offers the enterprise a single version of the truth. Without a clearly defined

master data, the enterprise runs the risk of having multiple copies of data that are inconsistent with one

another.

Data Warehousing:

 A Data Warehouse is a subject-oriented, integrated, time-variant and non-volatile collection of data in

support of management's decision making process.

 For example, a company that sells software would include sales records, financial data, inventory data in

their data warehouse. Users could query it to find out how much company revenue had increased based on

number of sales per year, or the trend of inventory tax payments over the past 10 years.
 INFORMATION MANAGEMENT (2013 REGULATION)

The main differences between MDM & Data warehousing:

1) Different Goals
 The main purpose of a data warehouse is to analyze data in a multidimensional fashion, while the main

purpose of MDM is to create and maintain a single source of truth for a particular dimension within the

organization. In addition, MDM requires solving the root cause of the inconsistent metadata.
2) Different Types of Data

 Master Data Management is only applied to entities and not transactional data, while a data warehouse

includes data that are both transactional and non-transactional in nature.

3) Different Reporting Needs
 In data warehousing, it is important to deliver to end users the proper types of reports using the proper type

of reporting tool to facilitate analysis. In MDM, the reporting needs are very different -- it is far more

important to be able to provide reports on data governance, data quality, and compliance, rather than

reports based on analytical needs.

 INFORMATION MANAGEMENT (2013 REGULATION)

The different stages involved in MDM implementation are as follows:

1- Identify sources of master data

 Identifying sources that produce master data is an activity which needs to be carried out thoroughly.

2- Identify the provider and consumer of master data

 The application producing master data and the application using master data are identified. (Updation can be done in database level)

3- Collect and analyze metadata for your master data

 The master data entities are identified (Metadata of this master data such as attribute of entities, relationships, constrains, dependencies,

owner of the data entities etc.)

4- Appoint data stewards

 Domain experts having knowledge of the current source data and the ability to determine how to transform the source data into the master

data format have to be appointed.

5- Implement a data governance program and data governance council

 This council is responsible for taking decision with their knowledge and authority. Ex: life span of the data, what are the master data

entities etc. many decisions to be taken throughout the MDM project, this group should have the necessary expertise and knowledge to

visualize the effect on the business and MDM implementation.

6- Develop the master data model

 Complete knowledge of the format of master records, their attributes constrains etc. To develop the master data model, and perform

mapping between the master data model and current data sources. (Perfect balance leads to inconsistence & give optimum performance)
 INFORMATION MANAGEMENT (2013 REGULATION)

7- Choose a toolset:

 Cleaning, transforming and merging the source data to create master list with help of tools already present.

Sometimes there is a need to build the tools along with the infrastructure to maintain the master list.

Two categories of tools:

1- Customer data integration (CDI) Creating the master data of customer

2- Product information management (PIM) Creating the master data for product.

 The toolset should be capable of finding and fixing data quality issues and maintaining version and hierarchies.

8- Design the infrastructure:

 The major concern while designing the infrastructure is maintaining availability, reliability and scalability.

9- Generate and test the master data:

 Interfacing and mapping of proper data source with the master data list is done. This is an iterative and interactive

process.

 After every mapping, results are verified for their correctness which depends on the perfect match of data sources and

master data list.

 If the result is not correct, some settings and rules need to be revised with help of tools and human intervention.
 INFORMATION MANAGEMENT (2013 REGULATION)

10- Modify the producing and consuming systems:

 Depending on MDM implementation design, there might be a need to change or alter the systems that

produce, maintain or consume master data to work with the new source of master data.

 Both master data and other systems communicate each other. (Synchronized communication) to maintain the

consistency of the master data.

 The master data whether used by the source system or any other system should always remain consistent and

updated.

11- Implement the maintenance processes:

 MDM is iterative and incremental in nature. MDM implementations include processes, tools and people for

maintaining data quality.

 All data must have a data steward who is responsible for ensuring the quality of master data.

 The data steward is normally a business person who has knowledge of the data, can recognize incorrect data

and has the knowledge and authority to correct issues.

 INFORMATION MANAGEMENT (2013 REGULATION)

Master data management architectural dimensions:

 The scope of MDM is very broad and complex.

 MDM is multidimensional and comprises of a huge amount of data, various data types and formats, technical
and operational complexities.

To manage and organize these complexities, Need proper classification and categorization is must. this
classification will be helpful in deciding the working principles and providing a MDM solution to respective
domains.

Three types of MDM architectural dimensions:

1. Design and deployment dimension

2. Use pattern dimension

3. Information scope or data domain dimension

 INFORMATION MANAGEMENT (2013 REGULATION)

1- Design and deployment dimension

 This dimension can support MDM implementation, which only refers a single layer to the master data store
includes MDM data hub and data models that manage all the data attributes of a particular domain.
 The MDM data hub is a database with software to manage the master data stored in the database and keep it
synchronized with the transactional systems that use the master data.
 The MDM hub contains functions and tools required to keep the MDM entities and hierarchies consistant
and accurate.
The design and deployment dimension include the following architectural styles:
1. Register style
 The registry style of MDM data hub represents a registry of master entity identifiers that are created using
identity attributes.
 The data hub is responsible for creating and maintaining links with data source to obtain attributes.
The implementation of registry style data hub has the following requirements:

 The data hub should act as a master of unique identifiers.

 It should generate and maintain links with all upstream systems and about the customers’ data automatically.

 It should maintain customer profiles that can be used to generate customer identifiers such as name, address
etc.
 INFORMATION MANAGEMENT (2013 REGULATION)

2. External reference style

 The external reference style of architectural style maintains a MDM reference database that points to all
source data stores.

 Sometimes MDM data hub may not have reference pointer to the actual data of the given domain that put
architectural concern related to external reference style.

In such a case

 The data hub may contain only a reference to the source records that continues to reside on a legancy data
store that needs to be uploaded.

 External reference data hub is that it requires a continues update of interconnection link between the
source system and data hub to maintain valid references to master data at all times.

3. Reconciliation engine style

 The Reconciliation engine style maintains a system of records for all entity attributes.

 It is responsible for providing active synchronization between the MDM data hub and legancy system.

 The Reconciliation engine data hub relies on the source system for maintaining data attributes.

Limitations of this method:

 The master data handled by some applications may have to be changed based on business process.
 INFORMATION MANAGEMENT (2013 REGULATION)

4. Transaction hub style

 The data hub becomes the primary source of records for the entire master data domain with reference
pointer.

 The data hub in this architecture is responsible for maintaining data attributes about the entities, that is the
data hub becomes the master of all entities and attributes.

Therefore,

 The data hub has to manage the complete transactional environment that maintain data integrity.

Limitations of this method:

 The legancy applications may undergo modifications to update the data hub instead of updating existing
legancy system, and it needs synchronization mechanism to propagate the changes from data hub to
system.
 INFORMATION MANAGEMENT (2013 REGULATION)

2- Use pattern dimension

 The Pattern dimension is reusable approach to the solution that has been successfully implemented in the
real world to solve specific problem space.

 This MDM classification is done based on master data usage.

The different patterns of master data :

1. Analytical Master Data Management

 The analytical MDM is composed of different business processes and applications that use master data for
analyzing the business performance, it also provide appropriate reports based on analytics by interfacing
BI (Business Intelligence) tools and package.

 The analytical MDM mostly performs read operation, which does not change or update source data in
operational system.

 The analytical MDM can be integrated with data warehouse to create accurate, cleaned integral view of
master data inside the warehouse and BI tools can be deployed on that cleansed, enriched and integrated
data for performing reporting and data analytics.
 INFORMATION MANAGEMENT (2013 REGULATION)

2. Operational Master Data Management

 The operational MDM is intended to collect and change master data for processing business transactions.

 It is designed to maintain consistency and integrity of master data affected by fractional activity, it is also
responsible for maintaining a single and accurate copy of data maintained in a data hub, which propagates
to other core systems and provide different mechanism for improving data quality in an operational
system.

3. Collaborative Master Data Management

 The collaborative MDM uses a process to create and maintain the master data associated with matadata.

 It also allow users to author the master data objects.

 The collaborative process involves cleaning and updating operations to maintain the accurate master data.
 INFORMATION MANAGEMENT (2013 REGULATION)

3- Information scope or Data domain dimension

 The information scope or data domain dimension deals with the primary data domain managed by the MDM
solution.

The Different domains of MDM are:

 Customer data domain using customer data integration

 Product data domain using product information management

 Organizations data domain using organizations information

The architectural implications of data domain dimensions as follows:

 Privacy and security concern put risk on the given data domain

 Difficult o acquire and manage external reference entity

 Complex design for entity resolution and identification

Before implementing the MDM solution in any organization, it is essential to assess the organization’s enterprise data
management. Assessing and understanding the present mechanisms for MDM, different forms of data governance,
data quality and architecture management, metadata and other data integration mechanism are essential for choosing a
suitable MDM solution for any organization.
 INFORMATION MANAGEMENT (2013 REGULATION)

 The initial assessment is necessary for developing and defining the right MDM solution for enterprise data
management.

 MDM solution includes designing the MDM framework.

 This framework further comprises of evaluating and selecting the appropriate MDM technology, MDM
architecture and development of strategic goals.

The following steps to be followed for implementing MDM:

1- Discovery  Include identifying data sources, defining metadata, modeling business data etc.

2- Analysis Defining rules for transforming and evaluating the data flow, identifying data stewards, refining
and defining metadata and data quality requirement for master data.

3- Construction  MDM database is constructed as per MDMD architecture

4- Implementation  This steps include gathering the master data and it’s metadata according to the subject or
domain, configuring access rights, reviewing the quality levels of the MDM and deciding rules and policies for
change management process

5- Sustainment It sustains the initial iteration of changes done internally to the system along with parallel
deployment of a similar iteration until the whole MDM solution is used.
 INFORMATION MANAGEMENT (2013 REGULATION)

Master Data Management Reference Architecture:

 The MDM reference architecture is an abstraction of technical solution particular problems domain.

 It has a set of services, components and interfaces arranged in functional layers.

 Each layer provides services to layers above it and consumes services from below layer.
 INFORMATION MANAGEMENT (2013 REGULATION)

Layer 5: Business processes layer

 The business process layer deals with management activities.

Layer 4: Data management layer

 This layer responsible for providing many services for data management.

 It is composed of authoring service for creating, managing and approving definitions of master data,
interface service for publishing consistent entry point to MDM service, search service for searching
information, Manipulating and maintaining metadata etc..

Layer 3: Data ruler layer

 It includes key services driven by business defined rules for entity resolution, aggregation, synchronization
etc.

Layer 2: Data quality layer

 This layer responsible for maintaining data quality using various services.

Layer 1: System service layer

 System service layer responsible for providing system level services to above layers.
 INFORMATION MANAGEMENT (2013 REGULATION)

 Creating MDM architecture is not easy with the existing infrastructure. MDM should seamlessly integrate
with the existing infrastructure such as Data Warehousing (DW) , Enterprise Performance Management
(EPM), Business Intelligence (BI) etc.

 To manage the master data across the enterprise for furnishing the right information to the right entity at the
right time.

 In addition, MDM solution has to support data governance. Data governance defines quality rules, access
rights, data definition and standards. By executing these rules MDM facilities data controls across the
enterprise.

Prior to MDM,

There were attempts to transform business from account-centric to entity centric model, various solutions have
been developed:

1. ETL technologies (Extract, Transform, Load)

2. Enterprise Data Warehouse (EDW)

3. Operational Data Store (ODS)

4. Customer Relationship Module (CRM)

5. Enterprise Information Integration (EII) etc.

 INFORMATION MANAGEMENT (2013 REGULATION)

Risk Management:
 In recent times, Most of the businesses had to upgrade their traditional software solutions to modern solution for data
management, storage management and computing.
 This upgrade includes different types of risks associated with them such as financial risk, legal risk, operational risk,
transactional risk, reputational risk, compliance risk etc.
 There is need to recognize and mitigate risks before the business gets affected, to avoid loss.
 Operational/Transactional risk  Transaction processing arising from errors and frauds in credit, interest rate etc.
 Strategic risk  Providing inaccurate and incomplete information that can cause a poor business decision.
 Legal risk or the compliance  unauthorized disclosure of confidential and non-public information about business transaction,
disclosure of consumer compliance etc.
 Creating and management of the core data stores
 Management of processes that implement data governance and data quality
 Metadata management
 Extraction, Transformation and Loading of data from sources to target
 Backup and recovery
 Customer analytics
 Security and visibility
 Synchronization and persistence of data changes
 Transaction management
 Entity matching and generation of unique identifiers
 Resolution of entities and relationships
 INFORMATION MANAGEMENT (2013 REGULATION)

MDM Architecture addresses multiple architectural and management concerns as follow:

 Creating and management of the core data stores

 Management of processes that implement data governance and data quality

 Metadata management

 Extraction, Transformation and Loading of data from sources to target

 Backup and recovery

 Customer analytics

 Security and visibility

 Synchronization and persistence of data changes

 Transaction management

 Entity matching and generation of unique identifiers

 Resolution of entities and relationships