Technical Requirements Document

Design Weight Inputs Loads

Program Name: Warehouse Convergence

Project Name: MCH data loads
Product Name: Diary Design Weight Input Data Loads
Program Manager: Brian Charlow
Project Lead: Deepak Abburi
Technical Lead: Jack Vorsteg
SA Indira Kalvakolanu
ID#:
Module Weights
 Document Version History Log

Version Date
Number Author Version Date Description of Changes Distributed

0.1 Indira Kalvakolanu 04/17/2014 Draft Version 04/21/2014

PREFACE

Purpose of this Document

This document describes technical requirements for the Nielsen Audio Design Input data loads
module of Warehouse Convergence project. This document is a comprehensive package to
support technical requirements, high-level design and process flows for various components and
areas as needed to support the approved program/ project.
o This documents scope will include high-level or conceptual design details
o Impacted business areas must be considered for content input
o Enterprise Architecture to provide direction and support
o The program/ project plans, a Solution Requirements Package and other applicable
approved documentation will be key drivers to building this package
The content of this Package is a key driver to updating the program/ project estimates.
The contents will drive the build of the approved solution.
Using this Document
The following guidelines are to be followed while working on this document:
The template must be initiated at the beginning of the BPQ Design Phase and completed
during the Design Phase. The work-in-progress and completed document shall be stored in
the approved program/ project repository. If an approved change requires an update,
complete the required, secure sign-off, post the updated document back into the
repository and notify key stakeholders.
The document is divided into several sections; each section includes specific instructions on
the content required for that section.
o If a section in the document is not required or not applicable, leave the section in
tact (including the <metadata> and type Not Applicable and include a brief
explanation in the next line.
o If a section needs to be added to the document, be sure to update the TOC.
o To update the Table of Contents (TOC): right click on the TOC and select the
Update Field option; then select the Update Entire table option. Click on OK to
complete the process. Review the TOC to learn if any reformatting changes will be
required within the core document. If needed, correct then repeat the TOC update
instructions until clean. Perform this after all changes have been made to the
document.>

References
The following references have been referred to or used as input while preparing this document:
 Reference Description Location
Name
https://ishare.nielsen.com/teams/Port_Man2/ANSI/Pr
Project Plan Provides a baseline on
oject%20Management%20Documents/Nielsen%20Audi
the project
o%20[MCH-
Foundations]%20Convergence%20Plan.mpp
https://ishare.nielsen.com/teams/Port_Man2/ANSI/BP
Data Source Provides the input
Q%20Documentation/Forms/AllItems.aspx
details
Link to the data model \\nmrflpntldtg1\Projects\MediaContentHub\Audio.pd
Data Model
location m
 Table of Contents

1. Nielsen Audio Design Weights Input Data Loads .............................................................................. 6

1.1 Technical Requirements ............................................................................................................... 6
1.1.1 Design Weight Input Data Extracts ........................................................................................... 6
1.1.1.1 Source Data ...................................................................................................................... 6
1.1.1.2 File Location ...................................................................................................................... 6
1.1.1.3 File Format ........................................................................................................................ 7
1.1.2 MCH Loads ............................................................................................................................... 7
1.1.2.1 Regular Loads ................................................................................................................... 7
1.1.2.1.1 WGHTNG_DSGN_GRP ................................................................................................ 8
1.1.2.1.2 WGHTNG_DSGN_FRM ................................................................................................ 8
1.1.2.1.3 WGHTNG_DSGN_FCTR .............................................................................................. 8
1.1.2.2 Delete Strategy.................................................................................................................. 8
1.1.2.3 Target Tables .................................................................................................................... 9
1.1.2.4 Order of Data loads ........................................................................................................... 9
1.1.2.5 Successful Loads .............................................................................................................. 9
1.1.2.6 Load Failures and Error Handling ..................................................................................... 9
1.1.3 MCH Load Dependencies ......................................................................................................... 9
1.1.4 MCH Load Triggers ................................................................................................................. 10
1.1.5 High-Level Data Flow Diagram ............................................................................................... 10
1.1.6 Detailed Technical Requirements ........................................................................................... 10
2. Appendix .......................................................................................................................................... 11
3. Reviews and Sign-Off....................................................................................................................... 12
DWIS Inputs-MCH Technical Requirements Document

1. Nielsen Audio Design Weights Input Data Loads

1.1 Technical Requirements
1.1.1 Design Weight Input Data Extracts

MCH will extract Design Weights Input data from UDP database and load the same into the MCH data
structure. In the Diary world, markets are closed out for each phase in a given quarterly survey before
the data can be reported. Markets are usually closed out in groups. As the markets are closed out,
county level Design Weight Input data is sourced from SMS and loaded into the UDP tables.

1.1.1.1 Source Data

Following are the source UDP tables used for the Design Weight Input data. The Reference data related
for Group and Frame Components is loaded into MCH as one time only (OTO). Only Design Weight
Factors data is extracted by Foundations on a regular basis and will be loaded into the MCH.

Source Data Source Table Name

Design Weight Frame components Reference UDP_FRAME_COMPONENT
Design Weight Group components Reference UDP_GROUP_COMPONENT
Design Weights Component Factors UDP_DESIGN_WGHT_CMPNT_FACTOR

1.1.1.2 File Location

Foundations will place the Design Weight Inputs data files into the MCH incoming directory. The MCH
directory is as follows for each environment:

o Dev: dayrheorad002: /odi/mchnas/dev/aud/<two digit country>/Incoming

o QA : dayrheorad003: /odi/mchnas/qa/aud/<two digit country>/ Incoming
o UAT: dayrheorau003: /odi/mchnas/qa/aud/<two digit country>/ Incoming
o Prod: dayrhemchp008: /odi/mchnas/prod/aud/<two digit country>/ Incoming
<two digit country>
us for USA
jp for Japan

File Name Convention:

Data files:
Rules that apply to all the files are as follows:
o marketID (in the file name) Indicates the 2 letter ISO country code
US Indicates USA, JP Indicates Japan
o yearID is year value like 2014
o periodID is Legacy Quarter Code value like 01 or 13
o fileCreationDateTime: Date and Time of File Creation in
yyyymmdd_hhmmss format

Page 6 of 12
DWIS Inputs-MCH Technical Requirements Document

Example Data File Names:

o audio_dwis_fctr_y<yearID> p<periodID>_m<marketID>_f<filecreationDateTime>.txt

1.1.1.3 File Format

Each data extract file will have following format and data elements:

o No header records with column names in the extract files.

o Each data value is Pipe delimited |.
o Extracted files are in text format.
o The format of the fields in the data files that are of time type should be YYYY-MM-DD
HH:MM:SS. Milliseconds shall not be included.
o The format of the fields in the data files that are of date type should be YYYY-MM-DD.

Design Weight Component Factors:

audio_dwis_fctr_y<2014> p<02>_m<us>_f<20140417_120256>.txt

SOURCE DATA DATA TYPE EXAMPLE

VALUES
RPT_PERIOD_YEAR NUMBER 2014
LEGACY_QTR_CODE NUMBER 01 or 02 (Interim)
13 (Quarterly)
ARBSU_ID NUMBER 01009
FRAME_CMPNT_ID VARCHAR OTHABS
(50 Byte)
FRAME_COMPONENT_DESC VARCHAR Other ABS Frames
(100 Byte)
GROUP_CMPNT_ID VARCHAR EST_USABLE_HH
(50 Byte)
GROUP_COMPONENT_DESC VARCHAR Estimated Usable
(100 Byte) HHs
DW_FACTOR NUMBER 244.627891844997

Example Record:
RPT_PERIOD_YEAR|LEGACY_QTR_CODE|ARBSU_ID|FRAME_CMPNT_ID| FRAME_COMPONENT_DESC
|GROUP_CMPNT_ID| GROUP_COMPONENT_DESC |DW_FACTOR

2014|02|01009|OTHABS| Other ABS Frames |EST_USABLE_HH| Estimated Usable HHs |

244.627891844997

1.1.2 MCH Loads

1.1.2.1 Regular Loads

MCH extract process will be triggered by the Webservice call. The Design Weight Input data file,
at the county level, is available after each phase closeout of Diary markets for a given survey.
The load process will populate data to the MCH target table. Following load strategies will be
applied to different Design Weights data:

Source Data Target Table Load Strategy

Page 7 of 12
DWIS Inputs-MCH Technical Requirements Document

Design Weight Frame components Reference Insert

Design Weight Group components Reference Insert
Design Weights Component Factors Insert, and/or Delete

1.1.2.1.1 WGHTNG_DSGN_GRP

The Design Weight Group reference data is loaded into the WGHTNG_DSGN_GRP table
as OTO. This data is not expected to change frequently.
During Design Weight Factors load, consider the distinct Group Component ID and
Group Description combination and check against the target MCH table. If a given
Group combination exists do nothing otherwise insert the new data into the target
table.

1.1.2.1.2 WGHTNG_DSGN_FRM

The Design Weight Frame reference data is loaded into the WGHTNG_DSGN_FRM table
as the extract is available. This data is not expected to change frequently.
During Design Weight Factors load, consider the distinct Frame Component ID and
Frame Description combination and check against the target MCH table. If a given
Frame combination exists do nothing otherwise insert the new data into the target
table.

1.1.2.1.3 WGHTNG_DSGN_FCTR

The Design Weight Factors data is loaded into the WGHTNG_DSGN_FCTR table as the
data extract is available for each closed out counties.
All newly inserted data will have:
o Same Insert and Update Date and timestamp for each record.
o The Load ID for inserted for each record.
o All records are flagged as Y.
For a given Reporting Year/Legacy Quarter Code/Sample Go ID context, if the Design
Weight Factors data within the extract already exists within the MCH then:
o The data is logically deleted by flagging each record as inactive N.
o Update timestamp value is updated
o The Load ID for reprocessing is inserted.
o Incoming new data will be inserted.

1.1.2.2 Delete Strategy

There will be no delete processing for Design Weight Group component reference data
within MCH.
There will be no delete processing for Design Weight Frame component reference data
within MCH.
During load process for Design Weight Factors data, if data already exists within the
MCH for a given Report Period Year/ Legacy QTR Code/ ARBSU ID context then the load
process would perform a logical delete of this data first and then insert the incoming
data.

Page 8 of 12
DWIS Inputs-MCH Technical Requirements Document

1.1.2.3 Target Tables

Following are the Target tables:

Data Extract Name MCH Target Table Name

audio_dwis_fctr_y<2014> p<02>_m<us>_f<20140417_120256> WGHTNG_DSGN_GRP
WGHTNG_DSGN_FRM
WGHTNG_DSGN_FCTR

1.1.2.4 Order of Data loads

Target Table Order Sequence Dependent Load

WGHTNG_DSGN_GRP 1 None
WGHTNG_DSGN_FRM 1 None
WGHTNG_DSGN_FCTR 2 WGHTNG_DSGN_FRM
WGHTNG_DSGN_GRP

1.1.2.5 Successful Loads

The target tables shall be populated successfully on load completion. The following actions shall be
performed on successful load:
Insert Job completion status into the audit table.
Archive the source data files to a specific directory as listed below:
o Dev: dayrheorad002: /odi/mchnas/dev/aud/<two digit country>/Archive/DWIS
o QA : dayrheorad003: /odi/mchnas/qa/aud/<two digit country>/Archive/DWIS
o UAT: dayrheorad003: /odi/mchnas/qa/aud/<two digit country>/Archive/DWIS
o Prod: : dayrhemchp008: /odi/mchnas/prod/aud/<two digit country>/Archive/DWIS
<two digit country>
us for USA
jp for Japan

1.1.2.6 Load Failures and Error Handling

If data loads fail or error out, system shall perform the following actions:
Rollback to the previous state and no partial data commits allowed
Insert failure status into the audit table
Move the Source Extract files to Error Directory for analysis
Send an email notification to support group.
The Error tables shall get updated and not the Final tables on load failures.

1.1.3 MCH Load Dependencies

None.

Page 9 of 12
DWIS Inputs-MCH Technical Requirements Document

1.1.4 MCH Load Triggers

MCH load process will be triggered by the Webservice call.

1.1.5 High-Level Data Flow Diagram

DWIS_MCH.vsd

1.1.6 Detailed Technical Requirements

The detailed technical requirements are provided in the Technical Requirements Spreadsheet (TRS)
below.

Arb_Nielsen
Convergence DWIS_Input_Loads - Mapping.xls

Page 10 of 12
DWIS Inputs-MCH Technical Requirements Document

2. Appendix
A. Definitions, Acronyms, and Abbreviations
The following acronyms and abbreviations have been used in this document: The general definitions
are provided in the SRP.

Definition/Acronym/ Description
Abbreviation

MCH Media Content Hub

UDP Unified Data Prep
DWIS Design Weight Input System
SMS Sample Management System
LETS Last Extract Time Stamp

B. Assumptions
The assumptions are as follows:

Foundations will extract Frame and Group reference data based on LETS value.
Foundations will extract Frame component Factors data based on Close out parameter
value.
Foundations will provide MCH data values wherever possible.

C. Notes
None

D. Risks and Constraints

None

Page 11 of 12
DWIS Inputs-MCH Technical Requirements Document

3. Reviews and Sign-Off

Program Manager * Signature:
Print Name: Brian Charlow
Title: Program Manager
Date:
Project Lead* Signature:
Print Name: Deepak Abburi
Title: Technical Manager
Date:
Data Architect Signature:
Print Name: Doug Whittacre
Title: Data Architect
Date:
Solution Architect* Signature:
Print Name: Jack Vorsteg
Title: Solution Architect
Date:
Business Analyst Signature:
Print Name: Thiagu Sethuraman
Title: Business Analyst
Date:
Development Lead
Signature:
Print Name: Sangeeth Kumar
Title: Development Lead
Date:
QA Lead
Signature:
Print Name: Brian Lupson
Title: QA Manager
Date:
Table 1: Requirement Approvals

Page 12 of 12