Module 8

Manual Data
Section 1 – Manual Data Acquisition Service (MDAS) 8-3
Lab 16 – Importing Original Data 8-7
Lab 17 – Importing Versioned Data 8-11
8-2 Module 8 – Manual Data

Module Objectives
z Explain and define Old Data
z Explain MDAS and how it acquires data
z Acquire MDAS data

Wonderware Training
 Section 1 – Manual Data Acquisition Service (MDAS) 8-3

Section 1 – Manual Data Acquisition Service (MDAS)

Section Objective
z Explain MDAS and how it acquires data.

This section explains the Manual Data Acquisition Service (MDAS) and how it acquires data.

Introduction
MDAS is designed for ArchestrA and Wonderware Application Server to define tags and store
within the Historian. MDAS provides access to data storage, data retrieval and tag configuration
functionality in the Historian.
MDAS is part of the low-level retrieval system that makes it possible to enter old or legacy data
and provide update capabilities. It provides this capability for all users.
The MDAS service provides rich set of functionality, useful not only to internal Historian clients like
the OLE-DB Provider, but also to Historian external clients.

MDAS Overview
MDAS is a thin client-side internal component that provides high-speed remote access to Historian
for storing data in history (real-time data as well as original or modified old data), retrieving data
from history and configuring the tag database.
The primary benefits for MDAS are:
z Rich functionality
z Performance
z Remote-ability
z Ease of use
The Historian OLE-DB provider is an internal MDAS client.

Data Categories
The Historian distinguishes between two main categories of data, based on the timestamp of the
data and the time it is presented for historization.
z Real-time data: Real-time data is defined as data arriving at the server in time sequential
order, with a time stamp that is within +999 milliseconds/-30 seconds of current server
time.
z Old data: The term "old data" is loosely used to describe all non-real time data. Old data
can arrive at the server at any time, in any order and with any time stamp relative to
current server time (The Historian does not support historization of data in the future, i.e.
when the time stamp is ahead of server time). Old data includes real-time data arriving late
from a store/forward cache.
The Historian allows modification of historized data by either inserting one or more new values into
history after the original data for that time period had been historized, or by updating a value or a
time region of values.
All modifications are versioned, and previous versions are preserved in all cases, allowing the user
to view the original data prior to the modifications.
The term "original data" refers to the original set of values historized for a tag.

Wonderware System Platform Course - Part 2

8-4 Module 8 – Manual Data

For example, a real-time stream of data from an I/O Server represents original data and is written
to disk as the original version. Inserts or updates performed on the original stream of data results
in a new version.
Data modification may be performed for any tag category (I/O Server, system or manual). For
manual tags, new data can be presented as either original data (non-versioned) or inserted data
(versioned).

Note: The Historian supports viewing of only the original version or the latest version of data.
Interim versions are preserved in history but are not exposed through the retrieval layer. Future
releases of the Historian may allow full access to all data versions.

MDAS has full store-and-forward support for real-time data.

MDAS supports storing real-time data for delta tags. Modifying data for both cyclic and delta tags is
supported, as well as cyclic and delta retrieval for all types of tags.

Importing History Data

History data can be imported into Historian extension tables in two ways:
z Manually formatting it in a .csv file according to a specification and then copying the file
into a special folder on the Historian computer.
z Existing InTouch history data can be imported using the InTouch History Importer utility.

.csv File Notes

The .csv file format should be used when small amounts of existing data need modification. The
insert of an entire .csv file results in a single new version of the data. If an inserted data point falls
exactly on an existing timestamp, the existing data value is updated.
For a normal import, the .csv file format and the format of the data contained within the file is very
flexible; however, this flexibility requires the system to perform a large amount of processing on the
data before it can be imported.
Thus, there is an inverse relationship between amount of data to process and import speed. The
time required to process a file is at least exponentially related to the number of values contained in
the file.
Additional considerations for a normal import are:
z The total size of the .csv file is limited to less than 4 MB.
z The .csv file may contain no more than 100,000 values.
z The number of tags represented in the file may not exceed 1000.

Note: Formatting and importing a normal .csv file is demonstrated within the context of Lab 16,
“Importing Original Data”.

Wonderware Training
 Section 1 – Manual Data Acquisition Service (MDAS) 8-5

Fast Load .csv

Using the Fast Load .csv import mechanism, original data can be imported very quickly, using
essentially the same .csv file format as a normal import.
The following information applies to a fast load import scenario:
z A fast load import is much faster than a normal .csv import. For example, a .csv file that is
4 MB will import approximately 100 times faster. For larger files, the speed improvement
gets substantially better. There are no restrictions on the size of the file to import, or the
number of tags or data values in the file. However, the data that is contained in the .csv file
for a fast load import must be formatted in time-sequential order.
It is this ordering that enables the system to process a fast load .csv file more quickly than a
normal .csv file.
z A .csv file formatted for a fast load import is copied into a special \FastLoad import folder.
The fast load import should be used only when it is not feasible to perform a normal .csv import.
Use the following guidelines when considering a fast import:
z You need to import very large .csv files.
z You want storage rules applied to the data you are importing. A normal .csv import does
not apply storage rules; everything is stored as delta.
z Do not use the fast load import if you need to import a large number of small inserts for the
same block. (The fast load import will create a new data stream for each imported fast
load .csv file.)
z Do not import fast load data for a tag if there is existing stored data for that tag in the same
time range.
z The data points must be sorted in time sequential order for a successful fast load import.

Format Differences
The format for the fast load .csv file is essentially the same as the normal format, with the following
exceptions:
z All data in the file is treated as original data.
z The OperationType field in the file header is ignored.
z The Missing Block Behavior field in the header is used for a different purpose. A value of
10 denotes that tags in the file are specified by name, whereas a value of 11 denotes that
the wwTagKey attribute is used to specify tags.
z By default, missing blocks are created as necessary.
z The actual data values in the file must be in time sequential order, starting at the top of the
file.
z Values that have out-of-sequence timestamps are ignored.
In other words, if a data value in the file has a timestamp that is earlier than the timestamp
in the previous line in the file, the data value is discarded, regardless of whether it belongs
to the same tag or a different tag.
z The file should contain only one data value per line.

Wonderware System Platform Course - Part 2

8-6 Module 8 – Manual Data

The following information explains the correct .csv format for use with MDAS:

Line Field Description

Header 1 0 Description of the .CSV file format, either UNICODE or ASCII.
2 0 Character that is used as a delimiter. This can be either a comma ( , ) or a pipe ( | ).
3 0 User name
3 1 Time format: 0 = UTC; 1 = Local.
3 2 Name of the time zone for the timestamps in the .CSV file.
3 If field 1 is set to 0, the value of this field is ignored. However, a value that is at least 1
character long must appear for this field.
3 If field 1 is set to 1, the value of this field should be the literal name of the time zone, as it
is specified in the TimeZone table.
3 To use the local time of the IndustrialSQL Server Historian, specify "Server Local" for this
field.
3 3 Default behavior if a missing history block is encountered: 0 = Do not create replacement
blocks; 1 = Create replacement blocks. If this field is set to 0, and data values in the .CSV
fall within a time period where no blocks are present, then no data will be inserted.
3 4 Time span for the replacement block(s): 0 = Recreate block from first data value to the
present time; 1 = Recreate block for the duration of the .CSV file data.
3 If field 3 is set to 0, then the value of this field is ignored.
3 If this field is set to 0, then history blocks will be recreated from the time of the first value to
the current time.
3 If this field is set to 1, only the history blocks necessary to import all data in file will be rec-
reated.
3 The start and end times for the history block is adjusted based on the current block dura-
tion setting or the start and end times of existing blocks.
Value 4…n 0 Tagname
4…n 1 Type of operation: 0 = Original value; 1 = Insert; 2 = Update; 3 = Multi-point update.
4…n 2 Value start date, formatted as: YYYY/MM/DD
4…n 3 Value start time, formatted as: HH:MM:SS.MSEC
4…n 4 (This field is used only in the case of an update.) Value end date, formatted as: YYYY/MM/
DD
4…n 5 (This field is used only in the case of an update.) Value end time, formatted as:
HH:MM:SS.MSEC
4…n 6 Used to indicate whether the value is already in the proper format or whether you want to
scale the value after the import. Values are: 0 = Engineering units; 1 = Raw value.
4…n 7 Value
4…n 8…n QualityDetail. Fields 2 through 8 can be repeated.

If two multipoint update .csv files for the same tag are simultaneously copied to the
\DataImport directory, the update will span across the total time for the two files. A query
returning latest data will hide (mask) the original version of the data from the end of the first file
to the start of the second file.
For example, if the update in one file ranges from 00:00:00 to 00:05:00, and the other ranges
from 00:10:00 to 00:15:00, the result will be an update starting at 00:00:00 and ending at
00:15:00 ("latest"); the original data from 00:05:00 to 00:10:00 will be masked as "original"
data.
No data is lost. To view either data from a query, use the wwVersion column to specify either
"original" or "latest." By default, the latest data is displayed. To prevent the masking of the original
data, process the .CSV files one at a time.

Wonderware Training
 Lab 16 – Importing Original Data 8-7

Lab 16 – Importing Original Data

Introduction
This lab demonstrates how the Historian MDAS Acquisition engine takes data from a .csv file and
"pushes" it (automatically inserts it) into a viewable Historian tag. It is then available for analysis by
Query Analyzer or ActiveFactory Trend.
Participants use the ArchestrA System Management Console, MS Explorer, ActiveFactory Query,
and ActiveFactory Trend applications.

Objective
Upon completion of this lab you will be able to:
z Use a .csv File to insert data values into a Historian tag as original data
z View the Results Using Microsoft SQL Server Management Studio
z View the Results Using ActiveFactory Trend

Wonderware System Platform Course - Part 2

8-8 Module 8 – Manual Data

Prepare the Import .csv file.

In this section, you will insert original data for the LIT_001.PV tag into a time period where no data
was available.
1. Start Windows Explorer.
2. Browse to the MixerTemp-Original.csv file located in the C:\Wonderware Training folder.
Your instructor will update the file with a date from last week. The objective is to use a date
where there is no data in the LIT_001.PV tag.
3. Right-click the file and select Open With…. The following figure shows the menu that appears
when the file has been opened previously (Notepad is in the submenu that appears).

4. If the file has not been opened before, scroll down to Notepad and highlight it.

5. Ensure the Always open using this program option is de-selected.

6. Click OK.

Note: Do not open this file with Excel!

Excel will change the datetime format and make the file unusable.
If the file is opened with Excel, close it without saving it.

This file contains values changing every second for the LIT_001.PV tag. There is no data available
for this tag in the history blocks within the period shown in the file, so this data will be inserted as
original data.

Wonderware Training
 Lab 16 – Importing Original Data 8-9

7. Close the file.

Notice that the file uses “0” in

the second column of the data
rows, which means the data will
be inserted as original data.

8. Copy the file to C:\InSQL\Data\DataImport.

After a moment the file will disappear.
9. Start ActiveFactory Query.
10. Open the VersionedData.sql file in C:\Wonderware Training.
Your instructor will have updated the file with the same dates used in the MixerTemp-
Original.csv file.
In the SQL tab, notice that both versions of data are being queried: original and latest.
11. Click the Data tab.

Wonderware System Platform Course - Part 2

8-10 Module 8 – Manual Data

Both versions retrieved the same value for each instance in time.

View the results in ActiveFactory Trend.

12. Start ActiveFactory Trend (keep ActiveFactory Query open).
13. Add the LIT_001.PV tag to the Trend.
14. Select the time range that reflects the date used in the last few steps.
15. Select Tools / Options.
16. Change the Retrieval style to BestFit-5.
17. Change the Interpolation type to Linear.
The Latest data for the LIT_001.PV tag displays.
18. Leave ActiveFactory Trend and Query open for the next lab.

Wonderware Training