Professional Documents
Culture Documents
9 Utilities
http://it.nees.org/library/telepresence/data-turbine-39-utilities.php
Library /
Contents
1. Overview 2. Using the NEESit Data Turbine Utilities Automated Archive AxisSource DaqToRbnb DLinkSource FileToRbnb FlexTpsSource JpgLoaderSource JpgSaverSink OpenSeesToRbnb Orb2Rbnb RbnbToFile 3. Testing Your Data Turbine Sources 4. Obtaining the Source
Code
1 of 24
1/7/08 3:47 PM
http://it.nees.org/library/telepresence/data-turbine-39-utilities.php
the ring buffer prior to starting sink applications, such applications are typically only useful when data is present. See Getting Started With Data Turbine for further information on setting up and running a Data Turbine server. To prevent naming conflicts, ensure each source and channel have distinct, meaningful names. In the future, RDV will be able to hide source names and display only channel names in an effort to more efficiently use available screen real estate. Therefore, each channel should be given a distinct, meaningful name that does not rely on the source name to provide cues as to the origin of the data. This is especially true for DAQ channels and video channels. For DAQ input, ensure each channel has a globally distinct name. This is especially important to consider when using more than one DAQ server. The default channel name for video channels is the rather non-descriptive "video.jpg." NEESit strongly recommends assigning a more descriptive name to video channels when starting either AxisSource or DLinkSource. For video it may be desireable to assign the same name to both the source and the channel. References to a source application's ring buffer are left in memory when the source application exits. This allows users to continue to view historical data after the source is no longer active. Reusing a source name without first terminating the original source will cause a duplicate source to be created, followed by an underscore and the next available integer. To avoid this problem, use Creare's admin.jar (described in
$RBNB_HOME/Documentation/Admin/rbnbAdmin.html) to
terminate the old source before reusing its name. To ensure all data channels contain the same amount of information (e.g., video and DAQ sensor channels both contain two hours of data) you can either use the -r option, or you can perform manual calculations to determine the appropriate number of frames to archive for each source application. Manual calculations require access to the application's source code and can be performed as follows:
2 of 24
1/7/08 3:47 PM
http://it.nees.org/library/telepresence/data-turbine-39-utilities.php
number-of-frames Number of frames to archive; supply this number in conjunction with the -Z (uppercase) option. time-in-seconds The amount of data to be stored, in number of seconds. For example, two hours is represented as 7200 (60 sec/min 60 min/hr 2 hrs). flush-rate-in-Hz The rate at which Data Turbine frames are created, in hertz. A Data Turbine frame is created every time a
flush() call is made in the source code.
The following parameters apply to all NEESit Data Turbine utilities, except Orb2Rbnb:
Option
-h
Required? no
Description Prints help message describing command arguments and default values.
-p [port]
no
3333
-s
no
localhost
[hostname]
-v
no
none
[contents]
3 of 24
1/7/08 3:47 PM
http://it.nees.org/library/telepresence/data-turbine-39-utilities.php
designed to archive the sample data derived from an experiment and to upload this data to the NEEScentral data repository. The goal behind developing this system was to forego the manual archival process that was necessary prior to the release of Automated Archive. This operation is not limited only to an experiments numeric data, but also applies to uploading and archiving video files in NEEScentral. Users have the option of specifying any combination of sources, channels, start/end times, and event markers for the archival and upload. Automated Archive also streamlines the process by which data can be uploaded into NEEScentral for the purposes of integration with playback and replay tools.
Option
-C [string]
Required? no
Description Source names or channel names (comma-separated). Wild card may be used when multiple channels should be selected.
-S [time]
no
none
When this parameter is specified, only data starting from this time is retrieved.
-E [time]
no
none
no
none
Event Markers specified as key=value. The keys are either "source" or "content." The values are the corresponding values for each key. Multiple values for a key may be specified as long as they are separated by a plus sign (+), in which
[key2=value2+value3]
4 of 24
1/7/08 3:47 PM
http://it.nees.org/library/telepresence/data-turbine-39-utilities.php
Option
Required?
Default Value
-U [string]
yes
none
yes yes
none none
The password for this user. The destination directory on NEEScentral to which the uploaded files will be stored. The user must have permission to write to this path. The user will be prompted as to whether they wish to overwrite the files on the server if they already exist in this path.
-f [string]
no
data.txt
Name of the data file to upload to NEEScentral (for numeric channels only).
-x [string]
no
Prefix name to add to the defaul name of the video channel zip file to upload to NEEScentral (for video channels only).
-B
no
zero
Number of seconds from now to go back to set a start time for channels upload to NEEScentral.
[contents]
2.2. AxisSource
AxisSource acquires data from an Axis camera or video server and puts it into the ring buffer. By default, video is acquired at a resolution of 704480 pixels at a rate of 30 frames per second.
5 of 24
1/7/08 3:47 PM
http://it.nees.org/library/telepresence/data-turbine-39-utilities.php
Option
-A
Required? yes
[hostname]
-S [string]
no
AxisVideo
becomes the name of the archive file and should be noted if you want to reload data into the ring buffer at a later time.
-C [string] -R [string]
no no
video.jpg
Channel name. Selects an arbitrary resolution. The supplied string is sent directly to the camera; check your camera's manual to determine what resolutions are supported.
none
-n -f
no no
1 30
-U [string]
no
none
-P [string]
no
none
6 of 24
1/7/08 3:47 PM
http://it.nees.org/library/telepresence/data-turbine-39-utilities.php
Option
-z [integer]
Required? no
Default Value
900
-Z [integer]
no
-r [double]
no
none
Data Turbine archive size, in number of hours. This option is ignored if -Z is specified. Used in conjunction with
-m.
-m [double]
no
10
Data Turbine cache size as a percentage of the calculated archive size when -r is specified. This option is only used in conjunction with
-r.
[contents]
2.3. DaqToRbnb
DaqToRbnb connects to a DAQ server, subscribes to all data channels, converts the timestamps from ISO-8601 format to Java format, then sends the data to the Data Turbine server.
Option
-q
Required? no
Default Value
localhost
Description IP address or
7 of 24
1/7/08 3:47 PM
http://it.nees.org/library/telepresence/data-turbine-39-utilities.php
Option [hostname]
-c [port]
Required?
Default Value
no
55055
-d [port]
no
55056
-n [string]
no
FromDAQ
This also becomes the name of the archive file and should be noted if you want to reload data into the ring buffer at a later time.
-o [float]
no
none
Time offset, in hours. Only needed if the DAQ server and Data Turbine server are in different time zones. Due to limitations in LabVIEW, the DAQ timestamps do not include time zone information; this offset converts the DAQ timestamps from local time (on the DAQ server) to GMT.
-T
no
none
8 of 24
1/7/08 3:47 PM
http://it.nees.org/library/telepresence/data-turbine-39-utilities.php
Option
Required?
Default Value
Description number of samples) to decide when to flush records to the Data Turbine server. This is the default mode of operation; a Data Turbine frame is created every 100 milliseconds (10 Hz).
-t [long]
no
100
Number of milliseconds between data flushes. Used in conjunction with the -T option.
-K
no
none
Flag to use number of samples (as opposed to a set period of time) to decide when to flush records to the Data Turbine server. If -k is not also specified, a Data Turbine frame is created every 100 samples.
-k [long]
no
100
Number of samples to collect before flushing data to the Data Turbine server. Used in conjunction with the -K option.
9 of 24
1/7/08 3:47 PM
http://it.nees.org/library/telepresence/data-turbine-39-utilities.php
Option
-D
Required? no
-z [integer]
no
900
-Z [integer]
no
-r [double]
no
none
Data Turbine archive size, in number of hours. This option is ignored if -Z is specified. Used in conjunction with
-m.
-m [double]
no
10
Data Turbine cache size as a percentage of the calculated archive size when -r is specified. This option is only used in conjunction with
-r.
[contents]
2.4. DLinkSource
DLinkSource acquires data from a D-Link DCS-900 camera and sends it to the Data Turbine server.
10 of 24
1/7/08 3:47 PM
http://it.nees.org/library/telepresence/data-turbine-39-utilities.php
Option
-A
Required? yes
[hostname]
-S [string]
no
DLinkVideo
also becomes the name of the archive file and should be noted if you want to reload data into the ring buffer at a later time.
-C [string] -U [string]
no no
video.jpg
none
-P [string]
no
none
-z [integer]
no
900
-Z [integer]
no
-r [double]
no
none
Data Turbine archive size, in number of hours. This option is ignored if -Z is specified. Used
11 of 24
1/7/08 3:47 PM
http://it.nees.org/library/telepresence/data-turbine-39-utilities.php
Option
Required?
Default Value
-m [double]
no
10
Data Turbine cache size as a percentage of the calculated archive size when -r is specified. This option is only used in conjunction with -r.
[contents]
2.5. FileToRbnb
FileToRbnb reads from a DAQ-formatted ASCII file and sends the contents to the Data Turbine server. The input file begins with a metadata header that describes the active channels and their units. Each line following the metadata header represents a single point in time (always in GMT) and contains the value of each channel at that point in time. Values are seperated by tab characters (\t) and missing values are indicated with a sequence of two tabs in a row with no intervening spaces (i.e., \t\t). Note the use of scientific notation for the values. Below is a short sample file. To use the file for test purposes, replace all \t's with actual tab characters before importing it into the Data Turbine server.
Active channels: ATL1,ATT1,ATL3,ATT3 Channel units: g,g,in,kip
12 of 24
1/7/08 3:47 PM
http://it.nees.org/library/telepresence/data-turbine-39-utilities.php
Option
-n
Required? no
Default Value
FileDump
[string]
This also becomes the name of the archive file and should be noted if you want to reload data into the ring buffer at a later time.
-c
no
data.txt
Channel name.
[string]
-d
no
[string]
-f
no
Data.txt
[string]
-S
no
\t
Delimiter used within the input file to separate data items. \t represents a tab character.
[string]
-R [long]
no
Number of milliseconds to wait before sending the next data sample to the Data Turbine server. Leaving -R set to 0 causes the data to be sent to the Data Turbine server at full speed.
-Q
no
none
13 of 24
1/7/08 3:47 PM
http://it.nees.org/library/telepresence/data-turbine-39-utilities.php
Option
Required?
Default Value
Description channel created by this utility to close when the utility exits.
-o
no
none
[double]
-z
no
900
[integer]
-Z
no
1800
[integer]
[contents]
2.6. FlexTpsSource
FlexTpsSource acquires data from a FlexTPS video stream and sends it to the Data Turbine server. Unlike the other NEESit source utilities, FlexTpsSource does not currently make use of append mode. This means that every time FlexTpsSource is started with a non-unique name, the name will be followed by an underscore and the next available integer (e.g., MySource_4) regardless of whether the previous source with the same name has been terminated. This behavior will be fixed in a future release of FlexTpsSource.
Option
-A
Description IP address or hostname of the FlexTPS source. This option cannot be used in conjunction
[hostname]
14 of 24
1/7/08 3:47 PM
http://it.nees.org/library/telepresence/data-turbine-39-utilities.php
Option
Required?
Default Value
-F [int]
no
10
Video frame rate, in frames per second. This option cannot be used in conjunction with -R.
-N [string]
none
Name of the FlexTPS stream. This option cannot be used in conjunction with -R.
-T [string]
none
Name of the FlexTPS feed. This option cannot be used in conjunction with -R.
-V [string]
no
1.x.x
FlexTPS version number, either 1.x.x or 0.4.x. This option is ignored if -R is supplied.
-R [string]
yes, if -A,
-N, and -T
none
URL of the FlexTPS video source. Robotic PTZ camera control is not allowed by Data Turbine clients (e.g., RDV) if this option is used; use -A, -F,
-N, -T, and -V
15 of 24
1/7/08 3:47 PM
http://it.nees.org/library/telepresence/data-turbine-39-utilities.php
Option
Required?
Default Value
-U [string]
no
none
-P [string]
no
none
-S [string]
no
TpsVideo
becomes the name of the archive file and should be noted if you want to reload data into the ring buffer at a later time.
-C [string] -z [integer]
no no
video.jpg 900
Channel name. Data Turbine cache size, in number of Data Turbine frames.
-Z [integer]
no
-r [double]
no
none
16 of 24
1/7/08 3:47 PM
http://it.nees.org/library/telepresence/data-turbine-39-utilities.php
Option
Required?
Default Value
-m [double]
no
10
Data Turbine cache size as a percentage of the calculated archive size when -r is specified. This option is only used in conjunction with
-r.
[contents]
2.7. JpgLoaderSource
JpgLoaderSource sends timestamped JPEG images within a specified time range from disk to the Data Turbine server. JpgLoaderSource expects the image files in the user-supplied base directory to be labeled and organized in a manner consistent with the file system structure used by JpgSaverSink.
Option
-n
Required? no
Default Value
Archive
[string]
This also becomes the name of the archive file and should be noted if you want to reload data into the ring
17 of 24
1/7/08 3:47 PM
http://it.nees.org/library/telepresence/data-turbine-39-utilities.php
Option
Required?
Default Value
-c
no
video.jpg
Channel name.
[string]
-d
no
JpgStore
[string]
-X
no
none
Flag used to print a list of the files that would be sent to the Data Turbine server; no files are actually sent.
-S
no
[string]
-E
no
-z
no
1024
[integer]
-Z
no
[integer]
Data Turbine archive size, in number of Data Turbine frames. If an archive size is not provided, it will be calculated automatically.
[contents]
2.8. JpgSaverSink
JpgSaverSink requests images from the Data Turbine server and saves them to disk (independent of the Data Turbine ring
18 of 24
1/7/08 3:47 PM
http://it.nees.org/library/telepresence/data-turbine-39-utilities.php
buffer) so they can be backed up or otherwise manipulated. Images in the ring buffer whose timestamps (in YYYY-MM-dd:hh:mm:ss.nnn format) fall between the user-supplied start and end times are saved as files named
YYYYMMddhhmmssnnn.jpg in the directory <base-directory>/YYYY/MM/dd/hh/mm. Images saved to disk by
JpgSaverSink can be reloaded into the Data Turbine ring buffer with JpgLoaderSource. They can also be imported directly into RDV for offline viewing.
Option
-k
Required? no
Default Value
JpgSaver
[string]
-n
no
VideoSource
[string] no
-c
video.jpg
[string] no
-d
JpgStore
[string]
Path of the base directory to which the requested images will be saved.
-S
no
[string]
-E
no
[string]
[contents]
2.9. OpenSeesToRbnb
This utility reads an OpenSees xml file in the format per XSD:
19 of 24
1/7/08 3:47 PM
http://it.nees.org/library/telepresence/data-turbine-39-utilities.php
http://opensees.berkeley.edu//xml-schema/xmlns/OpenSees.xsd and imports the data to Data Turbine server. The format has all metadata header as elements which are parsed to create the header and all data under one element which is parsed and imported to the server
Option
-R
Required? no
Default Value 0
Description Number of milliseconds to wait before sending the next data sample to the Data Turbine server. Leaving -R set to 0 causes the data to be sent to the Data Turbine server at full speed.
[long]
-f
yes
none
[string] [contents]
2.10. Orb2Rbnb
Orb2Rbnb requests data from an Antelope ORB, translates it into a form compatible with RDV, and transmits it to a Data Turbine server. Due to network latency, viewing Antelope ORB data in real-time mode within RDV may fail and result in a blank screen. If real-time mode does not work, use playback mode instead to ensure the Antelope data is available within the Data Turbine server. Note: Orb2Rbnb does not implement the four standard options (-h, -p, -s, -v) described in section 2.
20 of 24
1/7/08 3:47 PM
http://it.nees.org/library/telepresence/data-turbine-39-utilities.php
Option
-a
Required? no
-d
no
none
Print debugging output. Print extremely verbose debugging data. Use in conjunction with -d.
-D
no
none
-f
no
none
[string]
Regular expression used by the Antelope ORB filter for selecting ORB data channels.
-o
yes
none
[string]
The fully-qualified domain name and port of the ORB (e.g., <fqdn>:<port>.
-S
no
none
Scale factor.
[double]
-r
yes
localhost
Fully-qualified domain name of the Data Turbine server. Do not append a port number; 3333 is assumed.
-s
no
none
[double] no none
-v
Print version information. Data Turbine cache size, in number of Data Turbine frames.
-z
no
1024
[integer]
-Z
no
10240
[integer]
21 of 24
1/7/08 3:47 PM
http://it.nees.org/library/telepresence/data-turbine-39-utilities.php
Option
Required?
Default Value
[contents]
2.11. RbnbToFile
RbnbToFile extracts numeric data from channels of interest and creates an ASCII-formatted output file. This file conforms to standard data files as described in the FileToRbnb section. The output file consists of a metadata header, followed by lines of data. The header lists channels and their units of measurement. Directly following the header, the lines of data utilize consecutive time increments that correspond to the data output by the channels.
Option
-C
Required? yes
[string] no none
-S
Start time from which to commence data extraction. (Needs to be GMT and in ISO8601 format, e.g., 2007-11-13T15:48:55.26499.)
[string]
-E
no
none
End time at which to cease data extraction. (Needs to be GMT and in ISO8601 format, e.g., 2007-11-13T15:48:55.26499.)
[string]
-M
no
none
[string]
-f
no
data.txt
[string]
-k
no
RbnbToFile
[string]
22 of 24
1/7/08 3:47 PM
http://it.nees.org/library/telepresence/data-turbine-39-utilities.php
[contents]
$ svn co
https://svn.nees.org/svn/telepresence/dataturbine
The last stable release is in the branches directory, while the latest developments are available in the trunk directory. Also included are two simple "Hello, world!" style programs that demonstrate how to create a Data Turbine source and sink. Look in
<dataturbine>/branches/3.7/src/org/nees/rbnb/hellodt to
see the code. The tsource.java file demonstrates how to create a source and the tsink.java file demonstrates how to create a sink. The Creare Data Turbine Java API documentation is essential reference material and can be accessed locally at
$RBNB_HOME/Documentation/JavaDoc on machines that have the
23 of 24
1/7/08 3:47 PM
http://it.nees.org/library/telepresence/data-turbine-39-utilities.php
[contents]
24 of 24
1/7/08 3:47 PM