Context SQL Sync

Context SQL Sync
The Context SQL Sync REST service allows importing (either one-time or by scheduling a
continuous job) of context data from an SQL database (PostgreSQL, MySQL, or MS SQL) to
TrendMiner.
The steps to sync context data are:
● Request an access token for authentication

● Define a datasource
● Define a query
● Perform a one-time sync
- and/or -
● Define and execute a continuous sync job
Authentication
Currently, only OAuth2 authentication is supported.
An access token can be requested with the TrendMiner security API.
Method: POST
Path: /security/oauth/token
Authorization header: Basic
dHJlbmRtaW5lckFwaTpjdmJwa3R6SmVGUENLVk10dFN0ZVQ1ejNlYXlkeUZaYQ==
Body
grant_type: password
username: [TrendMiner username]
password: [TrendMiner password]
Context Sync API

A swagger interface is shipped with your appliance and is available on /context-sql-
sync/swagger-ui.html
Define a datasource
New data sources can be defined by the API:
Method: POST
Path: /context-sql-sync/datasource
Authorization header: Bearer [access_token]
Body
{
"name": "localdb",
TrendMiner N.V. | VAT BE 0894.414.630 | RPR Hasselt, Belgium | IBAN: BE82 737022894568 | BIC: KREDBEBB | T +3211263830
info@trendminer.com | www.trendminer.com
"jdbcUrl":
"jdbc:postgresql://localhost:5432/tm_context_sql_sync",
"dbType": "POSTGRES",
"username": "postgres",
"password": "postgres"
}
name - an arbitrary string

jdbcUrl - a JDBC url, specifying the connection to the database
dbType - specifies the type of the database; supported values are: POSTGRES, MSSQL,
and MYSQL
username/password - the credentials used to access the database; can be omitted if the
database does not require authorization
The result will contain the id of the newly created datasource. This id will be used later when
executing a historical sync or when defining sync jobs.
Define a query
Method: POST
Path: /context-sql-sync/query
Body
{
"name": "queryAlias",
"supportsParameters": true,
"supportsNamedParameters": true,
"componentLookupStrategy": "ASSETS|TAGS|ASSETS_THEN_TAGS",
"fieldNames": "customField1, customField2",
"sqlString": "Select id, componentId, type, description,
start_event, start_time, stop_event, stop_time, customField1,
customField2 from LOGBOOK where :startDate <= start_time and
start_time < :endDate;"
"typeMapping": [
{ "PRODUCTION": "MAPPED_TYPE" },
{ "SOURCE_TYPE": "TM_TYPE" }
]
}
If the datasource contains a lastModifiedDate column, the query can be modified to select
records based on their lastModifiedDate. For example:
{
...
"sqlString": "Select id, componentId, type, description,
start_event, start_time, stop_event, stop_time, last_modified_date
from LOGBOOK where :startDate <= last_modified_date and
last_modified_date < :endDate;"
...
}
This way, if an item is modified, it will be selected for sync, even if its start time is
untouched.
name - an arbitrary string

supportsParameters - a boolean value(true|false) indicating whether the query supports
positional parameters. Defaults to true
supportsNamedParameters - a boolean value(true|false) indicating whether the query
supports named parameters. Defaults to true
componentLookupStrategy - a value specifying how to interpret the componentId
returned by the query. The allowed values are ASSETS (componentId is interpreted as the
external id of an asset), TAGS (componentId is interpreted as a tag name), and
ASSETS_THEN_TAGS (componentId is interpreted as an asset and, if no asset is found,
then componentId is interpreted as a tag name). Defaults to ASSETS
fieldNames - a comma-separated list of values, each representing the name of a column
in the SQL query (see below). These columns contain custom fields to be associated with
the newly created or updated context items. This field is optional and defaults to null. Bear in
mind that the specified values are validated against the specified SQL query and the request
will fail if the field names are not found in the SELECT clause.MSS
sqlString - the string of the SQL query that will be executed against the datasource to
fetch the items. There are several requirements for this query:
● The query should return the following columns (in order): id, external_asset_id, type,
description, start_event, start_time, end_event, end_time
● The id will be prepended by prefix (see below) and used as an external identifier of
the context item
● The componentId should identify an already existing asset or tag in TrendMiner
(according to the currently used lookup strategy, as specified by
componentLookupStrategy).
To get an overview of your synced asset structure and the external IDs of the different
assets you can use the "/assets/browse" endpoint of the TrendMiner Assets API. The
external ID of the assets is identified by the “externalId” field.
● The type will be looked up in typeMapping (see below) and the associated value will
be used as the type of the context item
● The description will represent the description of the context item
● The start_event and start_time will be used for the start event of the context item
● The end_event and end_time will be used for the end event of the context item
● If the query supports positional parameters and is run for a specific time interval, the
start and end of the interval will be passed to the query to be used in the selection
criteria
● If the query supports named parameters and is run for a specific time interval, the
start and end of the interval will be passed to the query through the :startDate and
:endDate parameters respectively, so they need to be utilized in the selection criteria
typeMapping - a table used to map the values in the type column to actual context types in
TrendMiner
The result will contain the id of the newly created query. This id will be used later.
Historical Sync
Method: POST
Path: /context-sql-sync/sync
Body
{
"datasourceId": "1",
"queryId": "2",
"externalIdPrefix": "prefix",
"startDate": "2018-10-17 12:29:01",
"endDate": "2018-10-18 20:29:01",
"chunkSize": "2"
}
datasourceId - the id of the datasource against which to run the sync

queryId - the id of the query to be used for the sync
externalIdPrefix - the prefix that will be prepended to the id obtained from the database
in order to produce the external id of the context item. This prefix allows supporting same
external IDs for different data sources
startDate - the start of the sync interval
endDate - the end of the sync interval
chunkSize - the sync interval will be split to subintervals with length specified by this value
(expressed in minutes)
The startDate and endDate can be omitted. In this case the associated query will be run only
once without parameters.
If startDate and endDate are specified, the resulting interval is split to small subintervals
based on chunkSize. Then the SQL query is executed continuously for each of the
subintervals and the start and end of the subintervals are passed to the query as
parameters.
The historical sync provides means to perform a “dry” run, where the data is fetched from the
database and returned to the customer, but no actual context items are created. To use the
dry run:
Path: /context-sql-sync/sync/test
Live Sync
To use live sync, a sync job must be defined first:
Method: POST
Path: /context-sql-sync/syncjob
Body
{
"datasourceId": "1",
"queryId": "2",
"externalIdPrefix": "prefix",
"syncJobInterval": "10",
"chunkSize": "2"
}
datasourceId - the id of the datasource against which to run the sync

queryId - the id of the query to be used for the sync
externalIdPrefix - the prefix that will be prepended to the id obtained from the database
in order to produce the external id of the context item. This prefix allows supporting same
external IDs for different datasources
syncJobInterval - the interval (expressed in minutes) between subsequent job
executions
chunkSize - the sync interval will be split into subintervals with length specified by this
value (expressed in minutes)
The result will contain the id of the newly created sync job. This id can be used to start and
stop the job.
To start the job:
Method: POST
Path: /context-sql-sync/syncjob/{id}/start
To stop a job:
Method: POST
Path: /context-sql-sync/syncjob/{id}/stop
To get a list of all currently defined sync jobs:
Method: GET
Path: /context-sql-sync/syncjob
A sync job is started if its “scheduled” field is true.
Important notes
● The access token will expire after 12 hours. Best practice is to request an access
token before every API call.

Context SQL Sync

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Context SQL Sync

Uploaded by

Copyright:

Available Formats

Context SQL Sync

The steps to sync context data are:

● Request an access token for authentication

An access token can be requested with the TrendMiner security API.

Context Sync API

name - an arbitrary string

name - an arbitrary string

datasourceId - the id of the datasource against which to run the sync

datasourceId - the id of the datasource against which to run the sync

To start the job:

A sync job is started if its “scheduled” field is true.

You might also like