Performance Issues

RapidResponse Resources & Performance
December, 2015
Duncan Klett, Leah McGuire, Paul Haviland
Copyright © 2015 Kinaxis All Rights Reserved.

Agenda
• RapidResponse Architecture & design considerations

• Custom data model
• Worksheet design
• Best practices
• Worksheet design - examples
• Parking lot, additional questions, etc

RapidResponse System Architecture
Host ERP
Extraction
Query Engine
Worksheet - Demand:
Input Data Calculated Data [filter] {fields,,,}
Analytic: Netting
Worksheet – SR
Part PartType Netting: PlannedOrder
SupplyOrder Netting: Activity
Worksheet – CPO
ScheduledReceipt Netting: IndDmd Extensions
Netting: SR Extensions
DemandOrder Etc…
IndependentDemand Worksheet – Supply
Union
BillOfMaterial
OnHand Worksheet – Planning Sheet

Scenario: A ____
Scenario: B __ Join
Scenario: C

Design guidelines
• Requirements (for example…)
– Inputs / Outputs
– Usage
– Security Roles
– Performance expectations
• Analytics, Tables
• Calculation Definitions (logic)
• Custom Data Model
• Performance considerations (data, analytics, queries)
– Design iterations to improve performance
• Conventions & Rules of thumb
• Formatting
• Documentation
• Estimates
– Design
– Develop
– Test

Customer Success - Checklist
 Requirements are understood by development team

 Playback of prototype accepted by customer
 User processes & scenario/server usage defined,
documented, accepted by customer USERS
 Limitations, performance expectations understood by
customer
 Solution includes management reports
 Resources and process documented (Help, Task Flows,
Process Orchestration, etc)

Design - Checklist
 New data model aligns with requirements
 Reduce usage of Lookup & Joins
 Data model & resources follow naming conventions
 Resources follow formatting conventions
 Test performance
1. Restrictive filter
2. Less restrictive filter (usual usage model?)
 Iterate design for functionality & performance
 Review/Improve performance
1. Worksheet Performance tool
2. IN clauses, other “dull knife” filtering
3. Defer bucketing
4. LOD(), restructuring,
5. Data Model, Expression fields
6. Design minimizes data churn
7. Design minimizes query data volume

Analytics and solution design DataModel
• Owner Table
– Netting owned by Part
– WhereConsumedForDemand owned by Analytics
IndependentDemand
• One analytic can calculate different things

– Netting: PlannedOrder table, Data
RescheduledDate field on ScheduledReceipt
• Pull-based, On-Demand, cached

– Results shared across users, by scenario Scenarios
• Only calculate what is required

Choosing the table to report
• Queries on Input data are fast (input tables, input fields)
• Use specific table UNLESS several data sources required
– PlannedOrder, Forecast, DependentDemand
– Vs. Activity, etc
• Use Activity if “driver” and AvailableDate not needed
• Use CTPActivity (or CTPPlannedOrder) if AvailableDate is needed
but Driver is not needed
• Full-level-peg tables are slower, but have driver information
– SupplyDemand has 3 levels available
– WhereConsumed is often faster than more-obvious WCFS, WCFD
• FlatBillUp and FlatBillDown can be slow if multiple
time-phased BOMs, substitute BOMs, and/or PartSources
Use new table: FlatComponent,

Part.FlatComponents
8

Deleting Data in RapidResponse DataModel
• Marked for delete vs. permanently deleted

– Delete operation is at scenario (version) level
– Other scenarios “see” the deleted record
Analytics
• Cascading deletes
– Delete a record  any record(s) referencing it will be
deleted Data
– Exception: Nullable Reference fields
Scenarios
9

Updating Data in RapidResponse DataModel
• Data Update (import/load) operation
– Data record Insert/Modify/Delete supported
– Decision is made by comparing the key fields
• Key value should not be transient Analytics

– If a record previously existed, it will be modified
– Avoid Modify operation implemented as Delete/Re-insert
– Do not use current date or any other dynamic values (if a
record existed yesterday, the key value should be the same Data
today)
• Data change commands

– Use specific action (Insert, Delete) if practical
– Use Read-Only fields as virtual keys Scenarios
10

Data Model design
• References, not strings

• Key references
• Key references not included in ID string
• Naming:
– New table: Useful name (use Namespace to differentiate)
– New field in standard table: U_*
– Fields in custom tables: <ActualName>, no need for U_
– Key string with “real” content: ID
– Key string expected to be blank: BaseKey
• Data integrity through structure
• Expression fields
11

Data Model Reference vs. String DataModel
Make
Condition Why?
Reference?
Has related fields Y Saves memory by not storing fields on every
Example: ScheduledReceipt.U_Batch is a record. However if it is just one additional
reference to a table U_Batch which has field this is not a compelling reason.
various fields describing a production order
Analytics
Field might be used in a LOOKUP to get Y You want to use the database linkages for
related data from another base table. reporting and avoid the use of LOOKUP in
reports.
Field will be used in a hierarchy Y Filtering is much more efficient if the system
Example: Part.Commodity1 (which is a subset can read a small referenced table rather than
of Commodity2 etc.) have to read all the detailed records. Data
Field will be used for filtering in queries Y Query can be more efficient. This may not be
a compelling reason because the system is
so fast.
Allow drop-down list of values when user edits Y Insures valid values are used and makes
field in a referencing table. data entry easier.
None of these apply and are unlikely to apply N Less work because you don’t need to set up
Scenarios
in the future. Field is “report only” as the Reference table. Memory consumption
supplemental information and will not be used for a single field is slightly less if it is NOT a
in a filter. Field will not be used to do a reference.
LOOKUP. Editing from referencing table is an
issue. This is a commonly over-used option.
12

Common Data Model Design Mistakes DataModel
Design Element Slow/Inefficient Fast/Efficient
New user added field to be Add the field to a detailed Add the field to the header table
added at the right level in table (e.g. Order Line) (e.g. Order Header)
the data model
Analytics
Multiple new, related String Add each field to an existing New table with the related fields;
fields, to be added to the table Reference field to the new table
data model added to the existing table
Interdependent (one to Add all fields to an existing New tables, with a one-to-many
many relationships) fields to table relation to one another;
be added to the data model The new table at the top linked
Data
(though a reference) to an
existing table
String field vs. Reference String field Reference field
String field vs. Boolean String field Boolean field

Scenarios
13

Worksheet Naming Conventions
Acronym Full Words Description
SRC Source Component worksheet typically table based query worksheet not
in one of the other categories below
CMP Composite Intermediate Composite worksheet not in one of the other
categories below
SEL Selected Worksheet that is used with IN functions from other worksheets
so when it is used, the expression would be like Part IN SelParts!
LKP Lookup Worksheet that is used to perform Lookups
(can also be used for IN expressions)
INS Insert Command worksheet that is used for Insert Automation
UPD Update Command worksheet that is used for Update Automation
MOD Modify Command worksheet that is used for Modify Automation
DEL Delete Command worksheet that is used for Delete Automation
GET Get The worksheet where data for the Command worksheets
(Ins, Upd) is calculated
14

Formatting Conventions
Formatting is best done while the SRC worksheet is being created. Easier
for you, the author, to understand what is being shown at each level.
Suggestions:
• Leading columns (especially if GroupBy) should use Hide Duplicates
• Quantities use Hide Zeros
• Conditional formatting so negative numbers are Bold Red
• Editable fields use cyan or light yellow background
• Center Boolean fields and other “short” fields (often Site, Type, etc)
• Conditional formatting on Boolean fields so the “normal” case has light
grey (almost hidden) text and the “exception” case is normal text.
• Where data in a column typically exceeds column width, set adjacent
columns to CENTER their contents
• Define drill to details sheets when presenting aggregated data
• Highlight “exception” data with Bold Red text or Yellow background
15

Assert function
This function tests for an assumed condition and returns a specified value as long as that condition is
satisfied. If the condition is not satisfied, the query is canceled and a worksheet error returned. …
ASSERT(logical, value, errorMessage)

• logical is a valid boolean expression that evaluates to true or false (with the true condition being the
expected result). …
• value is the value returned as long as the boolean expression is satisfied (TRUE).
• errorMessage is a text string included in the error message returned if the boolean expression is not
satisfied (FALSE).
The following examples are based on the Part table.
Entering Returns
ASSERT(has PartSources, The fixed lead time for the part's source. However, if at
PrimaryPartSource.FixedLeadTime, 'There least one part returned does not have a part source
are parts without sources') defined, the query is canceled and the error message
"There are parts without sources" is displayed instead.
1 / ASSERT(StdUnitCost <> 0, One divided by the standard unit cost for the part.
StdUnitCost, 'Undefined unit costs') However, if at least one part returned does not have a
standard unit cost value, the query is canceled and the
error message "Undefined unit costs" is displayed instead.
16

Assert example
• Stop a w/s query on WhereConsumedForDemand if more

than 5 parts selected
• Several ways, but complications with query optimization
– Unused columns in component sheet get removed automatically
– Define Assert column in SELParts so it is GroupBy
– IN does not need to match fields with GroupBy columns
17

Best Practices – Business Processes Pre‐requisites
• Create and maintain data in one place
• Update (child) scenarios after last data change
• Be mindful of server/system operations Resources
• Scheduled vs. automated operations on “noisy”

scenarios
Processes
18

Best Practices – Solution Development
DataModel
• Enable analytics settings one at a time and only when needed

– (to verify new functionality)
• Define telescopic time horizon; bucket records

Analytics
• Use the right “Consolidation” (analytic) table
• Use analytics variables for scope control Data
• Consider performance/benefit ratio
• Filter as much / as early as possible

Scenarios
19

Best Practices - Scenario Management DataModel
• Create/Modify/Delete
– Data import/update
– Choose interactive, scheduled or automated operations
Analytics
• Record uniqueness
– Critical for quick access to a record
– Mapping data in key (string) fields impacts the system
Data
performance
• Special behavior in RapidResponse

– Data manipulation is scenario-based
Scenarios
20

Best Practices - Data Model
• Eliminate unneeded custom fields
• Use structure rather than “flat” fields
– Structure custom fields at top of relevant data structure
• References better than Strings
• Use “expression-based fields” in place of macros
• Don’t duplicate Key Reference strings in table ID field
Use Avoid
Keys Reference1, ID=Ref1+Ref2
Reference2, ID (string) + String
21

Best Practices – Filters, Hierarchies
• Make filter expression explicit and clear
– Good: Part.Name = ‘AC001’; Part.Name in (‘AC001’, ‘AC002’)
– Not so good: Part.Name like ‘AC*’; (better: Left(Name,2) = ‘AC’)
Part.Description like “Trial*”
• Don’t create/use filters returning thousands of items

– Use ASSERT (on part count) to stop users using these filters
• Avoid string concatenation in Hierarchy Level definitions

– Slow, breaks capabilities like disaggregation
• Avoid any expressions in Hierarchy Level definitions where it will be

used in disaggregation.
– If the worksheet is showing "Each level to the level below the user's
selection“, then the edit does not work as expected.
• Don’t create/use hierarchy levels with thousands of entries
• Where possible with logical expressions, use IN for filters rather than
Lookup
22

Best Practices - Workbooks Pre‐requisites
• Best Recommendation ever:

Use Worksheet Performance Tool!
• “Progressive disclosure” of data

– On initial opening of a report, only display inexpensive
but important information Resources
• Use workbook variables + conditionally hidden columns
• Use drill-to-details techniques to get more/focused
information
• Make it an explicit operation for user
Processes
23

Best Practices - Workbooks Pre‐requisites
• Efficient Filtering on tables to JOIN. Always reduce
the number of records that you process/touch.
• Always filter calculated data by its owner table/entity.
• Build performance into Composite Worksheets
– Filter and Group early
– Bucket late Resources
– Use “As Reference” fields all through components
– Instantiate (flatten) the “As Reference” at top-level
worksheet
– Expression-based fields can be used to keep editability
Processes
24

Query Performance
25

Automatic query optimization
• “Line of descent”
– Query automatically chooses path it thinks will eliminate most
records, soonest  using AND conditions
– Can be overridden using LOD(<path>) function
• Query reduction
– Unused columns in component sheets are removed by query
• Intent is that unneeded columns are removed, but only if they
don’t effect the visible result
• GroupBy columns are always seen as USED
• All columns are seen as USED if any running balance in sheet
• All columns (visible or not) in visible sheet are kept
• New (fast) Lookup function
– Lookup ( {keys, key:Ref..}, Table {keys, key:Ref..} )
26

Tune worksheets – Worksheet structure
• Choose source table carefully

– Activity, CTPActivity, Demand, DependentDemand,
PlannedOrder,WhereConsumed, etc.
– WhereConsumed on many parts is often faster than WCFS on a
few. RR_Analytics_WhereConsumedForSupply_MaxLevel
• Keep references in Source sheets
– “Instantiate” at top
– Use references as key to most IN, Lookup, Join sheets
• Re-use common source sheets, ONE source query per table
• Bucket at top of composite tree
• Avoid Sets. Use Joins, IN, Lookup, etc or use Calculated Fields
• Avoid auto-save worksheets
27

Tune worksheets - Filtering
• Filter based on input data and “owner table” first

(query re-arranges flow, but clearer for other authors)
• Filter at source level of composite tree
• Enclose expensive analytics in rough-cut filters
• To force analytics to re-run:

– In PRIVATE scenario
– Change ANY setting in any ControlTable
i.e. OnHandType.Description
– Select worksheet to analyze
28

Identify Slow Queries, analyze performance
• Query Manager
• Worksheet Performance … tool
29

Example
Planning Sheet / Supply Analysis

• Uses WhereConsumedForSupply
– (one of the most “expensive” calculated tables)
– Logically easiest to understand to identify component
availability
30

Original Implementation
• Applies Filter to Driver Part – but ‘expensive’
• In example, looks at 140K records to select 23K
• Total time (5 parts) is 3.2s
31

Faster approach, pre 11.2
• Use “dull knife” to identify parts which “might” be of interest
• Report from WhereConsumed (much faster when looking at
more than a few driver parts with WCFS)
• Looks at 386 parts, 338K records to select 23K, in 1.2s
32

Faster approach 11.2 and later
• Use new analytics variable to limit levels searched:
!RR_Analytics_WhereConsumedForSupply_MaxLevel=1
• Looks at just 25K records to select 23K in 0.9s
• Query performance rules continue to evolve!!
33

Summary – Lessons Learned
• Measure, monitor & improve worksheet performance
• Avoid unnecessary updates, commits
• Avoid processes leading to “insertion conflicts”
• Regularly purge/update scenarios – monitor ages
• Move scheduled tasks to non-critical times
• Data Model
– Use References and expression-based fields
– Don’t repeat Key References in Key String
• Tune worksheets
– Don’t expand references in Source sheets
– Use references as keys for IN, Lookup, Join sheets (if possible)
– Choose source table carefully
– Re-use common source sheets, ONE source query per table
– Filter at source of composite tree
– Enclose expensive analytics in rough-cut (input data) filters
– Use Assert to protect users from their own foolish filters
– Bucket at top of composite tree
– Avoid Sets. Use Joins, IN, Lookup, etc or use Expression Fields
– Avoid auto-save worksheets
• Pay attention to Analytics & engine improvements in each release!
34

Resource performance – Use Cases
• Some examples – the devil is always in the details!
• PLEASE ask questions, make suggestions
• Don’t be afraid to try things
– Communicate what you learn!
– Consultant Forum is a great place to record what you learn
• Prepared examples:
1. Pegged demand and supply
2. Multi-site shortage report
3. Capture sales forecast
4. Firm-up near-in planned orders
5. Cross-tab to edit customer price
35

Use Case 1 – Pegged Demand & Supply
Requirement:
• Report all the parts, all levels, needed to satisfy selected end-item
demand
Solution – Alternatives
• What is report to be used for?
– Pegging to demand or supply?
– Which demands/supplies to report?
• WhereConsumedForDemand
• WhereConsumedForSupply
• WhereConsumed
• Supplies Used sheet (A Parts/Dwest, or Europe)
36

Case 1 - Solution
• Use SEL:Parts sheet for filtering (with Assert embedded)

• For WhereConsumed, use SEL:FlatComponent to identify
possible components of the demands
37

Case 1 - Example
38

Use Case 2 – Multi-site shortage report
• Intention is to identify parts:

– Has a planning shortage during the next 4 weeks
– Is short in multiple sites
– Report the site that goes short first
– If two sites go short on the same date report the site whose
shortage is larger
39

Multi-Site Shortage report - design
• Decisions required:
– What is meant by 4 weeks: 4 Week, 28 Days, 1 Month??
– Shortage based on planning or AvailableDate?
• Report from Activity or CTPActivity
– How to handle “bad” data (blank ref part)
• Design
1. Running projected balance by Part (reference, includes Site)
2. Identify first shortage date and quantity by Part
3. Summarize to identify more than one part in Ref Part set
4. Final report:
Filter for multiple parts, Group to identify earliest
40

Case 2 - Example
41

Case 2 Structure
42

Use Case 3 – Capture Sales Forecast
• Basic idea: Capture current sales forecast to Historical

Demand Series Detail
• Additional requirements:
– Consensus Forecast
– Elements used to generate consensus forecast
– Bucketing (monthly?) and duration (12 months?)
• Design:
– Data change commands now support:
• Autocreate of referenced tables (if enabled in data model)
• Record definition by Reference
(but, only ONE level – DR pending)
43

Case 3 - Structure
• Insert/Modify/Delete
ONLY Quantity, Unit Price, Value editable
(other columns act like keys)
• Group except above fields
Includes fields for autocreate headers
44

Use Case 4 – Firm-up near-in planned orders
• Simple concept: convert planned orders to

ScheduledReceipts
• But, lots of issues to consider:
– Types, Status
– Priority
– Closed Loop
• What comes back from ERP
• Rationalizing RapidResponse data and extract data
45

Scheduled Receipt – Simplified process
Determine new/changed Firmed Planned Orders

Near-in supply plan to Sched Receipts (Ignored)
Export plan to ERP
Extract from ERP

Import (active)
To RapidResponse
46

Scheduled Receipt – Closed Loop process
Determine new/changed
Near-in supply plan Firmed Planned Orders
to Sched Receipts
Calculate / update
Next order number
Export plan to ERP
Extract from ERP

Import to RapidResponse
• Delete “export” records
matching those in extract
• Delete export records if
too old Rationalize firmed and
Imported data
47

Use Case 5 – Cross-tab to edit Customer Price
Requirements:
• Edit CustomerPrice table (insert, modify, delete) for S&OP
forecast process
• Provide cross-tab view for editing
• “clean up” resulting records so only changes are tracked, by
EffectiveDate
• Allow editing for every defined PartCustomer
– Part and Customer must be used in at least one PartCustomer – results in too
many potential unwanted rows!
– Allow edits for defined PartCustomers PLUS any additional CustomerPrice
combinations where Part and Customer individually are in existing PartCustomer
records
– Create PartCustomer records if the CustomerPrice Part and Customer equivalent
does not yet exist
• Avoid pre-building all possible CustomerPrice records
• New Insert definition for CustomerPrice
48

Case 5 - Notes
• Can’t display Hierarchy fields because there is no PartCustomer
on CustomerPrice
– Add expression-based field to data model?
Lookup ( {Part, Customer},
{Part=Part :Ref :By , Customer=Customer :Ref :By ,
PartCustomerRef=Self :Ref :Max },
Null(PartCustomer), Exact )
• Need to include a Date (in bucket range) for Src:PartCustomer
• No need for disaggregation, once Insert definition is correct
• Command to create any missing PartCustomers
• Command to “clean-up” redundant records
• TEST commands
– Run them to be sure they work
– Run them again to ensure “no changes”
– Enable logging if change summary does not make sense
49

Case 5 - Example
50

Performance Issues

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Performance Issues

Uploaded by

Copyright:

Available Formats

RapidResponse Resources & Performance

Copyright © 2015 Kinaxis All Rights Reserved.

• RapidResponse Architecture & design considerations

Copyright © 2015 Kinaxis All Rights Reserved.

OnHand Worksheet – Planning Sheet

Copyright © 2015 Kinaxis All Rights Reserved.

Copyright © 2015 Kinaxis All Rights Reserved.

 Requirements are understood by development team

Copyright © 2015 Kinaxis All Rights Reserved.

Copyright © 2015 Kinaxis All Rights Reserved.

• One analytic can calculate different things

• Pull-based, On-Demand, cached

• Only calculate what is required

Copyright © 2015 Kinaxis All Rights Reserved.

Use new table: FlatComponent,

Copyright © 2015 Kinaxis All Rights Reserved.

• Marked for delete vs. permanently deleted

Copyright © 2015 Kinaxis All Rights Reserved.

• Key value should not be transient Analytics

• Data change commands

Copyright © 2015 Kinaxis All Rights Reserved.

• References, not strings

Copyright © 2015 Kinaxis All Rights Reserved.

Copyright © 2015 Kinaxis All Rights Reserved.

String field vs. Reference String field Reference field

String field vs. Boolean String field Boolean field

Copyright © 2015 Kinaxis All Rights Reserved.

Acronym Full Words Description

Copyright © 2015 Kinaxis All Rights Reserved.

Copyright © 2015 Kinaxis All Rights Reserved.

ASSERT(logical, value, errorMessage)

The following examples are based on the Part table.

Copyright © 2015 Kinaxis All Rights Reserved.

• Stop a w/s query on WhereConsumedForDemand if more

Copyright © 2015 Kinaxis All Rights Reserved.

• Create and maintain data in one place

• Update (child) scenarios after last data change

• Be mindful of server/system operations Resources

• Scheduled vs. automated operations on “noisy”

Copyright © 2015 Kinaxis All Rights Reserved.

• Enable analytics settings one at a time and only when needed

• Define telescopic time horizon; bucket records

• Use analytics variables for scope control Data

• Consider performance/benefit ratio

• Filter as much / as early as possible

Copyright © 2015 Kinaxis All Rights Reserved.

• Special behavior in RapidResponse

Copyright © 2015 Kinaxis All Rights Reserved.

Copyright © 2015 Kinaxis All Rights Reserved.

• Don’t create/use filters returning thousands of items

• Avoid string concatenation in Hierarchy Level definitions

• Avoid any expressions in Hierarchy Level definitions where it will be

Copyright © 2015 Kinaxis All Rights Reserved.

• Best Recommendation ever:

• “Progressive disclosure” of data

Copyright © 2015 Kinaxis All Rights Reserved.

Copyright © 2015 Kinaxis All Rights Reserved.

Copyright © 2015 Kinaxis All Rights Reserved.

Copyright © 2015 Kinaxis All Rights Reserved.

• Choose source table carefully

Copyright © 2015 Kinaxis All Rights Reserved.

• Filter based on input data and “owner table” first

• To force analytics to re-run:

Copyright © 2015 Kinaxis All Rights Reserved.