SmartSignal Confidential 8/14/2022

Five Principles for Setting Residual Thresholds

Executive Summary

The SmartSignal Availability and Performance Center developed five principles for
setting residual thresholds. The APC thoroughly tested the five principles and found
application of these principles greatly reduces false positive alert rates, increases the
actionability of SmartSignal incidents, and increases value of the SmartSignal solution to
our customers.

Recommendation

The Availability and Performance Center recommends that the SmartSignal

Implementation Team utilize the five principles when they implement the SmartSignal
solution for our customers.

Background

In early 2009, Availability and Performance Center personnel noticed a deterioration of

WatchList performance in terms of percentage of assets on WatchList. Further
investigation identified residual threshold settings as a key source of false positive alerts
and consequently increased percentage of assets on WatchList.

Percentage of assets on WatchList greater than 30% often reflects unacceptable false
positive alert rates in SmartSignal implementations. Successful SmartSignal
implementations should strive to minimize false positive alert rates for two key reasons.
First, false positive alerts clutter the WatchList and obscure true positives from the
WatchList analysts’ view; and WatchList analysts should never miss any true positive
notifications. Second, false positive alerts decrease the productivity of WatchList analyst
staff. WatchList analysts investigate every incident on every WatchList everyday.

APC personnel hypothesized that wider residual thresholds would decrease the false
positive rate without increasing the false negative (missed catch) rate. In order to test
this hypothesis, the APC devised five principles for setting standard residual thresholds
and organized a special project designated Project HoHoKam to implement the five
principles quickly enough to constitute a controlled test.

The APC planned to measure the false positive rate before and after Project HoHoKam to
determine the efficacy of the five principles of residual thresholds. A statistically
significant reduction in false positives would suggest the efficacy of the standardized
thresholds.

Page 1 of 9
SmartSignal Confidential 8/14/2022

Five Principles of Residual Thresholds

The APC Customer Monitoring Managers used their deep industry experience and their
extensive experience with the SmartSignal solution to categorize the characteristics of
residual thresholds which will minimize false positives without increasing false negative
rates.

Five Principles of Residual Thresholds

1. Mechanical models variables should utilize standard residual thresholds.

2. Residual thresholds have a minimum value based on instrument precision and

actionability; smaller values have not impact on actionability regardless of model
accuracy.

3. Performance model variables should utilize large residual thresholds on a

percentage basis than mechanical model variables.

4. Symmetrical instruments use identical residual thresholds.

5. Sibling equipment use identical residual thresholds.

Discussion of Principle 1 – Standard Mechanical Residual Thresholds

The Advance Application Group found that mechanical models provide SmartSignal
customers with the most valuable early warnings of failure. Furthermore, customers
display a high level of sensitivity to mechanical variables since they often indicate
incipient catastrophic failure of bearings, shafts, and other components.

Standard residual thresholds on these key variables help customers understand the
significance of an incident. For example, the customer will get more value from
discussing with a CMM how to address a positive residual on bearing vibration than from
discussing why the X and Y direction vibrations have different residual threshold values.

Furthermore, customers lack time to prioritize mechanical incidents by any factor other
than magnitude. For example, the reliability engineer will choose to quickly investigate a
1.5 mil positive residual in a bearing vibration while indefinitely deferring the
investigation of a bearing with the 0.25 mil positive residual.

The Customer Monitoring Managers combined their experience with the SmartSignal
solution along with their industrial experience to develop a recommendation for standard
mechanical residual threshold values.

Table 1 below lists the standard residual threshold values for mechanical variables.
.

Page 2 of 9
SmartSignal Confidential 8/14/2022

Table 1: Standard Mechanical Variable Residual Thresholds

Variable Eng Unit Positive Residual Negative Residual

Bearing Temps deg F 5 15
Lube Oil Temps deg F 5 15
Vibration Peak to mil 1 2
Peak
Vibration in/sec 0.1 0.3
(velocity)
Axial mil 1 1
Displacement
(thrust position)

During the course of implementation, new variables will appear. The APC is prepared to
offer consultation around mechanical variables not list above during the tag mapping
phase and also during the turnover phase.

Principle 2: Minimum Residual Threshold Values due to Instrument Precision

Instrument precision sets the minimum value for residual threshold regardless of model
type. The APC identified some of these variables during Project HoHoKam as listed
below.

Table 2: Minimum Residual Threshold Values

Variable Eng Unit Minimum Threshold

Temperature Deg F 5
Pressure/Flow psi, mmcf/d… 2.5% of maximum
Control Valve Position % 5
Acidity pH 0.1
Conductivity Mmho 50

Pressure measurements (flow measurements are based on differential pressure) depend on

the type of transmitter, and SmartSignal rarely learns the transmitter characteristics. A
simple rule of thumb is to set the residual threshold a minimum of 2.5% of the maximum
value measured over the course of the year. Pressure and flow typically reside in
performance models which Principle 3 discusses further.

Principle 3: Use Larger Residual Thresholds on Performance Model Variables

Performance models find early warning of loss of efficiency in pumps, compressors,

turbines and other types of equipment. Compared to mechanical models which often find
early warning of catastrophic failure, performance models suggest failure modes and
repairs which require less urgency.

Page 3 of 9
SmartSignal Confidential 8/14/2022

Project HoHoKam set all performance variables between 3% and 10% of the average
operating value of the variable depending on the type of variable.

Performance variables fall into three categories of controlled independent, random

independent variables, and dependent variables. A controlled independent variable is
determined by the setpoint input into a controller by an operator. The controller has the
job to adjust a valve to cause the controlled process variable to achieve the value of the
controller setpoint. Controlled independent variables use tighter residual thresholds than
either random independent or dependent variables. A random independent variable is
exemplified by ambient temperature. No one can control the random independent
variable; however the variable impacts the equipment or process. A dependent variable
results from the independent variables and the fixed construction of a piece of equipment
or process.

Table 3 below summarizes the recommendation for residual thresholds for performance
model by category.

Table 3: Performance Model Variables

Independent Control Independent Random Dependent

% of average operating 3% 10% 10%
value

The APC is prepared to offer consultation around performance variable residual

thresholds during the tag mapping phase and also during the turnover phase.

Principle 4: Symmetrical instruments use identical residual thresholds

Certain assets such as gas turbine engines, reciprocating engines, and reciprocating
compressors utilize symmetrical instrumentation as key variables for failure detection. In
the gas turbine engine, the turbine exhaust consists of many exhaust temperature
measurements around the circumference of the engine. Each of these symmetrical
measurements should use the same residual thresholds.

Certainly other assets will exhibit symmetrical instrumentation and the APC is prepared
to offer consultation during the tag mapping and also during the turnover phase to
provide for the best application of Principle 4.

Principle 5: Sibling equipment use identical residual thresholds

Plants often install multiple pieces of similar equipment in the same service to provide for
redundancy or scalability. For example, coal fired power plants often utilize between 4
and 12 nearly identical coal pulverizers. If each of these pulverizers share the same
model number, they are sibling equipment. Furthermore, all General Electric LM2500
gas turbine engines are sibling equipment. Each of these assets should share the same
residual threshold values for the variable they share in common subject to principles 1
through 4.

Page 4 of 9
SmartSignal Confidential 8/14/2022

Certain implementations will require exceptions to this principle. Further Principle 5

proposes a repository of assets such as a standard LM2500 and a standard GE 7FA. The
APC have developed a set of standard residual thresholds for a number of gas turbine
engines and the APC is available for consultation during the tag mapping and turnover
phase for these assets.

Project HoHoKam Findings – 65% Reduction in False Positive Rate

The APC implemented the residual threshold principles during the week of 9 March 2009
and measured the results of the project using the SQLDump tool on the Chevron Global
Power WatchList. The APC found that the residual threshold principles resulted in a
65% reduction in false positives and no increase in false negatives.

Table 3 breaks down the decrease in the incident rate before and after the APC
implemented the residual threshold principle. Figure 1 shows the incident rate week by
week during the first 18 weeks of 2009. The reduction in incident rate demonstrated
persistence. Furthermore, Chevron Global Power reported no false negatives during the
initial 10 week test period.

Table 3: Summary of Incident Rate Reduction – Chevron Global Power

Figure 1: 2009 Incident Rate by Week – Week 1 – Week 18

Page 5 of 9
SmartSignal Confidential 8/14/2022

Conclusion

The APC devised and tested five principle of residual threshold settings to decrease false
positive alert rates. The application of the principles decreased false alerts and did not
increase false negatives.

The APC recommends that the Application Engineering Implementation team use the
five principles for setting residual thresholds to minimize false positive incident rates.

Page 6 of 9
SmartSignal Confidential 8/14/2022

Table 1: Standard Mechanical Variable Residual Thresholds

Variable Eng Unit Positive Residual Negative Residual

Bearing Temps deg F 5 15
Lube Oil Temps deg F 5 15
Vibration Peak to mil 1 2
Peak
Vibration in/sec 0.1 0.3
(velocity)
Axial mil 1 1
Displacement
(thrust position)

Page 7 of 9
SmartSignal Confidential 8/14/2022

Table 2 Minimum Residual Thresholds

Variable Eng Unit Minimum Threshold

Temperature Deg F 5
Pressure and Flow Psi, mmcf/d… 2.5% of maximum value
Control Valve Position % 5
Acidity pH 0.1
Conductivity Mmho 50

Page 8 of 9
SmartSignal Confidential 8/14/2022

Table 3: Performance Model Variables

Independent Control Independent Random Dependent

% of average operating 3% 10% 10%
value

Page 9 of 9