System Robustness

No Statistics Define Note

For a system to be robust, we
should not see a sudden dip in
Profit profit measures when parameters
1
measures are changed slightly. Stability of
results is more important than
total profits.
The ratio of gross profit to gross
It must be above 1.0, or the
loss.
system is losing, and preferably
As a measure of general
The profit above 2.0. Although a high
1.1 performance, the profit factor only
factor number suggests greater than 10
includes profits and losses, not
is a warning that the system has
drawdowns. It, therefore, does not
been curve-fitted
represent statistics on risk.
IF the profit factor is reduced by
Outlier- adjusted profit to loss is a
this anomaly and ends up below
profit factor that has been
Outlier- 1.0, the system is a bust because
adjusted for the largest profit.
1.2 adjusted it depended solely on the one
Sometimes a system will generate
profit to loss large profit. The largest winning
a large profit or loss that is an
trade should not exceed 40% to
anomaly.
50% of total profit.
In trend-following systems, this
percentage is often only 30% to
Percentage Obviously, the more winning
50%. Most systems should look
1.3 winning trades there are the less chance of
for a winning trade percentage
trades a run of losses against a position.
greater than 60%. Any percentage
greater than 70% is suspect.
Annualized Annualized rate of return is used
1.4 rate of for relating the results of a system
return against a market benchmark.
The payoff It is a ratio of the average winning For trend-following systems, it
1.5
ratio trade to average losing trade. should be greater than 2.0
The length
The length of the average winning
of the
trade to average losing trade
average
should be greater than 1. Greater than 5 is preferable for
1.6 winning
Otherwise, the system is holding trend-following systems.
trade to
losers too long and not maximizing
average
the use of capital.
losing trade
The efficiency factor is the net
profit divided by the gross profit.
The Successful systems usually are in
This factor is mostly influenced by
1.7 efficiency the range of 38% to 69% - the
the win percentage. It suggests
factor higher the better.
that reducing the size of the losses,
as through stop-loss orders.
 High profits are good, but we
Risk
2 must balance them against any
measures
increased risk.
The maximum cumulative
drawdown of losing trades can The rule of thumb is that a
The
also be thought of as the largest maximum drawdown of two
maximum
2.1 single trade paper loss in a system. times that found in optimizing
drawdown
The maximum loss from an equity should be expected and used in
(MDD)
peak is the maximum drawdown anticipated risk calculations.
(MDD).
The MAR ratio is the net profit
percent as a ratio to maximum
The MAR
drawdown percent. It is also called
ratio (the In any system, the ratio should be
2.2 the Recovery Ratio, and it is one of
Recovery above 1.0
the best methods of initially
ratio)
screening results from
optimization.
The time to
The time to recovery from large
recovery Ideally, this time should be short
2.3 drawdowns is a measure of how
from large and losses recuperated quickly
long it takes to recuperate losses.
drawdowns
Maximum favourable and adverse
excursions from list of trades
Maximum
informs the systems’ designer of Its primary use is to give hints as
favourable
how much dispersion exists in to where trailing stops should be
and adverse
2.4 trades. It can be used to measure placed to take advantage of
excursions
the smoothness of the equity favourable excursions and reduce
from list of
curve but also give hints as to adverse excursions.
trades
where and how often losing trades
occur.
First, it doesn’t include the actual
annual return but only the
average monthly return. Thus,
The ratio of excess return irregularities in the return are not
(portfolio return minus the T-bill recognized.
The Sharpe
2.5 rate of return) divided by the Second, It doesn’t distinguish
ratio
standard deviation of the excess between upside and downside
return fluctuations.
Finally, it does not distinguish
between intermittent and
consecutive losses.
This is the average annualized
compounded return divided by MR
Return (maximum of either decline from
2.6 Retracemen prior equity peak [that is, worst
t ratio loss from buying at peak] or worst
loss at low point from any time
prior)
 This is the arithmetic average of
Sterling annual net profit divided by
2.7 ratio (over average annual maximum
three years) drawdown; It is similar to the gain
to pain ratio.
This is the worst possible loss from
the highest point; using this
Maximum
2.8 measure by itself is not
loss
recommended because it
represents a singular event.
This is similar to the Sharpe ratio,
but it considers only downside
volatility. It is calculated as the
It is more realistic than the Sharp
2.9 Sortino ratio ratio of the monthly expected
ratio.
return minus the risk -free rate to
the standard deviation of negative
returns.
Two graphs commonly are used as
Smoothness
a visual analysis of a system’s
3 and the
performance: the equity curve and
Equity curve
the underwater curve.
- Positive expectation: Greater
than 13% annually
- Small number of robust trading
rules: Less than 10 each is best for
entry and exit rules
- Able to trade multiple markets:
Can use baskets for determining
parameters, but rules should work
A good across similar markets, different
4 trading stocks, different commodities
system futures, and so on.
- Incorporates good risk control:
Minimum risk as defined by
drawdown should not be more than
20% and should not last more than 9
months.
- Fully mechanical: No second –
guessing during operation of the
system.