JOURNAL OF CHEMOMETRICS

J. Chemometrics 2007; 21: 427–439

Published online 4 September 2007 in Wiley InterScience
(www.interscience.wiley.com) DOI: 10.1002/cem.1086

A randomization test for PLS component selection

Susanne Wiklund1*, David Nilsson1, Lennart Eriksson2, Michael Sjöström1,
Svante Wold1 and Klaas Faber3
1
Research Group for Chemometrics, Department of Chemistry, Umeå University, S-901 87 Umeå, Sweden
2
Umetrics, Tvistevägen 48, P.O. Box 7960, S-90719 Umeå, Sweden
3
Chemometry Consultancy, Rubensstraat 7, 6717 VD Ede, The Netherlands
Received 7 August 2006; Revised 29 November 2006; Accepted 12 March 2007

During the last two decades, a number of methods have been developed and evaluated for selecting
the optimal number of components in a PLS model. In this paper, a new method is introduced that is
based on a randomization test. The advantage of using a randomization test is that in contrast to cross
validation (CV), it requires no exclusion of data, thus avoiding problems related to data exclusion, for
example in designed experiments. The method is tested using simulated data sets for which the true
dimensionality is clearly defined and also compared to regularly used methods for 10 real data sets.
The randomization test works as a good statistical selection tool in combination with other selection
rules. It also works as an indicator when the data require a pre-treatment. Copyright # 2007 John
Wiley & Sons, Ltd.

KEYWORDS: randomization test; permutation test; component selection; factor selection; latent variable selection;
partial least squares

1. INTRODUCTION We present a new randomization test that intends to

Partial least squares projections to latent structures (PLS) is a
commonly used regression method for the calibration of
multivariate data [1]. The selection of the number of PLS
components (also known as latent variables or factors) is a
crucial step in reaching the optimal model. Retaining too few
components implies that the calibration data are under-fitted
and there is still information left that can be modeled.
Choosing too many components leads to over-fitting. This
means that, although the calibration data are well described,
the model will have inferior predictive ability for future
objects due to spurious incorporated noise.
In 1996, Denham [2] noted that ‘Unfortunately, the
statistical behavior of PLS regression is still not well
understood and this has meant that it has been extremely
difficult to perform the usual inferential tasks associated with
modeling, such as choosing an optimal model, assessing
uncertainty in coefficient estimates and producing prediction
intervals for future responses. As a result, a number of rules
for choosing an optimal model have been suggested with
varying degrees of statistical validity.’ During the last
decade, this statement has not lost any of its validity.
Indeed, developing [3–12] as well as comparing [2,6,13,14]
PLS component selection rules have been and apparently
continue to be subjects of active research in chemometrics.
*Correspondence to: S. Wiklund, Research Group for Chemo-
metrics, Department of Chemistry, Umeå University, S-901 87 Umeå,
Sweden.
E-mail: susanne.wiklund@chem.umu.se
provide objective guidance for PLS component selection.
This type of test is well known in statistics. Fisher [15]
introduced the principle in 1935, which is why it is often
called Fisher’s randomization test. Randomization tests have
received considerable attention in the chemometric-
s-oriented literature, where they have been proposed for
testing the significance of models (e.g. obtained after
extensive variable selection) [16–19], the equality of per-
formance of two prediction methods [20,21] and also for
selecting the number of components in principal component
analysis (PCA) [22] and PLS [23]. It is important to note that
the currently proposed method intends to improve on the
one introduced by Sheridan et al. [23] To demonstrate this
point, simulated data sets are used for which the true
dimensionality is clearly defined.
The objective of this study is to compare the new
randomization test with three regularly used methods, that
is cross validation (CV) [24,25], leverage correction [26–29]
and the size of the eigenvalue [30]. The results for the
regularly used methods are obtained through two chemo-
metric software programs, namely SIMCA 11.0 (Umeå,
Sweden) [30] and Unscrambler v8.0.5 (Oslo, Norway) [31].
Two final notes seem to be in order. First, we restrict
ourselves to ‘conventional’ implementations of CV. We are
fully aware that Monte Carlo CV [32–34] has recently been
introduced in chemometrics for component [35–37], as well
as variable [19,37–40] selection. Typical for Monte Carlo CV
is the deletion of rather large segments, often even (much)
larger than half the calibration set. The results of a simulation

Copyright # 2007 John Wiley & Sons, Ltd.

428 S. Wiklund et al.
study indicate that the method may outperform ‘conven-
tional’ implementations for small and medium-sized cali-
bration sets [35]. However, Gourvénec et al. [36] noted that a
major concern remains with this method, namely whether
making a calibration model with so few objects will be
representative for the structure of the data. Moreover, since
our goal is to compare a new randomization test with
regularly used methods, we consider its inclusion out of the
scope of this paper. Second, other statistical tests have been
proposed [3,4,9,11]. We prefer not to include them either.
Instead, they are discussed in Section 2.7.
2. PLS COMPONENT SELECTION
This section presents a description of the selection rules under
study. Although there are many ways to select the rank of a
model, the optimal one is to use a large external validation set
that is not involved in the model building stage. Unfortunately,
this requires highly redundant data sets; hence the model must
usually be validated the best alternative way. Mean-centered
models are assumed throughout, but this does not affect the
generality of the presentation.
2.1. CV
CV amounts to a number of rounds of setting calibration
objects apart, for which predictions are obtained using the
model that describes the remainder of the objects [24,25]. In
this study we will restrict ourselves to two common variants
of CV, namely leave one out (LOO) CV, where single
calibration objects are set apart, and so-called r-fold CV [41],
where the calibration objects are divided in r segments, here
r ¼ 7. For these variants all objects are left out once. The root
mean square error of CV (RMSECV) is calculated as
vffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi rffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi
u
u X N
PRESS
RMSECV ¼ tN
1 ðyobs;i
yA pred;i Þ ¼
2
(1)
i¼1
N stage. Consequently, PLS consumes more degrees of freedom
where N is the number of objects in the calibration set,yobs;i is
the observed response for object i and yA pred;i is the associated
prediction with A components. The summation under the
radical sign is the prediction error sum of squares (PRESS).
The division by N to obtain an average measure of prediction
uncertainty is motivated by CV not consuming degrees of
freedom, unlike leverage correction (see below).
Ideally, the model with the lowest RMSECV is considered
to be the optimal one. In practice, however, a clear global
minimum, achievable for a large external validation set, is
often not obtained and one has to resort to softer rules such as
‘the first local minimum’ or ‘the beginning of a plateau’.
Furthermore, there are various ways of implementing the
exclusion procedure in CV, which may considerably impact
the actual CV performance. The results may also depend on
whether mean centering is applied prior to PLS modeling
only, or for each deletion step in CV [42]. Consequently, CV
like most statistics based on real data is inherently somewhat
uncertain. However, as seen in the result section, the
prediction error (RMSECV) has almost identical values for
models with a range of the number of components. The
important issue is that the selected number of components by
CV or randomization test correctly finds this range, the actual
Copyright # 2007 John Wiley & Sons, Ltd.
value of the number of components is immaterial as long as
the prediction error is close to its minimum.
2.2. Leverage correction
Leverage correction is a simplified method used for
validation and component selection [26–29]. The goal of
leverage correction is to convert a fit residual into a
prediction residual, without performing any actual predic-
tions as in CV. The leverage is a measure of the influence of
an object on the PLS model and is closely related to the
Mahalanobis distance and Hotelling’s T2. The leverage of
object i using A components is calculated as
XA
t2ia

1
hA
i ¼N þ T
(2)
t t
a¼1 a a
where ta denotes the orthogonal score vector of component a
(a ¼ 1;    ; A) and tia is the object score value. It is noted that
mean centering is included in the leverage calculation as the
N
1 -term. The leverage-corrected fit residual is obtained as
A
ðyobs;i
yA
fit;i Þ
fi;LC ¼ (3)
ð1
hA
i Þ
where yA fit;i is the fitted response for object i with A
components. The root mean square error of leverage
correction (RMSELC) then follows as
vffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi
u
u X N
RMSELC ¼ tdf
1 A Þ2
ðfi;LC (4)
i¼1
where df stands for degrees of freedom. This conversion of fit
residuals to predictive ones is correct for multiple regression
models with orthogonal variables, including principal
component regression (PCR), but inappropriate for PLS.
As a general principle, degrees of freedom are consumed for
each piece of information that enters the model building
than top-down PCR since the response values are deployed.
This may explain why Lorber and Kowalski [28] found
leverage correction to work better for top-down PCR than for
PLS. Consequently, when applied to a PLS model, leverage
correction should be regarded only as a ‘quick-and-dirty’
alternative to the more time-consuming CV. It is noted that
van der Voet [43] has proposed more rigorous degrees of
freedom. Unfortunately, their calculation requires CV hence
their utility is limited in the current context.
2.3. Size of the eigenvalue
The eigenvalue is related to the amount of variation
explained by a component. It is calculated in SIMCA as [30]
SSXðAÞ
lðAÞ ¼ minðN;KÞ (5)
SSXð0Þ
where min(N, K) is either the number of objects (N) or
variables (K) in the X-data matrix, SSXðAÞ is the sum of
squares explained by component A and SSXð0Þ is the total
sum of squares that is initially present in the X-data (A ¼ 0).
This formula enables one to see how many objects (N) or
variables (K) are explained by a component. For example PLS
is often used to calibrate X-data for which the number of
objects is (much) smaller than the number of variables. In that
J. Chemometrics 2007; 21: 427–439
DOI: 10.1002/cem

case, the eigenvalue will show how many objects are
explained by the component. The relative amount of
variation explained by a model follows as the ratio of the
sum of the SSX(A)s and SSXð0Þ. It should be noted that the
eigenvalue is less popular for PLS component selection than
CV or leverage correction, although it constitutes the basis for
many component selection rules in PCA [44].
2.4. SIMCA
1. A component is significant according to Rule 1 when the
amount of predicted (CV) variance Q2 > limit. With a PLS
model, Limit ¼ 0 for models with more than 100 objects
and Limit ¼ 0.05 for models with 100 objects or less.
2. A component is significant according to Rule 2 when at
least one Y-variable for a PLS model has Q2V >limit. Q2V is
the fraction from one Y-variable, that is Q2V ¼ 1

PRESS=SSy, where SS is sums of squares for one response
vector y. The total from all responses is Q2.
A PLS component is not significant if:
1. Rule 1 or 2 is not fulfilled.
2. The data have insufficient number of degrees of freedom
after previous component, that is if (N
A) or (K
A) ¼ 0.
3. The explained variance for Y (PLS) is less than 1% and
no single Y-variable has more than 3% (PLS) explained
variance.
2.5. Unscrambler
For Unscrambler [31], the selection follows as the software
recommendation for CV and leverage correction. This
recommendation is based on the minimization of the
following criterion: [45]
CritðAÞ ¼ RMSEVðAÞ þ c  A  RMSEVð0Þ (6)
where RMSEV stands for the root mean square error of
validation (irrespective of how this has been estimated), the
number of components is given in parentheses and c denotes
a ‘punish factor’ for adding components. RMSEVð0Þ
symbolizes the validation error (RMSECV or RMSELC)
when the model consists of the center only (zero com-
ponents), and is therefore expected to resemble the spread in
the validation responses. The second term on the right-hand
side constitutes a penalty for adding components. When
RMSEV is approximately equal for two models, the smaller
one is favored by the criterion (6). The purpose is therefore to
add robustness to the component selection. It is important to
note, however, that the optimum ‘punish factor’ depends on
the data. Martens and Dardenne [45] give the value c ¼ 0.05.
Westad and Martens [46] mention that ‘The ‘‘punish factor’’
may be tuned to a smaller value for models with low
explained y-variance’ [relatively high RMSEP (RMSEP
stands for root mean square error of prediction, a synonym
for RMSEV)]. Three per cent seems to be a good ad hoc value,
although setting it lower will not affect the model
performance in general’. However, the value c ¼ 0.01, as
deployed by Høy et al., [47] may lead to severe under-fitting
[48]. Finally, Lorber and Kowalski [28] have found that CV
and leverage correction perform similarly for top-down PCR,
Copyright # 2007 John Wiley & Sons, Ltd.
PLS component selection 429
but not for PLS; hence one should not expect a single value to
work well for both validation methods when PLS is used.
2.6. The proposed randomization test
2.6.1. Why a statistical test?
PLS component selection is most frequently carried out in
practice using some sort of validation. However, each
validation-based selection rule has drawbacks. CV and
leverage correction handle the available data economically,
but like any data-based statistical test gives an interval
results and hence sometimes gives either an under-fit or an
over-fit, that is they reach the minimum RMSEV for a lower
or higher model rank than would be achieved using an
infinitely large independent validation set. In addition, the
estimated RMSEs carry a considerable uncertainty, as noted
by Martens and Dardenne [45]. When extreme outliers are
removed, the mean squared error (square of RMSEV) is
distributed approximately proportional to a x2-variable [49].
Asffiffiffiffiffiffiffiffiffiffiffi
p a result,
ffi the relative uncertainty can be estimated as
1=2N in which N is the number of objects. This estimate can
be used in the selection rule given by Breiman et al., [41]
which starts with a candidate model and works backwards,
see Figure 1. However, this procedure does not account for
the fact that the models are nested; hence subsequent RMSEs
are correlated.
In summary, it is not clear with CV-based selection rules
how the uncertainty in the input data translates into an
uncertainty in the output number of PLS components. Here
we try to develop a distribution-based statistical test that
may provide more objective guidance.
2.6.2. The procedure for successive PLS components
The proposed method assesses the statistical significance of
each individual component that enters the model. Theoretical
approaches to achieve this goal (using a t- or F-test) have
been put forth but they are all based on unrealistic
assumptions about the data, for example the absence of
spectral noise (see Section 2.7). A pragmatic data-driven
approach is therefore called for. The randomization test is
entirely data-driven and therefore ideally suited for avoiding
unrealistic assumptions. For an excellent description of this
methodology, see van der Voet [20]. The rationale behind the
randomization test in the context of regression modeling is
illustrated in Figure 2. Randomization amounts to permuting
indices. For that reason, the randomization test is often
referred to as a permutation test. In quantitative structure
activity relationship (QSAR) applications it is known as
‘Y-scrambling’. Clearly, a complete scrambling of the
elements of Y while keeping the corresponding numbers
in X fixed destroys any relationship that might exist between
the X-and Y-variables. Randomization therefore yields PLS
regression models that should reflect the absence of a real
association between the X- and Y-variables—in other words:
insignificant models. However, in practice any scrambling of
the Y-data still leaves some correlation between the
scrambled and original data which needs to be taken in
account. For each of these random models, a test statistic, T, is
J. Chemometrics 2007; 21: 427–439
DOI: 10.1002/cem
430 S. Wiklund et al.
0.25 0.2

0.2
(a) (b)
0.15

RMSEV
0.15
0.1
0.1

0.05
0.05

0 0
0 2 4 6 8 10 0 2 4 6 8 10
PLS components PLS components

Figure 1. Gas oil data set: RMSEV as a function of PLS components from (a)
sevenfold CV (84 objects) and (b) independent validation set (155 objects). The
RMSEV estimate for the five-component candidate model (*) is adorned with
uncertainty limits (---). The selection rule given by Breiman et al. [41] checks
whether smaller models yield an RMSEV estimate that is within the uncertainty
limits associated with the candidate model. For the current example data set, the
candidate model would be kept.

calculated. We have opted for the covariance between the
t-score and the Y-values because it is a natural measure of
association, thus T¼ ðt0 yÞ=N. Clearly, the value for a test
statistic obtained after randomization should be indistinguish-
able from a chance fluctuation, except to the small remaining
correlation to the original Y. For this reason, it will be referred
to as a ‘noise value’. Repeating this calculation a number of
times generates a histogram for the null-distribution, that is,
the distribution that holds when the component is due to
chance—the null-hypothesis (Ho ). Next, a critical value is
derived from the null-distribution as the value exceeded by a
certain percentage of noise values (say 5 or 10%). Finally, the
statistic obtained for the original data—the value under
test—is compared with the critical value. The (only)
difference with a ‘conventional’ statistical test is that the
critical value follows as a percentage point of a data-driven
histogram of noise values instead of a (fixed) theoretical
distribution that is tabulated, for example t or F. For
illustrative examples, see Figure 3.
2.6.3. Computational details
Step-by-step explanation. The procedure consists of the
following steps:
1. Initialize, that is (1) set the minimum model dimension-
ality for the X- and Y-data, A, to zero, (2) optionally
remove the mean from the data, yielding centered data
sets X0 and y0 (the 0th residuals) and (3) set the maximum
number of components to be considered to Amax.
2. Increase the model dimensionality, A, by one. Compute
the residual data sets as XA ¼ XA
1
tA pTA and yA ¼
yA
1
tA qA , where t is the score vector, and p and q
are the PLS loadings for the X- and Y-data, respectively.

1 2 4
2 1 5

N 4 3
ordered permuted y2 y4 x1
indice s indices
y1 y5 x2 dis t r i b u t i o n
PLS under H o
y1
y4 y3 xN
y2 ord e red
randomised
y ve c t o rs X ma t rix

yN
ordered
y-v e cto r

Figure 2. Generating the distribution under the null-hypothesis (Ho ) by build-

ing a series of PLS models after pairing up the observations for predictor (X)
and response (Y) variables at random. Any result obtained by PLS modeling
after randomization must be due to chance. Consequently, the statistical
significance of the value under test follows from a comparison with the
corresponding randomization results.

Copyright # 2007 John Wiley & Sons, Ltd. J. Chemometrics 2007; 21: 427–439
DOI: 10.1002/cem
 PLS component selection 431

A=3 A=6
250
300
(a) (b)
250 200

Frequency
200
150
150
100
100

50 50

0 0
0 0.2 0.4 0.01 0.02 0.03
Test statistic Test statistic

Figure 3. Gas oil data set: comparison of the histogram of noise values and the
value under test (---) for PLS components (a) 3 and (b) 6. For component 3, 3.3%
of the noise values exceed the value under test. This component is therefore
significant, depending on the confidence level deployed. Component 6 is clearly
insignificant, because the value under test is exceeded by 36% of the noise
values.

3. Compute the value under test, TA (cov(tN,yA)), from XA Chhikara and Folks [53]. The inverse Gaussian probability
and yA. density function is given by
4. Generate P permutations of the rows of yA, resulting in !
 g 
gðx
mÞ2
yA,p (p ¼ 1;    ; P). Compute for XA and yA;p (p ¼ 1;    ; P), gðx; m; gÞ ¼ exp x > 0; m; g > 0 (7)
the noise value, TA,p (p ¼ 1;    ; P) from the first PLS 2px3 2m2 x
component. where x represents the data (noise values) to be modeled, and
5. If A < Amax, return to Step 2. m and g are location and shape parameters. It holds that

It is emphasized that the calculation of the test statistics TA m ¼ mean

is conducted conditional on previous components having ðmeanÞ3 (8)
g¼
been (fully) extracted from the data. Since this is a crucial variance
aspect of the procedure, it is called conditional model where mean and variance denote the true values. From these
dimensionality test (COMODITE), sometimes this is called relationships, one obtains the method of moments estimators
partial model dimensionality test [30]. The currently for m and g as
proposed randomization test intends to improve on the ^ ¼x
m
one introduced by Sheridan et al. [23] Characteristic for their ðxÞ3
method is that only Y-residuals are updated in Step 2. This g^ ¼ (9)
P
P
incomplete updating is essentially correct for the PLS model
1
P
1 ðxp
xÞ2
p¼1
under test, for which the predictor and response variables are
‘correctly’ paired up, but not after randomization. Because where x denotes the observed mean and P is the number of
previous components are not fully removed, the noise values data points (permutations). A better fit is, however, obtained
are severely overestimated. As a result, the Sheridan test will by the maximum likelihood estimators (not shown),
have a tendency to under-fit, which will be illustrated using m^ ¼x
simulated data. g^ ¼ P  P 
How many permutations are needed? It is common in X 1 1 (10)


bootstrapping to use 1000 ‘draws’; this value has been used p¼1
xp x
also here in light of the close analogy between randomization
tests and bootstrapping [50]. The calculation of the risk involves evaluating a percentage
Histogram of noise values. Quite often, the test statistic TA point of the inverse Gaussian cumulative distribution
(far) exceeds all noise values for low-numbered components. function (cdf),
Then a smooth approximation of the histogram may provide pffiffiffi   
g^  ðTA =^m
1Þ 2^
g
a more sensible estimate of the associated risk of over-fitting ^ ; g^ Þ ¼ F
GðTA ; m pffiffiffiffiffiffi þ exp
TA m^
than the conventional value, 1/P. Because the test statistic is  pffiffiffi 
proportional to a correlation, it is tempting to use Fisher’s
g^  ðTA =^ m þ 1Þ
F p ffiffiffiffiffiffi (11)
Z-transform [51] (or Hotelling’s transform [52] for few TA
objects, i.e. 25) to convert it to a normally distributed where FðÞ denotes the standard normal cdf. Implementing
variable. This approach fails for PLS (results not shown), this expression ‘at face value’ may lead to numerical
because the two vectors involved in the calculation (t-score problems because the second term on the right-hand side
and response) are functionally related. We have therefore involves multiplying a large number, expðÞ, which may
opted for an empirical approach and selected the so-called cause an overflow, by a very small number, FðÞ. We have
inverse Gaussian distribution, because it is often suited for therefore implemented the solution proposed by Dennis et al.
modeling positive and/or positively skewed data, see [54].
Copyright # 2007 John Wiley & Sons, Ltd. J. Chemometrics 2007; 21: 427–439
DOI: 10.1002/cem
432 S. Wiklund et al.

N=84 N=28
50
250 (a) 40 250 (b)
30
20

Frequency
200 200
10
0
150 0.04 0.05 06 150

100 100

50 50

0 0
0.02 0.04 0.06 0.08 0.005 0.01 0.015 0.02
Test statistic Test statistic

(c) 0.02
(d)
0.08
Test statistic

0.06 0.015

0.04 0.01

0.02 0.005

0 0.2 0.4 0.6 0.8 1 1.2 0 0.2 0.4 0.6 0.8 1 1.2
2 2
r r

Figure 4. Gas oil data set: comparison of the histogram of noise values with
inverse Gaussian fit (—) and the value under test (---) for PLS component 5 for
(a) full and (b) reduced calibration set; corresponding scatter plots for (c) full, and
(d) reduced calibration set.

Finally, we have often observed that the tail is over-
estimated, see Figure 4 for an illustrative example; hence the
fit leads to conservative estimates of risk of over-fitting (often
still much smaller than 1/P, but more realistic).
Scatter plot. Lindgren et al. [18] have noted that the usual
histograms convey limited information. Chance correlations
between the randomized and original response vectors tend
to increase with decreasing number of objects. This useful
piece of information is lacking from the histogram repres-
entation. However, it is easily accommodated for in a scatter
plot where one axis (e.g. the ordinate) is reserved for the test
statistic and the other one (e.g. the abscissa) for the
correlation, see Wold and Eriksson [55] for a related plot.
Figure 4 presents a comparison with the usual histograms. It
is observed that by reducing the number of calibration
objects by a factor of three, the risk of over-fitting increases
from 2  10
3 to 3.6%. Simultaneously, chance correlations
increase, but certainly not to the point where one would start
to worry about the validity of the estimated risk. Thus, it
appears that the scatter plot representation enables one to
make a visual link between uncertainty in the input data (e.g.
is the calibration set large and diverse enough?) and the
uncertainty in aspects of the output model.
2.7. Alternative statistical tests
Haaland and Thomas [3] and Osten [4] proposed F-tests,
which have been criticized by Van der Voet [20]. More
recently, Holcomb et al. [9] proposed another F-test.
However, this test assumes full-rank errorless predictor
variables, which is extremely restrictive in practice. Finally,
Copyright # 2007 John Wiley & Sons, Ltd.
Lazraq and Cléroux [11] proposed a t-test. However, this test
tacitly assumes the scores and response vector to be
independent, which is clearly violated for PLS. The test
statistic therefore fails to follow the hypothesized
t-distribution under the null-hypothesis. The problems with
this test are illustrated by these authors’ recommendation to
test at the unusual level of 30%.
3. EXPERIMENTAL
3.1. Simulated data
Testing the claim that the currently proposed randomization
test improves on the one introduced by Sheridan et al. [23]
requires data sets for which the true dimensionality is clearly
defined. We therefore conducted a small Monte Carlo
simulation study where a three-component data set was
constructed as follows. First, Y-data for a calibration and
prediction set, 50 samples in each set, were generated from a
uniform distribution between 0 and 1. Next, the correspond-
ing X-data were constructed by multiplying the ‘noiseless’
Y-data and the profiles depicted in Figure 5 (K ¼ 100). Finally,
normally distributed noise was added to the ‘noiseless’
X- and Y-data with standard deviations 0.01 and 0.05,
respectively. For both randomization tests, an identical
initialization of the pseudo-random number generators is
deployed to enable a fair comparison.
3.2. Real data sets
The PLS component selection rules are tested on 10 real data
sets, namely four near-infrared (NIR), two red/green/blue
J. Chemometrics 2007; 21: 427–439
DOI: 10.1002/cem
 PLS component selection 433

between 15 and 30 mm. The water content was measured on

each plug, creating the response vector. The data set is
divided into a calibration and a validation set (see Table I).

3.2.3. ISO-brightness
This data set contains Vis-NIR measurements (400–2500 nm)
collected from samples of ground spruce wood. The
response variable is the ISO-brightness value obtained from
the peroxide bleached thermo mechanical pulp refined from
the measured wood.

3.2.4. Gas oil

This data set contains NIR measurements of gas oil samples
(4900–9000 cm
1). The property of interest is the hydrogen
content determined by NMR. The data set is divided into a
calibration and a validation set (see Table I).

Figure 5. Simulated profiles for Y-variable 1 (—), 2 (– – –) 3.2.5. RGB

and 3 (---).
(RGB) imaging, two nuclear magnetic resonance (NMR) and
two QSAR applications. Since most of these data sets are
detailed elsewhere [56–64],only a short description follows.
Table I gives an overview and also indicates how four data
sets are further divided in a calibration and validation set in
order to present a final test of the different component
selection rules. The data sets can be downloaded from http://
www.chemometrics.se.
3.2.1. Carra
This data set contains NIR measurements (1100–2500 nm)
collected from carrageenan powders with varying compo-
sitions. In the original data, the carrageenan is described by
the relative amounts of five carrageenan types; here only a
single response, kappa, is modeled. The data set is divided
into a calibration and a validation set (see Table I).
3.2.2. Water
This data set contains Vis-NIR measurements (350–2500 nm)
made at the surface of a wood plug using a LabSpex1 Pro
portable NIR spectrometer, model number A108310. Wood
plugs were collected from cut spruce logs in the northern
part of Sweden in 2001 (June 5–July 30) using a drill. The size
of each sample was 12 mm in diameter and the length varies
This data set contains histograms that are derived from RGB
images of bark from two wood species, namely Norway
spruce and Scots pine. The image pixels are binned according
to their intensities, thus providing 28 ¼ 256 bins for each color
channel and a total of 768 bins for each image. These 768 bins
are used as predictors while the wood specie class (spruce or
pine) is the response variable.
3.2.6. Wavelet
This data set contains RGB images (300  600 pixels) of pine
and spruce bark, compressed into 5000 wavelet coefficients.
The response variable is the wood specie. The data set is
divided into a calibration and a validation set (see Table I).
3.2.7. Internodes
This data set contains 1H HR/MAS NMR (high resolution
magic angle spinning NMR) spectra from transgenic and
wild type poplar trees. Each NMR spectrum was reduced to
657 spectral integral regions of equal width (0.02 ppm). All
objects were aligned and normalized prior to analysis. The
response vector is derived from the internodes of the trees,
creating a growth or time related response. Only the
transgenic type takes part in the modeling.
3.2.8. Poplar class
For the second NMR data set the two classes create the
response vector.

Table I. Overview of the real data sets

Objects

Data set Type Response Total Calibration Validation X-variables References

Carra NIR Kappa 128 102 26 699 56

Water NIR Water content 647 431 216 2151 57
ISO-brightness NIR ISO-brightness 42 42 — 1050 58
Gas oil NIR Hydrogen content 239 84 155 2128 59
RGB Image Wood specie 146 146 — 768 —
Wavelet Image Wood specie 146 98 48 5000 60
Internodes NMR Internodes 24 24 — 655 61
Poplar class NMR Poplar class 45 45 — 655 61
Kelder QSAR Log RA 55 55 — 24 62, 63
Hexapep QSAR BA2 16 16 — 18 64

Copyright # 2007 John Wiley & Sons, Ltd. J. Chemometrics 2007; 21: 427–439
DOI: 10.1002/cem
434 S. Wiklund et al.

3.2.9. Kelder
In this data set Kelder and Greven described a Free-Wilson
analysis of the behavior-modifying activities of ACH-
T-related peptides. The data contain 55 objects and 24
variables.
3.2.10. Hexapep
This data set contains hexapeptides that were synthesized
according to a molecular design and then tested with regards
to the biological activity BA2. The Z-scaled amino acids were
used to characterize the hexapeptide sequence giving 18
predictor variables. The response variable is BA2.
3.3. Calculations
All calculations concerning the randomization test were
carried out using Matlab version 7.0 (The Mathworks,
Natick, MA). A copy of the program is available on request.
the test. By contrast, the test introduced by Sheridan et al. [23]
assesses only the first component to be significant. As
indicated in Subsection ‘Step-by-step explanation’, this test
tends to under-fit because contributions from earlier
components are not properly removed during the random-
izations. Consequently, the ‘noise values’ are severely
overestimated.
4.2. Real data sets
Table III gives an overview of the selected model ranks using
SIMCA, Unscrambler and the present randomization test
while Table IV details the results for the randomization test
(1000 randomizations). All data sets are mean-centered prior
to PLS modeling. Some data sets required further pre-
treatment such as standard normal variate (SNV) scaling,
multiplicative signal correction (MSC) and unit variance
scaling (UVS). Table V presents root mean square errors
(RMSEs) obtained for the independent validation set.

4. RESULTS AND DISCUSSION

4.1. Simulated data
External validation using the (independent) prediction set
yields a minimum RMSEP for the three-component PLS
model for all three Y-variables (Table II). Consistent with this
result, the currently proposed randomization test yields
small significance levels for the first three PLS components,
whereas the fourth PLS component is clearly insignificant by
4.2.1. Detailed
Carra
 Spectra require pre-treatment due to offset variations that
are unrelated to response values. SNV-scaling appears to
work well over the entire wavelength range (Figure 6).
 SNV-scaling leads to smaller RMSEV for CV (not shown)
and independent validation set (Table V).

Table II. Prediction and randomization test results for simulated data sets

Y1 Y2 Y3

PLS component RMSEP a (%) current Sheridan RMSEP a (%) current Sheridan RMSEP a (%) current Sheridan

1 0.239 3  10
4 3  10
4 0.224 3  10
3 3  10
3 0.245 8  10
3 8  10
3

2 0.085 0.25 100 0.131 0.03 100 0.052 4  10
3 100
3 0.049 0.04 100 0.050 4  10
3 100 0.049 0.50 100
4 0.054 87.2 100 0.055 58.7 100 0.061 29.3 100

The models with minimum RMSEP are indicated in bold. The symbols are explained in the text.

Table III. Selected number of PLS components using LOO CV, CV with r ¼ 7 segments, size of the eigenvalue, rules of
significance, leverage correction and randomization test

SIMCA Unscrambler

Pre-treatment CV CV Rules of CV CV Leverage Randomization

Data set (þcentering) (LOO) (r ¼ 7) Eigenvalue significance (LOO) (r ¼ 7) correction test

Carra — 6 6 2 6 6 6 6 6–9
SNV 5 8 4 5 5 5 5 8–10
Water — 8 8 3 9 6 8 11 9
ISO-brightness — 6 3–6 1 6 7 5 18 3–6
MSC 5 5 2 5 5 5 18 3–5
Gas oil — 2 2 3 2 2 2 2 5
RGB — 3 3 8 5 5 6 10 4
Wavelet — 7–10 7–10 4 6 5 4 6 10
Internodes — 1 1 3 1 1 1 6 1
Poplar class — 4 4 3 5 4 4 7 4
Kelder — 2 2 2 2 3 5 3 2
Hexapep UVS 3 — 2 3 11 — 4 4

For more detailed results from the randomization test, see Table IV.

Copyright # 2007 John Wiley & Sons, Ltd. J. Chemometrics 2007; 21: 427–439
DOI: 10.1002/cem
 PLS component selection 435

Table IV. Risk of over-fitting (in %) for individual components, estimated from 1000 randomizations

PLS component
Pre-treatment
Data set (þ centering) 1 2 3 4 5 6 7 8 9 10

3
3
Carra — 21 0.06 1  10 0.23 0.32 2  10 30 0.03 3.1 11
SNV 3  10
3 0.07 8  10
3 6  10
3 2  10
6 1  10
3 0.50 0.07 7.5 0.01
Water — 2  10
5 1  10
6 4  10
6 2  10
5 0.13 6  10
4 1  10
4 6  10
5 7  10
3 16
ISO-brightness — 24 0.22 4.1 44 26 0.80 41 73 24 68
MSC 7.9 1.4 2.0 16 0.12 44 37 40 60 29
Gas oil — 9  10
4 0.02 3.3 6  10
4 2  10
3 36 43 38 11 16
RGB — 7  10
7 7  10
5 0.70 3  10
3 37 28 37 28 20 87
Wavelet — 1  10
7 1  10
3 0.01 1  10
7 4  10
6 7  10
5 4  10
3 0.09 6  10
4 0.03
Internodes — 6  10
3 20 11 37 41 13 4.6 21 13 2.6
Poplar class — 0.01 0.11 0.30 1.4 11 8.8 2.9 8.5 19 12
Kelder — 2  10
6 0.70 24 33 91 100 100 100 100 100
Hexapep UVS 32 0.36 5.1 2.2 58 82 88 95 80 98

Table V. RMSEV for independent validation set

Number of PLS components

Pre-treatment
Data set (þ centering) 1 2 3 4 5 6 7 8 9 10

Carra — 18.5 14.9 7.30 7.04 6.04 3.21 3.11 2.35 1.79 1.85
SNV 14.2 7.29 6.75 5.86 3.22 2.39 1.68 1.52 1.16 1.36
Water — 9.17 6.99 6.70 6.07 6.04 5.47 5.39 5.43 5.44 5.31
Gas oil — 0.18 0.10 0.085 0.055 0.045 0.039 0.039 0.037 0.039 0.038
Wavelet — 0.27 0.24 0.22 0.23 0.23 0.23 0.22 0.22 0.22 0.22

 With the exception of the eigenvalue criterion, general Water

agreement for SIMCA and Unscrambler when spectra
are not pre-treated (Table III).
 Erratic behavior for randomization test when spectra are
not pre-treated. The first component is insignificant by the
test (Table IV), whereas successive components clearly are.
In other words, the components do not enter the model in
the natural order of decreasing relevance. Note that the
first component has the same effect as SNV on RMSEV for
the independent validation set (Table V). It follows that
this behavior can be rationalized and is therefore indica-
tive that data pre-treatment that needs to be done prior PLS
modeling.
 The randomization test gives support for 8 or 10 com-
ponents after SNV-scaling.
 Agreement between SIMCA and Unscrambler CV
(Table III). Disagreement of CV and the eigenvalue and
leverage correction.
 The randomization test gives clear support for nine com-
ponents, in agreement with SIMCA’s rules of significance.
ISO-brightness
 With the exception of the eigenvalue criterion and leverage
correction, good agreement between SIMCA and
Unscrambler before and after pre-treatment with MSC
(Table III).
 Before pre-treatment with MSC, components 1, 3 and 4 are
not significant according to SIMCA’s rules of significance,

(a) (b)
Signal

1100 1500 1900 2300 1100 1500 1900 2300

Wavelength [nm] Wavelength [nm]

Figure 6. Carra data set: (a) raw and (b) SNV-scaled calibration set
spectra in arbitrary signal unit.

Copyright # 2007 John Wiley & Sons, Ltd. J. Chemometrics 2007; 21: 427–439
DOI: 10.1002/cem
436 S. Wiklund et al.
but 2, 5 and 6 are; this is in fairly good agreement with the
randomization test where components 2, 3 and 6 are
significant. After pre-treatment with MSC, component 1
is not significant according to the rules, which is corrobo-
rated by the randomization test.
Gas oil
 Good agreement between SIMCA and Unscrambler selec-
tion rules, while the randomization test stands out
(Table III). The decrease of RMSECV upon going from
two to five components (Figure 2a) is incorrectly found to
be insignificant by the SIMCA and Unscrambler selection
rules. Caution to avoid over-fitting has led to under-fitting:
RMSEV for the independent validation set increases from
0.045 to 0.10 when dropping components from 5 to 2
(Table V).
 Nonmonotonic behavior for randomization test: the esti-
mated tail probability for component 3 is 3.3% (Figure 3a),
while successive components 4 and 5 reach much smaller
values (see e.g. Figure 4 for component 5). Component 6 is
clearly insignificant by the test (Figure 3b). The reason for
this nonmonotonic behavior maybe the presence of tiny
nonlinear substructures in the spectra corresponding to
high response values. Indicative is that high response
values are underestimated by their prediction: clearly so
for the two-, and to a lesser extent for the five-dimensional
model (Figure 7).
RGB
 Disagreement between SIMCA and Unscrambler selection
rules (Table III).
 The randomization test clearly supports four components.
Such an uninterrupted series of highly significant com-
ponents can be considered as ideal behavior of the test.
Wavelet
 The wavelet transformed data set is a little bit tricky since
RMSECV never attains a local minimum. This is due to the
fact that X represents compressed image data and because
of the complex structure many latent variables correlate to
A=2 A=5
NIR prediction
14 14
13 13
12 12
11 (a) 11 (b)
11 12 13 14 11 12 13 14
Reference value Reference value
Figure 7. Gas oil data set prediction results for independent
validation set obtained using (a) two and (b) five PLS com-
ponents.
Copyright # 2007 John Wiley & Sons, Ltd.
Y. Y itself is a discrete, noiseless variable. As a result, both
CV and the randomization test will find many significant
components. Therefore we chose to display an interval of
7–10 components for SIMCA’s CV (Table III). This result is
in good agreement with the randomization test (Table IV).
It also agrees fairly well with the minimum RMSEV
obtained between 3 and 10 components for the indepen-
dent validation set (Table V).
Internodes
 With the exception of the eigenvalue criterion and leverage
correction, all SIMCA and Unscrambler selection rules
suggest that only the first component is predictive.
 The randomization test clearly supports only the first
component.
Poplar class
 Some disagreement between CV and other SIMCA and
Unscrambler selection rules.
 The randomization test supports four components, in
agreement with CV.
Kelder
 Disagreement between SIMCA and Unscrambler selection
rules (Table III).
 The randomization test clearly supports two components,
in agreement with the SIMCA selection rules.
Hexapep
 Since this data set originates from a design in X-space, only
the ‘mildest’ form of CV is used, namely LOO CV.
 Disagreement between SIMCA and Unscrambler selection
rules (Table III).
 Erratic behavior for CV (Figure 8). The purpose of CV is to
generate prediction residuals from the calibration set.
Consequently, the average cross-validated error
20
15
RMSE
10
5
0
0 2 4 6 8 10
PLS components
Figure 8. Hexapep data set: RMSE as a function of PLS
components from fit (*), LOO CV (&) and leverage correc-
tion (*).
J. Chemometrics 2007; 21: 427–439
DOI: 10.1002/cem

(RMSECV) should not differ much from the average fit
error, especially for under-fitting models. The failure of CV
can be explained from the designed character of the data.
In a strict sense, CV can only be used if the data constitute a
random sample from a population. In a loose sense, the
effect of a design can be negligible for practical purpose
but only if the calibration set is large enough, which
apparently is not the case here (N ¼ 16).
 Erratic behavior for the randomization test. The first com-
ponent is insignificant by the test (Table IV), whereas three
successive components are, depending on the level of
confidence. This is in agreement with SIMCA’s rules of
significance which also claim that the first component is
insignificant, but the second and third are significant.
 The Unscrambler recommends four components based on
applying criterion (6) to the RMSEV from leverage correc-
tion. This happens to agree with the randomization test.
However, the associated RMSEV is twice the value
achieved for the global minimum at eight components
(1.42 vs. 0.71). In this case, the user is confronted with a
difficult choice: follow the recommendation based on the
criterion (6), which depends on an ad hoc value for c, or
accept the rank that yields the global minimum? The
results of the randomization test are easier to interpret.
4.2.2. General
The suggested optimum number of components varies
between the two implementations of CV and the other
selection rules available in SIMCA and Unscrambler, as well
as the randomization test (Table III). Although this
disagreement may be confusing for the less experienced
user, indeed it is important to remember that an exact
number does not exist, but rather an appropriate interval. It is
comforting that CV as well as the randomization test seem to
find this interval in a reliable way as indicated by the results.
Picard and Cook [32] note that ‘In the analysis of large data
sets, an appropriate way to proceed is often not immediately
apparent. Consequently, some aspects of exploratory data
analysis naturally arise. Competing models may be tempor-
arily entertained and the final choice of a predictive equation
is influenced by many factors, including the personal
experiences and prejudices of the investigator’. In other
words, imprecise results only present an immediate problem
if a definite choice is desired. Otherwise, one could simply
keep several competing models and put them all to the test
on future data. This future prediction-testing phase would
then sometimes lead to the final model—a sound and safe
strategy. However, for the common selection rules studied in
this work, one observes that competing models may have
differing number of components. In these cases, it may be
hard to defend why certain models are considered as
competitors. By contrast, the randomization test leads to
models for which the disagreeing number of components has
a simple interpretation. Clearly, a higher degree of
confidence leads to smaller models and vice versa. Actually,
this is the situation for any statistical test including all the
ones looked at here; it often needs to be supplemented with a
rule that encourages a simpler model before a more
complicated one. Finally, it is emphasized that the proposed
Copyright # 2007 John Wiley & Sons, Ltd.
PLS component selection 437
randomization test need not be used in stand-alone mode. It
can also be used in an interactive fashion to guide the
component selection according to another rule, for example,
CV.
5. CONCLUSIONS
 The simulation results illustrate that the currently pro-
posed randomization test improves on the one introduced
by Sheridan et al. [23].
 The randomization test assumes the components to enter
the model in a natural order, that is according to their
relevance for describing the response variable. After all,
this property is a theoretical advantage of PLS over
methods like PCR [65]. The ideal natural order is not
always attained for practical data. It can, for example be
distorted by an improper pre-treatment of the predictor
variables. Fortunately, the resulting erratic behavior can
point at useful pre-treatment methods. Likewise, the test
appears to be sensitive to nonlinearities in the data.
 The proposed randomization test, possibly in combination
with other PLS component selection rules will make the
choice of the right model easier for the user.
 A comparison with the recently introduced Monte Carlo
CV [35–37] is an interesting topic for future research.
Likewise, a more systematic evaluation of the test for
designed data seems to be warranted.
 One area where CV works poorly both for PLS and PCR is
design of experiments, where exclusion of data has large
consequences for modeling. Here the randomization test
should have merit.
Acknowledgements
Chris Brown and Alejandro Olivieri are thanked for supply-
ing Matlab code for evaluating the inverse Gaussian fit.
REFERENCES
1. Wold S, Eriksson L, Sjöström M. PLS in chemistry. In The
Encyclopedia of Computational Chemistry, Schleyer PVR,
Allinger NL, Clark T, Gasteiger J, Kollman PA, Schaefer
HF, III, Schreiner PR (eds). Wiley: Chichester, UK, 1999;
2006–2020.
2. Denham MC. Choosing the number of factors in partial
least squares regression: estimating and minimizing the
mean squared error of prediction. J. Chemometr. 2000; 14:
351–361.
3. Haaland DM, Thomas EV. Partial least-squares methods
for spectral analyses. 1. Relation to other quantitative
calibration methods and the extraction of qualitative
information. Anal. Chem. 1988; 60: 1193–1202.
4. Osten DW. Selection of optimal regression models via
cross-validation. J. Chemometr. 1988; 2: 39–48.
5. Wakeling IN, Morris JJ. A test of significance for partial
least squares regression. J. Chemometr. 1993; 7: 291–304.
6. Höskuldsson A. Dimension of linear models. Chemometr.
Intell. Lab. Syst. 1996; 32: 37–55.
7. Messick NJ, Kalivas JH, Lang PM. Selecting factors for
partial least squares. Microchem. J. 1997; 55: 200–207.
8. Faber NM, Kowalski BR. Propagation of measurement
errors for the validation of predictions obtained by prin-
cipal component regression and partial least squares.
J. Chemometr. 1997; 11: 181–238.
J. Chemometrics 2007; 21: 427–439
DOI: 10.1002/cem
438 S. Wiklund et al.
9. Holcomb TR, Hjalmarsson H, Morari M, Tyler ML.
Significance regression: a statistical approach to partial
least squares. J. Chemometr. 1997; 11: 283–309.
10. Liu D, Shah SL, Grant Fisher D. Choice of latent expla-
natory variables: a multiobjective optimization approach.
J. Chemometr. 2000; 14: 79–92.
11. Lazraq A, Cléroux R. The PLS multivariate regression
model: testing the significance of successive PLS com-
ponents. J. Chemometr. 2001; 15: 523–536.
12. Green RL, Kalivas JH. Graphical diagnostics for regres-
sion model determinations with consideration of the
bias/variance trade-off. Chemometr. Intell. Lab. Syst.
2002; 60: 173–188.
13. Li B, Morris J, Martin EB. Model selection for partial least
squares regression. Chemometr. Intell. Lab. Syst. 2002; 64:
79–89.
14. Forina M, Lanteri S, Cerrato Oliveros MC, Pizarro Millan
C. Selection of useful predictors in multivariate cali-
bration. Anal. Bioanal. Chem. 2004; 380: 397–418.
15. Fisher RA. The Design of Experiments. Oliver and Boyd:
Edinburgh, 1935.
16. Klopman G, Kalos AN. Causality in structure-activity
studies. J. Comput. Chem. 1985; 6: 492–506.
17. Thioulouse J, Lobry JR. Co-inertia analysis of amino-acid
physico-chemical properties and protein composition
with the ADE package. Comput. Appl. Biosci. 1995; 11:
321–329.
18. Lindgren F, Hansen B, Karcher W, Sjöström M, Eriksson
L. Model validation by permutation tests: applications to
variable selection. J. Chemometr. 1996; 10: 521–532.
19. Baumann K, Stiefl N. Validation tools for variable
subset regression. J. Comput.-Aided Mol. Des. 2004; 18:
549–562.
20. van der Voet H. Comparing the predictive accuracy of
models using a simple randomization test. Chemometr.
Intell. Lab. Syst. 1994; 25: 313–323.
21. van der Voet H. Corrigendum to ‘comparing the pre-
dictive accuracy of models using a simple randomization
test’. Chemometr. Intell. Lab. Syst. 1995; 28: 315.
22. Dijksterhuis GB, Heiser WJ. The role of permutation tests
in exploratory multivariate data analysis. Food Qual.
Prefer. 1995; 6: 263–270.
23. Sheridan RP, Nachbar RB, Bush BL. Extending the trend
vector: the trend matrix and sample-based partial least
squares. J. Comput.-Aided Mol. Des. 1994; 8: 323–340.
24. Stone M. Cross-validatory choice and assessment of
statistical predictions. J. Roy. Stat. Soc. B 1974; 36:
111–133.
25. Wold S. Cross-validatory estimation of the number of
components in factor and principal components models.
Technometrics 1978; 20: 397–405.
26. Allen DM. The prediction sum of squares as a criterion for
selecting prediction variables. Technical Report No. 23,
Dept. of Statistics, University of Kentucky: Lexington,
KY, 1971.
27. Martens H, Næs T. Multivariate calibration by data
compression. In Near-Infrared Technology in the Agricul-
tural and Food Industries, Williams P, Norris K (eds).
American Cereal Association: St Paul, MN, 1987; 57–87.
28. Lorber A, Kowalski BR. Alternatives to cross-validatory
estimation of the number of factors in multivariate cali-
bration. Appl. Spectrosc. 1990; 44: 1464–1470.
29. Marbach R, Heise HM. Calibration modeling by partial
least-squares and principal component regression and its
optimization using an improved leverage correction for
prediction testing. Chemometr. Intell. Lab. Syst. 1990; 9:
45–63.
30. SIMCA-P and SIMCA Pþ 10, User Guide, Umetrics AB:
Umeå, Sweden, 2002.
31. Unscrambler for Windows, User’s guide. CAMO AS: Trond-
heim, Norway, 1996.
Copyright # 2007 John Wiley & Sons, Ltd.
32. Picard RR, Cook RD. Cross-validation of regression
models. J. Amer. Stat. Assoc. 1984; 79: 575–583.
33. Shao J. Linear model selection by cross-validation.
J. Amer. Stat. Assoc. 1993; 88: 486–494.
34. Zhang P. Model selection via multifold cross validation.
Ann. Stat. 1993; 21: 299–313.
35. Xu Q-S, Liang Y-Z. Monte Carlo cross validation. Che-
mometr. Intell. Lab. Syst. 2001; 56: 1–11.
36. Gourvénec S, Fernández Pierna JA, Massart DL, Rutle-
dge DN. An evaluation of the PoLiSh smoothed
regression and the Monte Carlo cross-validation for
the determination of the complexity of a PLS model.
Chemometr. Intell. Lab. Syst. 2003; 68: 41–51.
37. Xu Q-S, Liang Y-Z, Du Y-P. Monte Carlo cross-validation
for selecting a model and estimating the prediction error
in multivariate calibration. J. Chemometr. 2004; 18:
112–120.
38. Baumann K, Albert H, von Korff M. A systematic evalu-
ation of the benefits and hazards of variable selection in
latent variable regression. Part I. Search algorithm,
theory and simulations. J. Chemometr. 2002; 16: 339–350.
39. Baumann K, von Korff M, Albert H. A systematic evalu-
ation of the benefits and hazards of variable selection in
latent variable regression. Part II. Practical applications.
J. Chemometr. 2002; 16: 351–360.
40. Baumann K. Cross-validation as the objective function
for variable selection techniques. Trends Anal. Chem.
2003; 22: 395–406.
41. Breiman L, Friedman JH, Olshen RA, Stone C. Classifi-
cation and Regression Trees. Wadsworth: Belmont, CA,
1984.
42. Reeves JB, III, Delwiche SR. SAS1 partial least squares
regression for analysis of spectroscopic data. J. Near
Infrared Spectrosc. 2003; 11: 415–431.
43. van der Voet H. Pseudo-degrees of freedom for complex
predictive models: the example of partial least squares.
J. Chemometr. 1999; 13: 195–208.
44. Jackson JE. A User’s Guide to Principal Components. Wiley:
New York, 1991.
45. Martens HA, Dardenne P. Validation and verification of
regression in small data sets. Chemometr. Intell. Lab. Syst.
1998; 44: 99–121.
46. Westad F, Martens H. Variable selection in near infrared
spectroscopy based on significance testing in partial least
squares regression. J. Near Infrared Spectrosc. 2000; 8:
117–124.
47. Høy M, Steen K, Martens H. Review of partial least
squares regression prediction error in Unscrambler. Che-
mometr. Intell. Lab. Syst. 1998; 44: 123–133.
48. Faber NM. Comparison of two recently proposed
expressions for partial least squares regression predic-
tion error. Chemometr. Intell. Lab. Syst. 2000; 52: 123–134.
49. Faber NM. Estimating the uncertainty in estimates of
root mean square error of prediction: application to
determining the size of an adequate test set in multi-
variate calibration. Chemometr. Intell. Lab. Syst. 1999; 49:
79–89.
50. Efron B. Bootstrap methods: another look at the jack
knife. Ann. Stat. 1979; 7: 1–26.
51. Fisher RA. Statistical Methods for Research Workers. Oliver
and Boyd: Edinburgh, 1925.
52. Hotelling H. New light on the correlation coefficient and
its transform. J. Roy. Stat. Soc. B 1953; 15: 193–232.
53. Chhikara RS, Folks JL. The Inverse Gaussian Distribution:
Theory, Methodology, and Applications. Marcel Dekker:
New York, 1989.
54. Dennis B, Munholland PL, Scott JM. Estimation of
growth and extinction parameters for endangered
species. Ecol. Monogr. 1991; 61: 115–143.
55. Wold S, Eriksson L. Statistical validation of QSAR
results. In Chemometric Methods in Molecular Design,
J. Chemometrics 2007; 21: 427–439
DOI: 10.1002/cem

van de Waterbeemd H (ed.). VCH Publishers: Weinheim,
1995; 309–318.
56. Dyrby M, Petersen RV, Larsen J, Rudolf B, Nørgaard L,
Engelsen SB. Towards on-line monitoring of the compo-
sition of commercial carrageenan powders. Carbohydr.
Polym. 2004; 57: 337–348.
57. Persson E, Sjöström M, Sundblad L-G, Wiklund S, Wil-
hemsson L. Fresh timber—a challenge to forestry and
mensuration. Resultat 2002; 8: 2–4.
58. Nilsson D, Edlund U, Sjöström M, Agnemo R. Prediction
of thermo mechanical pulp brightness using NIR spec-
troscopy on wood raw material. Paperi Ja Puu (Pap.
Timber) 2005; 87: 102–109.
59. Fernández Pierna JA, Jin L, Wahl F, Faber NM, Massart
DL. Estimation of partial least squares regression (PLSR)
prediction uncertainty when the reference values carry a
sizeable measurement error. Chemometr. Intell. Lab. Syst.
2003; 65: 281–291.
Copyright # 2007 John Wiley & Sons, Ltd.
PLS component selection 439
60. Nilsson D, Edlund U. Pine and spruce roundwood
species classification using multivariate image analysis
on bark. Holzforschung 2005; 59: 689–695.
61. Wiklund S, Karlsson M, Antti H, Johnels D, Sjöström M,
Wingsle G, Edlund U. A new metabonomic strategy for
analyzing the growth process in poplar tree. Plant
Biotechnol. J. 2005; 3: 353–362.
62. Kelder J, Greven HM. A quantitative study on the
relationship between structure and behavi activity of
peptides related to ACTH. Rec. Trav. Chim. Pays-Bas
1979; 98: 168–172.
63. Kubinyi H. Evolutionary variable selection in regression
and PLS analyses. J. Chemometr. 1996; 10: 119–133.
64. Eriksson L, Johansson E, Kettaneh-Wold N, Wold S.
Multi- and Megavariate Data analysis. Principles and Appli-
cations. Umetrics AB: Umeå, Sweden, 2002.
65. Helland IS. On the structure of partial least squares
regression. Commun. Stat.—Simula. 1988; 17: 581–607.
J. Chemometrics 2007; 21: 427–439
DOI: 10.1002/cem