Ridge Analysis 25 Years Later

Author(s): Roger W. Hoerl

Source: The American Statistician, Vol. 39, No. 3 (Aug., 1985), pp. 186-192
Published by: Taylor & Francis, Ltd. on behalf of the American Statistical Association
Stable URL: http://www.jstor.org/stable/2683926
Accessed: 05-03-2015 08:55 UTC

Your use of the JSTOR archive indicates your acceptance of the Terms & Conditions of Use, available at
http://www.jstor.org/page/info/about/policies/terms.jsp

JSTOR is a not-for-profit service that helps scholars, researchers, and students discover, use, and build upon a wide range of content
in a trusted digital archive. We use information technology and tools to increase productivity and facilitate new forms of scholarship.
For more information about JSTOR, please contact support@jstor.org.

Taylor & Francis, Ltd. and American Statistical Association are collaborating with JSTOR to digitize, preserve and extend
access to The American Statistician.

http://www.jstor.org

This content downloaded from 128.235.251.160 on Thu, 05 Mar 2015 08:55:41 UTC
All use subject to JSTOR Terms and Conditions
 Ridge Analysis25 Years Later
ROGER W. HOERL*

wheneverthe statistician
faces a higher-dimensional
qua-
The responsesurfacetechniquecalled ridgeanalysiswas draticresponsesurface.
originallyintroduced by Hoerl (1959) morethan25 years
ago. Despite tremendous advantagesover moreconven- 2. REVIEW
tionalresponsesurfaceprocedureswhen more thantwo Ifwe havea quadraticsurfaceinp independent
variables,
independent variablesare present,ridgeanalysishas re- theequationis of theform
ceivedlittleattentionin thestatistical
literature
sincethen,
p7 p1- p p
althoughnumerousapplicationshave appearedin engi-
neeringjournals.This situation maybe partially due to the
Y= bo + Ebjx. + E b
b1xix1+ Ebx (1)
factthatthisprocedure led to thediscoveryofridgeregres-
sion,whichhas completely overshadowed ridgeanalysisin wherethethreesetsof termsmodelthelinear,interaction,
theliteraturesince. This discussionwill briefly
reviewthe and quadraticeffects,respectively.
We can writethisin
mathematics of ridgeanalysis,its literature,
practicalad- matrixnotationas
vantages,and relationship to ridgeregression. Y = bo + b'x + ( ?12)x'Bx, (2)
KEY WORDS: Response surface;Constrained
optimiza- whereb is thep x 1 vectorof linearcoefficients, x is the
tion;Graphics. p x 1 vectorof independent variablevalues,bois thecon-
stantterm,and B is a p x p symmetric matrixwhosedi-
agonalvaluesare twicethequadratictermsand whoseoff-
diagonalvalues are the interaction terms.If thevariables
1. INTRODUCTION are standardized to have zero meanand equal standard de-
viations,ourexperimental regioncan be interpreted easily
Ridgeanalysiswas originally developedbyHoerl(1959)
as somegeometric figurewiththecenterpointas theorigin.
forexamininghigher-dimensional quadraticresponsesur-
For a rotatabledesign (e.g., Box-Wilson), this is a
faces.In contrasttoridgeregression,whichis an alternative
(hyper-)sphere definedby x'x < C2 (i.e., thecontoursof
toleastsquaresestimation inmultipleregression,
ridgeanal-
predicted responsevarianceformspheres;theactualpoints
ysisgraphically portraysthebehaviorof thesesurfacesand
maynotdefinean exactsphere).Forp = 2 (or 3, ifthree-
locatesoveralland local optimalregions.Whileemployed
dimensionalgraphicsare available),contourplots of the
in thestatistical
groupat DuPont,Hoerlwas oftenaskedto
surfacecanbe drawntolookforoptimal areas,as inFigure1.
optimizeindustrial processesinvolvingmorethanthetwo These plotsare easily interpreted and make it obviousto
orthreeindependent seeninresponse
variablestraditionally thestatistician whenhe or she is extrapolating beyondthe
surfaceliterature.Althoughthemethodof canonicalanal-
experimental region.In higherdimensions, theseplotsre-
ysis had been developedby thattime,thiswas generally
quirefixingp - 2 of thevariables,whichleads to exactly
inadequateformultidimensional surfaces,forreasonsto be thesamedifficulties as withone-variable-at-a-timeoptimi-
discussedina latersection.Possessingan engineeringback-
zation.Itshouldbe keptinmindthatresponsesurfacemeth-
ground,he felttheneedformorethana numerical optimi-
odologywas developedspecifically to avoid thesepitfalls
zationof theestimated function. Ridgeanalysis,then,was thismay
(see Box et al. 1978). Froma practicalstandpoint,
theapproachhe proposedforthisproblem.Despiteprovid-
notevenbe feasibleforpgreater than5 or6, as thickstacks
ing insightfulgraphicsof theeffectsof all factorssimul- ofplotsmaybe required toadequatelyexaminethedomain.
taneously,as wellas optimizingthesurfaceforanydistance A canonicalanalysis(see Davies 1956)can be performed
fromthecenterpointof thedesign,thetechniquedid not
forp > 2, butthislacks theadvantagesof contourplots.
immediately catchon. It maynothavebeenwidelyunder- It shiftsthereference pointaway fromtheoriginand pro-
stoodat thattime,however,as responsesurfaceanalysis vides no graphics.Withthisprocedure,it is neverimme-
was thena rathernew concept.Box and Wilson's (1951)
diatelyobviouswhenone has lefttheexperimental region
classic paper had been publishedonly eightyearsprevi-
and,infact,a canonicalanalysismayuse a pointcompletely
ously. Anotherhindrancehas been the generaldearthof outsidethisregionforreference. of ridges,
Interpretations
on analysisrelativeto designin responsesurface
literature
maxima,or bothrelativeto a pointoutsidethedomainis
methodology. The confusionof ridgeanalysiswithridge
clearlyundesirable.In addition,thepathof steepestascent
regressionhas nothelpedthissituation.Ridgeanalysisap- fromthe originalcenterpointis not given. This method
pears to deservea betterfate. The techniqueshouldbe doeshaveitsadvantages, however.Thenature ofthesurface
routinelytaughtin responsesurfacecoursesandconsidered (max,min,orsaddlepoint)is determined, andtheexistence
and natureof possiblestationary ridgesare given.
*RogerW. Hoerlis a ResearchMathematician
Ridge analysiscombinesthe advantagesof bothproce-
in theEngineeringSci-
ences Divisionof HerculesIncorporated,
ResearchCenter,Wilmington, dureswithhigher-dimensional surfaces.Basically,it pro-
DE 19894.The authorwouldliketothanktherefereesfortheirsubstantial videsa canonicalanalysisrelativetotheoriginalcenterpoint
contribution. andproducesgraphicsofsurfacesofanydimension without

186 The American Statistician,August 1985, Vol. 39, No. 3 C) 1985 American Statistical Association
This content downloaded from 128.235.251.160 on Thu, 05 Mar 2015 08:55:41 UTC
All use subject to JSTOR Terms and Conditions

- 1-
Figure1. Response Contours.Thecurveleavingtheoriginat approximately
holdingany variablesfixed.In addition,secondary(i.e.,
local) optimalregionsare examined,and insightintothe
adequacyof thefittedmodelcan oftenbe gainedthrough
examination of thegraphics.This procedure has its draw-
backsas well,however,particularly in thatitsgraphicsare
not as easily understoodas contourplots,and it depends
on theuse of a quadraticmodel.
3. INTRODUCTION TO RIDGE ANALYSIS
Using thepreviousnotation,considerfixingx'x = R2
and maximizing equation(2) subjectto thisconstraint.
For
anygivenR, some maximumY(R) is defined(withprob-
ability1 if thecoefficients
are normally Con-
distributed).
nectingthecoordinates of theY(R) valuesfor0 < R2 <C2
would displaythe coordinatesof the maximumresponse
attainableforany givendistancefromtheorigin.This is
definedto be themaximumridge,and tracesthepathof
steepestascentfromtheorigin.Thecontour plotinFigure1
(fromHoerl 1964) has themaximumridgedrawnon it. It
is merelycoincidencethatthisridgeis nearlya straight
line
at 45?. The minimum ridgeis definedsimilarly and gives
thepathof steepestdescent.Mathematically, thesepoints
are determined (2) (withuse of a La-
by differentiating
grangianmultiplier) withrespectto x, equatingto 0, and
solvingforx. The resulting equationis
x = -(B - XI)-lb,
whereX is theLagrangianmultiplier thatdetermines
x, R,
and Y. If Xl ' X2 . . . ' Xp are the rankedeigenvalues of
B, thefollowing properties result:
1. The maximum ridgeis definedby X XI.
A
2. The minimum
ridgeis definedby A < Xp.
3. Secondaryridgesare definedforAj< A < Xj+ 1.
4. At leasttwo,and at most2p, ridgesexist.
5. No tworidgeplots(Y as a functionofR) cross.
6. All ridgeplotsare monotonicwithR withat most
one exception.
to local optimaon the
The secondaryridgescorrespond
hyperspheresx'x = R2.
This content downloaded from 128.235.251.160 on Thu, 05 Mar 2015 08:55:41 UTC
All use subject to JSTOR Terms and Conditions
RIDGE
MAX
- / -Y 82.5
Y
=
81.,4
Y=
80.14
N| Y 79.2
ascent.
nidge,orpathofsteepest
450is themaximum
The lastproperty followsfromaY/aR2= X/2.Since the
slopeofanyridgeis determined byitsXvaluesandtherefore
will changesign only if 0 is includedin its A range,an
overalloptimum willexistifandonlyifall eigenvaluesare
negative(maximum),or all are positive(minimum).If B
has bothnegativeand positiveeigenvalues,a saddlepoint
(minimax)existsin thefittedsurface.If Ap ' 0, themax-
imumridgeplotwill be increasing as it movesawayfrom
theorigin,will eventuallyhita maximum,and will begin
decreasing.See Draper(1963) or Hoerl(1964) fora more
detaileddiscussionof themathematical properties.
4. PRACTICAL ADVANTAGES
Numericalcoordinatesalone do notprovidethehuman
mindwithsufficient information abouthigher-dimensional
responsesurfaces.Graphicsarenecessaryforthesamerea-
sonstheyare necessaryin regression, timeseries,or other
statistical
analyses.Withridgeanalysis,thepredictedre-
sponsecan be plottedagainstR foreach ridge,thecoor-
dinatesof anyridgecan be plottedagainstR, and a plotof
R versusA enablesthestatistician to calibrateA withboth
desiredridgeandR. (Recall thatA is an undetermined La-
grangianmultiplier and therefore one can not solve forx
simplyby specifying a particular
ridgeandR).
The five-factoryieldexamplediscussedby Hoerl(1964)
will be used to illustratetheusefulnessof theplots.This
(3)
exampleis rather old, butprovesveryillustrative ofunique
information attainable usingridgeanalysiswith"live" data.
Figure2 is theplotof Y versusR. Each separatecurve
corresponds to a differentridge,or local optimum. At each
'cusp" pointwherea newridgebegins,maximum (ofthese
two) and minimum(of thesetwo) secondaryridgesexist
thatjointlyforma cone shape. The overallmaximumand
minimum ridgesbeginatR = 0, theorigin,andeachtakes
on thevaluebo at thispoint.Notethattheboundary of the
experimental region(here approximately 2.24) is clearly
discernable.The maximumridgeis virtually a horizontal
linecomingoutfrombo,indicating thatverylittleimprove-
mentoverthecenterpointyieldis attainable.(It shouldbe
mentioned thatthefittedsurfacehas an overallmaximum
August1985, Vol. 39, No. 3
TheAmericanStatistician, 187
 5-b

Sb N~~~~~~~N
LO IN,~~~~~~~~N

LuJ

8 4-\
c:, 8
-
LU - N
Cy IOK-I'\ ,

O. 00 0
0. 58 0
. 15 . 50 L 0 50
Dw 3.

Figure2. Response Ridges. For each ridge (local optimum),we see thepredicted value of the response versus R, the distance fromthe
origin.The verticalline shows the range of experimentation.

of 87.26 at R = 1.54). The flatnessof thisridgeafter thecoordinates of themaximumpoint(at R = 1.54) and

approximately R = .6 suggeststhatthecenterpointmay thepointat whichwe beginto extrapolate, butthebehavior
be lyingjustoffa stationaryridgein thetrueyieldsurface. of the individualcoordinatesthemselvesis available. In
The natureof this stationary ridgewill become obvious situationsin whichcoordinates rapidlyachievesome "op-
shortly.Another veryusefulpiece of informationfromthis timal" level and thenremainrelatively stable(recallthat
plotis a secondaryridgebeginning at approximatelyR = theoverallmagnitude of x is beingconstrained), we can
1.20. The levelofthemaximum secondaryridgebeginning believethatwe have founda realand stablesolution.This
hereis almostidenticalto thelevelof themaximum ridge. is thecase withvariables1-4 in thisplot.The behaviorof
This suggestsa possiblealternative regionforoptimizing thefifth coefficient
is particularly however.It
interesting,
yieldthatmaybe at muchmorefeasibleoreconomiclevels remainsvirtuallyzerountilapproximately R = 1.0 andthen
of theindependent variables. increaseswildly.Whenone remembers thaton thisridgeY
Figure3 is theplot of the maximumridgecoordinates is virtually
constantafterapproximately R = .6, thereis
versusdistancefromtheorigin(R). Not onlycan we see cause forsuspicion.The drasticincreaseinx5 afterR = 1

2~. 0 _ X2

w 1 7 _

X2

= X4

U I
x . 0 i

c I i~~~~~~~~~~~~~~~~~~~~x
x -1. 0-

;
-2. 0 I~~~~~~~~~~~~~~~~~~~~~~
O. 00 0 . 50 i. 00 1 . 50 2.s ',AL
i 2h 150

Figure3. Maximum RidgeCoordinates.Thisfigure

plotsthecoordinatesofthemaximum
ridge(pathofsteepestascent) as we move
away from lineshowstherangeofexperimentation.
thecenterpoint.Thevertical

188 TheAmericanStatistician,
August1985, Vol. 39, No. 3
This content downloaded from 128.235.251.160 on Thu, 05 Mar 2015 08:55:41 UTC
All use subject to JSTOR Terms and Conditions
 2.

(nI
z _ _ _ _ _ _ _
X2

Xo4

FI
v~~~~~~~~~~~~~~~~~~~~~i
al: q 1I X3

LU I \ N
\ I

-2. 0- ,X
0. 00 0. 50 1. 00 1.50 2.00 2.50
R

Figure4. Secondary Ridge Coordinates.Thisridgebegins at approximatelyR = 1.19. The verticallineshows therange ofexperimentation.
to Figure3 forall variables except x5.
Note the similarity

hasno noticeableeffect on Y. Viewedfromthisperspective, ticularridgefor some desiredrange of R. The vertical

it appearsobviousthatx5 is notcriticalto theoptimization asymptotes are theeigenvaluesof B. Recall thatthemax-
and thatthestationary ridgemustbe almostexactlyalong imumridgecorresponds to A valuesgreaterthanthemax-
thex5 axis. This conclusionis verifiedby thecoordinates imumeigenvalue,and theminimum to X values less than
of themaximumsecondary ridgediscussedearlier,seenin theminimum eigenvalue.The overallstationarypointcor-
Figure4. The coordinatesare almostexactlythesame as responds to A = 0.
thosefortheoverallmaximumridge,exceptthatx5 goes
offin theoppositedirection as R increases.
5. ADDITIONAL CONSIDERATIONS
If one had onlysolvednumerically forthemaximumof
this fittedsurface,however,the coordinateswould have Thereare severalotherconsiderations to keep to mind
been discovered to be xl = -.28, x2 = .77, x3 = - .69, whenone is examiningresponsesurfaces.One of theseis
x4 = .32, and x5 = 1.05. The coordinateforx5 is the the adequacyof the fittedmodel,or lack of fit.This is
largestinmagnitude! Thisinformation alonewouldleadone generallydone in theregressionstageof analysis,before
to believethatifone wishesto optimizeyield,thegreatest examinationof any fittedsurface,accordingto standard
changeof theindependent variablesfromthecenterpoint procedures (see Box et al. 1978). The powerof thesetests
mustbe forx5. Such information mayor maynotbe dis- is virtually
unknownforotherthanthehighertermsof the
cemablefromthesignificance oftheindividualtermsinthe includedvariables,however,and lack of significance does
regression model,dependingon thenatureof thetruesur- not prove the null hypothesis true.The behaviorof the
face. It is certainlyfeasiblethata variablethatis never optimalcoordinates as we moveawayfromthecenterpoint
significant at the95% level could be importantin theop- maygive insightintotheadequacyof thefittedmodel.As
timization, or thata variablethatis highlysignificant in previously mentioned, theoptimalridgecoordinators should
interaction termsmay become relativelyirrelevant when rapidlyachieve"optimal"levelsand remainrelatively sta-
othervariableshoveratcertainlevels.Theauthor'spersonal ble as R increases.If theexperimental regionis on a rising
experiencesuggeststhatthisplotoftenrevealslack of fit ridge,some coordinateswill be monotonically increasing
byconventional
thatis notdetectedas significant tests.Note or decreasing.In anycase, the "ridgetrace"of thecoor-
also thatthisplotprovidesthepathof steepestascent(de- dinatesshouldbe reasonablystable.Erraticbehavioris an
scent)fromtheoriginto the overalloptimum,or if none indicationthatthe modelmay not adequatelyfitthetrue
existswithinthedomain,to theoptimum withinthedesign surface.In thepresentexample,theerraticbehaviorseems
space. It is usefulto have theexactpathmappedout ex- to indicateinsignificance ofx5.
plicitlyif operatorsare reluctantto makedrasticchanges Anotherconcern(whichis oftenoverlooked)is thesta-
based on statistical This allowsan "evolution-
predictions. bilityof the selected"optimal"point.By usingcontour
ary" attainment of the optimumbased on one roundof plotsof a two-factor surface,one can easilysee howdrasti-
experimentation. callythesurfacedropsoffwhensmalldepartures fromthe
The plotof R versusA in Figure5 revealstherole of K selectedpointoccur. In an industrial setting,operatingat
in ridgeanalysis.This plot is oftenquite usefulfroma exactlevelsconsistently is impossible;hencea stablepoint
practicalpointof view because it revealsthe rangeof A forsettingtheindependent variablesis desired.Withhigher-
values neededto substitute intoequation(3) to plota par- dimensional surfaces, holdingall buttwovariablesconstant

August1985, Vol. 39, No. 3

TheAmericanStatistician, 189

This content downloaded from 128.235.251.160 on Thu, 05 Mar 2015 08:55:41 UTC
All use subject to JSTOR Terms and Conditions
 4

7- 1' /jIV
C,/ $1 l

-d -I3 -2 0
LAMBDA

Figure 5. Lambda Versus R. Thisreveals the range of lambda values needed to examine a particularridge fora particularrange of R.
The verticalasymptotesare the eigenvalues of B. The horizontalline shows the range of experimentation.

gives an unrealistic pictureof how theresponsemaydrop the particularsituation,ridgeanalysisis well suitedfor

offinan industrial environment, as itallowsdepartures from
the optimalpointonly along 2-dimensional planes. The
effectof all independent variableschangingsimultaneously
is completelymissed. This determination can be accom-
plishedwithridgeanalysis,however.Once a pointhasbeen
selected,the variablescan be restandardized so thatthe
selectedpointis nowthecenterpoint.Theresponsefunction
is mathematically identical,but is now givenin termsof
therescaledvariables.The ridgeanalysiscan thenbe re-
peated,concentrating on theminimum ridge(fora surface
to be maximized).This will clearlyshowhow rapidlythe
responsecan dropoffif the independent variablesdepart
fromtheselectedpoint.The exactpathofthesharpest drop-
offis also givenexplicitly. Ifourselectedpointis an overall
optimum, theridgeswill notbe uniquelydefinedwiththis
restandardization, butcanonicalanalysiswill give thesta-
bilityof thesurfacearoundtheoverallstationary point.
It shouldbe notedthatthechoiceofdesignmayinfluence
thecredibility of theridgeanalysisresults.A rotatable de-
sign is particularly useful,as the predictedresponsewill
haveconstant varianceon thex'x = R2 hyperspheres. Even
a rotatabledesigndoes notguarantee a desirabledistribution
ofpredicted responsevarianceovertheentiredesignregion,
however.Whendealingwithdesignsthatcontaindrastic
differences invarianceofpredicted response,thestatistician
shouldincorporate thisin theanalysisof thesurface.For a
rotatabledesign,thiscouldbe donewitha plotofpredicted
responsevarianceversusR.
The questionofhowtohandlemultiple responsesis often
ofgreatimportance toindustrialstatisticians.
Unfortunately,
this is basicallyan unsolvedproblem,unless additional
concessionsare made. One popularstrategy is to createa
single"summary"responsemade up of some function of
theoriginalresponses,suchas a weightedaverage.This is
thenregressedby usingthe independent variables,which
givesone surfaceto optimize.If thisprocedure is validfor
optimizationof thissummary response.Khuriand Conlon
(1981) gave a moreelaborateco-optimization procedure
based on thisgeneralidea, and Myersand Carter(1973)
discussedtheapplicationof ridgeanalysisto dual response
systems.
6. THE CONNECTION TO RIDGE REGRESSION
At about the same timethathe was developingridge
analysis,anotherproblemwas perplexing Hoerl.This was
thefrequent occurrence of nonsensicalestimatesfromleast
squaresmultipleregression.In an obscurearticle(Hoerl
1962),henotedthattheresidualsumofsquaresinregression
couldbe written as a quadraticfunctionofthecoefficients.
Ridgeanalysiscouldtherefore be usedto calculateandplot
thecoefficientsas one movedalongtheminimum ridgeof
theresidualsumofsquaresfromtheoverallminimum (least
squares)to somemorestablesolutioncloserto theorigin.
The unsolvedproblemofhowfartoremovethecoefficients
fromtheminimum (howto choosek) prevented anyfurther
publicationof this idea untilthe assistanceof Kennard,
whichled to thepublication of the"first"paperson ridge
regressionin 1970 (Hoerland Kennard1970a,b).
The following describesthederivationofridgeregression
by ridgeanalysis.
The residualsum of squaresin multipleregression can
be writtenas
(Y - Y)'(Y - Y) = Y'Y - 2 B'X'Y + pf'(X'X)/3,
whereY = X,3. This is in thesame formas (2) forridge
analysis(multipliedby two) with13as ourx coordinates,
-X'Y as ourb vector,and (X'X) as our "B" matrix.We
aretherefore
trying to minimizetheresidualsumof squares
as a function to note
It is interesting
of our coefficients.
thatin somecases theremaybe a minimum secondary ridge
at almostthe same level as theoverallminimum.In this

190 TheAmericanStatistician,
August1985, Vol. 39, No. 3
This content downloaded from 128.235.251.160 on Thu, 05 Mar 2015 08:55:41 UTC
All use subject to JSTOR Terms and Conditions
case, thereare two separateregionsin thespace in which have appearedin thelit-
of ridgeregression
interpretations
to nearlyminimizeresidualsum of squares. If (X'X) is erature,thisis thephilosophyby whichit was originally
positivedefinite,all eigenvaluesof our "B" matrixare developedand interpreted.
positive,implying thatan overallminimum existsat (X'X)-1
X'Y. We can plottheminimum ridgeof theresidualsum
7. APPLICATIONS AND EXTENSIONS
of squaresfromtheorigin(R = 0) to theoverallminimum
byusinga lambdavaluesmallerthanthesmallesteigenvalue As previously mentioned, thelackof popularity ofridge
of (X'X) in (3), giving(X'X - XI)-1 X'Y. (Note thatthe analysisin thestatistical literaturehas notprevented prac-
Lagrangianmultiplier is generallyk = -X in the ridge tical-minded engineers fromemploying it. Ofthenumerous
regressionliterature.) Lambda less than0 (i.e., k > 0) applicationspapersappearingin theengineering literature
resultsin a solutioncloser to the originthanthe overall (theauthorknowsof morethan25), manyhavecomefrom
minimum. outsidetheUnitedStates,including EasternEurope.Erhardt
We can also plotthebehaviorof thecoordinates as we et al. (1978, 1980) ofEast Germany discussedapplications
move towardthe minimum, in a "ridgetrace."
resulting dealingwithlowmethoxyl pectingels.A detaileddiscussion
Note thatwithridgeregression one plots,3 versusk (i.e., of thetechnique,alongwithapplications, appearedin Po-
- A), rather than,BversusR. Thishas theeffectof shifting land(Jaworski andSzelejewska1978).Applications dealing
the originto the overallminimum(A = 0) and moving withthcdealkylation ofxyleneisomersweregivenbySarma
backwardtowardR = 0. Thus Figure6 showstheridge andRavindram (1975) fromIndia.Mohammed et al. (1979)
coordinates plotfroma ridgeregression perspectiveas the of West Germanydiscussedthe use of ridgeanalysisto
"ridgetrace,"beginning at theoverallminimum (A = 0) optimizeFischer-Tropsch synthesis of olefins.
and movingon theA (i.e., - k) scale, rather thanon theR Ridge analysiswas recentlyappliedto theproblemof
scale. Notethatk is negativeherebecausewe are plotting estimating particlesize distributionparameters in smallan-
themaximumratherthantheminimum ridge.In thiscase, gle neutron scattering(Faticaet al. 1985). It was used as a
xl-x5 play the role of regressioncoefficients ratherthan sequentialsearchprocedure tominimizea multivariate error
design-variable levels as in ridgeanalysis.Again we see function thatcouldnotbe written in closedformbutcould
thatthecoordinates forxl-x4 are stable,butthoseforx5 be approximated witha quadratic function oftheparameters.
are not. In ridgeregressionwe are therefore makingthe This applicationand Hoerl's(1964) discussionof applying
interpretation relativeto a more important pointforthis ridgeanalysisto thesolutionof simultaneous linearequa-
application, theleastsquaressolution.Withtheridgetrace tionssuggestthatthetechniquehas potentialas a general
we are againconcernedwiththestability of the"optimal" numericalanalysisprocedure.It can be used foroptimi-
coordinates(coefficients). Whena coefficient dropserrat- zationandexamination ofthestability of "exact" solutions.
icallytoward0 as k increasesfrom0, thisis analogousto Althoughridgeanalysismay not have receivedthe at-
thecoordinate foran independent variablehovering at0 and tentionitdeservesin thestatistical literature,somenotable
thenincreasing wildlyin magnitude justbeforetheoverall papershave been published.As theoriginalpaper(Hoerl
optimum is reachedinridgeanalysis.In bothcases we view 1959) was publishedin an engineering journal,no proofs
the coefficient (coordinates)withsuspicionand consider of the properties discussedwere given.These were soon
settingit at some less drasticlevel. Althoughmanyother providedin theliterature in a rigorousfashionby Draper

LUi

H0 . 5 0

o.52 \\ -X4

___ X5
00

I
r
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ xi
X

E -0. 501 X3

-1. 00 1 I I I

0.000 00 0 0.4 0 0. 0 C 0.080 0. "00

L1MBDA
Figure 6. Ridge Trace. Thisis a ridge coordinates plot (see Figure3) in the formof a ridge trace froma ridge regression. The originhas
been shiftedto the stationarypoint,and the horizontalaxis is A (- k) ratherthan R. Note thatk is negative here because we are plottinga
maximumridge ratherthan a minimumridge as in ridge regression.

August1985, Vol. 39, No. 3

TheAmericanStatistician, 191

This content downloaded from 128.235.251.160 on Thu, 05 Mar 2015 08:55:41 UTC
All use subject to JSTOR Terms and Conditions
(1963), althoughtheyhad been independently derivedby [ReceivedMarch 1984. RevisedNovember1984.]
Jackson(see Hoerl1959). Hoerl(1964) thencombinedthis REFERENCES
mathematical rigorwiththepracticalinterpretation ofHoerl
Box, G. E. P., Hunter,W. G., and Hunter,J. S. (1978), Statistics for
(1959). Experimenters, New York:JohnWiley.
KhuriandMyers(1979) presented a modification
ofridge Box, G. E. P., and Wilson,K. B. (1951), "On theExperimental Attain-
analysisforuse withnonrotatable designs.Theysuggested mentofOptimum Conditions,"JournaloftheRoyalStatistical Society,
thatit may be moreappropriate to optimizeY forfixed Ser. B, 13, 1-38.
Davies,0. L. (ed.) (1956), DesignandAnalvsisofIndustrial Experiments,
varianceof prediction ratherthanforfixeddistancefrom
New York:Hafner.
theorigin.Theirpointis certainly welltaken,butthismod- Draper,N. R. (1963), "Ridge Analysisof ResponseSurfaces,"Techno-
ifiedprocedure loses someof theintuitive appealand ease metrics,5, 469-479.
of interpretationthatmakeridgeanalysisdesirablein prac- Erhardt,V., Krause, M., Seppelt,B., and Bock, W. (1978), "Losung
tice. The use of a rotatabledesignwill clearlysatisfyboth Von Lebensmittelchemischen Und -Technologischen Aufgaben MitHilfe
Der Statistischen Versuchsplanung," Lebensmittelindustrie,25, 151-
butstandard
criteria, ridgeanalysisstillmakeslogicalsense 154.
withdesignsthatare notexactlyrotatable.Onlywhenthe (1980), "Optimierung Von Zwei Und MehrZielgrossenBei Der
varianceof predictionvariesdrasticallyforfixedR will Anwendung Statistischer
Versuchsplane ZurLosungLebensmittelchem-
nonrotatabilitybe a majorconcern. ischerUnd-Technologischer Aufgaben," 27, 107-
Lebensmittelindustrie,
Myersand Carter(1973) discusseda proceduresimilar 110.
Fatica,M. G., Gelman,R. A., Wai, M. P., Hoerl,R. W., and Wignall,
to ridgeanalysisforoptimizing dual responsesystems.Es-
G. D. (1985), "NeutronScattering FromLatices Preparedby Seeded
thex'x = R2 constraint
sentially, is replacedwitha quadratic EmulsionPolymerization," manuscript in preparation.
constraint on a secondaryresponse.They also discussed Hoerl,A. E. (1959), "OptimumSolutionof ManyVariablesEquations,"
combining theridgeanalysisconstraint withthesecondary ChemicalEngineering Progress,55, 69-78.
responseconstraint. This resultsin a ridgeanalysissubject (1962), "ApplicationofRidgeAnalysisto Regression Problems,"
ChemicalEngineering Progress,58, 54-59.
to an additionalquadraticconstraint.
(1964), "Ridge Analysis,"ChemicalEngineering ProgressSym-
posiumSeries,60, 67-77.
Hoerl,A. E., and Kennard,R. W. (1970a), "Ridge Regression:Biased
8. SUMMARY Estimation forNon-Orthogonal Problems,"Technometrics, 12, 55-67.
(1970b), "Ridge Regression:Applicationsto Non-Orthogonal
Duringthe past 25 years,ridgeanalysishas received Problems,"Technometrics, 12, 69-82.
insufficient in thestatistical
attention literature.A popular Jaworski, A., and Szelejewska,I. (1978), "ZastosowanieMetodyHoerla
textby Myers(1976), whichcontainsa detaileddevelop- Do OptymalizacjiProcesuChemicznegoNa PrzykladzieSyntezyTe-
trahydrofuranu Z Butandiolu-1,4," Przemysl Chemiczny, 57, 564-567.
ment,is one of theonlysourcesto mentionit whendis-
Khuri,A. I., and Conlon, M. (1981), "SimultaneousOptimization of
cussingresponse surface Thissituation
analysis. is unfortunate, MultipleResponsesRepresented byPolynomial Regression Functions,"
as ridgeanalysisdisplaystheeffectsof all factorssimul- Technometrics, 23, 363-375.
taneously,findssecondary (local) optima,andinterprets the Khuri,A. I., andMyers,R. H. (1979), "ModifiedRidgeAnalysis,"Tech-
surfacerelativeto theoriginalcenterpoint.The popularity nometrics, 21, 467-473.
Mohammed,M. S., Schmidt,B., Schneidt,D., and Ralek, M. (1979),
and controversialnatureof ridgeregression, as well as the
"Optimierung Der Fischer-Tropsch-Flussig-Phasen-Synthese in Rich-
generallack of emphasison analysisin responsesurface tungDer MaximalenSelektivitat An C2 Bis C4-Olefinen,"Chemie
methodology, are majorcauses of thisoversight. IngenieurTechnik,51, 739-741.
Although no majorstatistical
computer packagesperform Myers,R. H. (1976), ResponseSurfaceMethodology, Blacksburg,VA:
ridgeanalysis,theauthorwillgladlysupplyinterested par- Author,VirginiaPolytechnic and StateUniversity.
Institute
Myers,R. H., and Carter,W. H. (1973), "ResponseSurfaceTechniques
ties witheithera Minitabmacroor FORTRAN 77 source
forDual ResponseSystems,"Technometrics, 15, 301-317.
code. The FORTRAN code analyzesthesurfaceandwrites Sarma,G. S., and Ravindram, M. (1975), "Studiesin Dealkylation-A
relevantdata to separatefilesthatcan be accessed in a Statistical
AnalysisofProcessVariables,"JournaloftheIndianInstitute
graphicspackageforplotting. ofScience,58, 67-83.

192 TheAmericanStatistician,
August1985, Vol. 39, No. 3
This content downloaded from 128.235.251.160 on Thu, 05 Mar 2015 08:55:41 UTC
All use subject to JSTOR Terms and Conditions