Secdocument 6089

Introduction to Linear Regression
Analysis (Wiley Series in Probability

and Statistics) 6th Edition Montgomery
Visit to download the full and correct content document:
https://ebookmass.com/product/introduction-to-linear-regression-analysis-wiley-series
-in-probability-and-statistics-6th-edition-montgomery/
MONTGOMERY
A comprehensive and current introduction PECK
VINING
to the fundamentals of regression analysis
Introduction to Linear Regression Analysis, 6th Edition is the most comprehensive,
fulsome, and current examination of the foundations of linear regression analysis. Fully
SIXTH EDITION
LINEAR REGRESSION ANALYSIS

updated in this new sixth edition, the distinguished authors have included new material
on generalized regression techniques and new examples to help the reader understand
retain the concepts taught in the book.
The new edition focuses on four key areas of improvement over the fifth edition:
INTRODUCTION TO
LINEAR
• New exercises and data sets
INTRODUCTION TO
• New material on generalized regression techniques
• The inclusion of JMP software in key areas
REGRESSION
• Carefully condensing the text where possible
Introduction to Linear Regression Analysis skillfully blends theory and application

in both the conventional and less common uses of regression analysis in today’s
ANALYSIS
cutting-edge scientific research. The text equips readers to understand the basic
principles needed to apply regression model-building techniques in various fields of
study, including engineering, management, and the health sciences.
DOUGLAS C. MONTGOMERY, PHD, is Regents Professor of Industrial Engineering

and Statistics at Arizona State University. Dr. Montgomery is the co-author of several
Wiley books including Introduction to Linear Regression Analysis, 5th Edition.
ELIZABETH A. PECK, PHD, is Logistics Modeling Specialist at the Coca-Cola

Company in Atlanta, Georgia.
G. GEOFFREY VINING, PHD, is Professor in the Department of Statistics at Virginia

Polytechnic and State University. Dr. Peck is co-author of Introduction to Linear
Regression Analysis, 5th Edition.
A Companion website with additional resources is available at

www.wiley.com/go/montgomery/introlinearregression6e SIXTH
EDITION
DOUGLAS C. MONTGOMERY | ELIZABETH A. PECK
Cover Design: Wiley
Cover Images: Abstract marbled background, blue marbling wavy lines G. GEOFFREY VINING
© oxygen/Getty Images, Linear Regression analysis graph Courtesy of
Douglas C. Montgomery
www.wiley.com
INTRODUCTION TO LINEAR
REGRESSION ANALYSIS
ffirs.indd 1 01/30/2021 3.47.08 PM

WILEY SERIES IN PROBABILITY AND STATISTICS
Established by WALTER A. SHEWHART and SAMUEL S. WILKS
Editors: David J. Balding, Noel A. C. Cressie, Garrett M. Fitzmaurice,

Harvey Goldstein, Iain M. Johnstone, Geert Molenberghs, David W. Scott,
Adrian F. M. Smith, Ruey S. Tsay, Sanford Weisberg
Editors Emeriti: Vic Barnett, J. Stuart Hunter, Joseph B. Kadane, Jozef L. Teugels
A complete list of the titles in this series appears at the end of this volume.
ffirs.indd 2 01/30/2021 3.47.08 PM

INTRODUCTION TO LINEAR
REGRESSION ANALYSIS
Sixth Edition
DOUGLAS C. MONTGOMERY
Arizona State University
School of Computing, Informatics, and Decision Systems Engineering
Tempe, AZ
ELIZABETH A. PECK
The Coca-Cola Company (retired)
Atlanta, GA
G. GEOFFREY VINING
Virginia Tech
Department of Statistics
Blacksburg, VA
ffirs.indd 3 01/30/2021 3.47.08 PM

This sixth edition first published 2021
© 2021 John Wiley & Sons, Inc.
Edition History
John Wiley and Sons, Inc. (5e, 2012)
All rights reserved. No part of this publication may be reproduced, stored in a retrieval system, or trans-
mitted, in any form or by any means, electronic, mechanical, photocopying, recording or otherwise, except
as permitted by law. Advice on how to obtain permission to reuse material from this title is available at
http://www.wiley.com/go/permissions.
The right of Douglas C. Montgomery, Elizabeth A. Peck, and G. Geoffrey Vining to be identified as the
authors of this work has been asserted in accordance with law.
Registered Office(s)
John Wiley & Sons, Inc., 111 River Street, Hoboken, NJ 07030, USA
Editorial Office
111 River Street, Hoboken, NJ 07030, USA
For details of our global editorial offices, customer services, and more information about Wiley products
visit us at www.wiley.com.
Wiley also publishes its books in a variety of electronic formats and by print-on-demand. Some content
that appears in standard print versions of this book may not be available in other formats.
Limit of Liability/Disclaimer of Warranty: While the publisher and authors have used their best efforts
in preparing this work, they make no representations or warranties with respect to the accuracy or com-
pleteness of the contents of this work and specifically disclaim all warranties, including without limitation
any implied warranties of merchantability or fitness for a particular purpose. No warranty may be created
or extended by sales representatives, written sales materials or promotional statements for this work.
The fact that an organization, website, or product is referred to in this work as a citation and/or potential
source of further information does not mean that the publisher and authors endorse the information or
services the organization, website, or product may provide or recommendations it may make. This work
is sold with the understanding that the publisher is not engaged in rendering professional services. The
advice and strategies contained herein may not be suitable for your situation. You should consult with
a specialist where appropriate. Further, readers should be aware that websites listed in this work may
have changed or disappeared between when this work was written and when it is read. Neither the
publisher nor authors shall be liable for any loss of profit or any other commercial damages, including
but not limited to special, incidental, consequential, or other damages.
Library of Congress Cataloging-in-Publication Data

Names: Montgomery, Douglas C., author. | Peck, Elizabeth A., 1953– author.
| Vining, G. Geoffrey, 1954– author.
Title: Introduction to linear regression analysis / Douglas C. Montgomery,
Elizabeth A. Peck, G. Geoffrey Vining.
Description: Fifth edition. | Hoboken, New Jersey : Wiley, [2020] | Series:
Wiley series in probability and statistics | Includes bibliographical
references and index.
Identifiers: LCCN 2020034055 (print) | LCCN 2020034056 (ebook) | ISBN
9781119578727 (hardback) | ISBN 9781119578741 (adobe pdf) | ISBN
9781119578758 (epub)
Subjects: LCSH: Regression analysis.
Classification: LCC QA278.2 .M65 2020 (print) | LCC QA278.2 (ebook) | DDC
519.5/36–dc23
LC record available at https://lccn.loc.gov/2020034055
LC ebook record available at https://lccn.loc.gov/2020034056
Cover Design: Wiley

Cover Images: Abstract marbled background, blue marbling wavy lines © oxygen/Getty Images,
Linear Regression analysis graph Courtesy of Douglas C. Montgomery
Set in 10/12pt TimesTenRoman by SPi Global, Pondicherry, India
10 9 8 7 6 5 4 3 2 1
ffirs.indd 4 01/30/2021 3.47.08 PM

CONTENTS
PREFACE xiii
ABOUT THE COMPANION WEBSITE xvi
1. INTRODUCTION 1
1.1 Regression and Model Building / 1
1.2 Data Collection / 5
1.3 Uses of Regression / 9
1.4 Role of the Computer / 10
2. SIMPLE LINEAR REGRESSION 12

2.1 Simple Linear Regression Model / 12
2.2 Least-Squares Estimation of the Parameters / 13
2.2.1 Estimation of β0 and β1 / 13
2.2.2 Properties of the Least-Squares Estimators
and the Fitted Regression Model / 18
2.2.3 Estimation of σ2 / 20
2.2.4 Alternate Form of the Model / 22
2.3 Hypothesis Testing on the Slope and Intercept / 22
2.3.1 Use of t Tests / 22
2.3.2 Testing Significance of Regression / 24
2.3.3 Analysis of Variance / 25
2.4 Interval Estimation in Simple Linear Regression / 29
2.4.1 Confidence Intervals on β0, β1, and σ 2 / 29
2.4.2 Interval Estimation of the Mean Response / 30
2.5 Prediction of New Observations / 33
v
ftoc.indd 5 01/30/2021 3.47.44 PM

vi CONTENTS
2.6 Coefficient of Determination / 35

2.7 A Service Industry Application of Regression / 37
2.8 Does Pitching Win Baseball Games? / 39
2.9 Using SAS® and R for Simple Linear Regression / 41
2.10 Some Considerations in the Use of Regression / 44
2.11 Regression Through the Origin / 46
2.12 Estimation by Maximum Likelihood / 52
2.13 Case Where the Regressor x is Random / 53
2.13.1 x and y Jointly Distributed / 54
2.13.2 x and y Jointly Normally Distributed: Correlation Model / 54
Problems / 59
3. MULTIPLE LINEAR REGRESSION 69

3.1 Multiple Regression Models / 69
3.2 Estimation of the Model Parameters / 72
3.2.1 Least-Squares Estimation of the Regression
Coefficients / 72
3.2.2 Geometrical Interpretation of Least Squares / 79
3.2.3 Properties of the Least-Squares Estimators / 81
3.2.4 Estimation of σ2 / 82
3.2.5 Inadequacy of Scatter Diagrams in Multiple Regression / 84
3.2.6 Maximum-Likelihood Estimation / 85
3.3 Hypothesis Testing in Multiple Linear Regression / 86
3.3.1 Test for Significance of Regression / 86
3.3.2 Tests on Individual Regression Coefficients
and Subsets of Coefficients / 90
3.3.3 Special Case of Orthogonal Columns in X / 95
3.3.4 Testing the General Linear Hypothesis / 97
3.4 Confidence Intervals in Multiple Regression / 99
3.4.1 Confidence Intervals on the Regression Coefficients / 100
3.4.2 CI Estimation of the Mean Response / 101
3.4.3 Simultaneous Confidence Intervals on Regression
Coefficients / 102
3.5 Prediction of New Observations / 106
3.6 A Multiple Regression Model for the Patient Satisfaction Data / 106
3.7 Does Pitching and Defense Win Baseball Games? / 108
3.8 Using SAS and R for Basic Multiple Linear Regression / 110
3.9 Hidden Extrapolation in Multiple Regression / 111
3.10 Standardized Regression Coefficients / 115
3.11 Multicollinearity / 121
3.12 Why Do Regression Coefficients Have the Wrong Sign? / 123
Problems / 125
ftoc.indd 6 01/30/2021 3.47.44 PM

CONTENTS vii
4. MODEL ADEQUACY CHECKING 134

4.1 Introduction / 134
4.2 Residual Analysis / 135
4.2.1 Definition of Residuals / 135
4.2.2 Methods for Scaling Residuals / 135
4.2.3 Residual Plots / 141
4.2.4 Partial Regression and Partial Residual Plots / 148
4.2.5 Using Minitab®, SAS, and R for Residual Analysis / 151
4.2.6 Other Residual Plotting and Analysis Methods / 154
4.3 PRESS Statistic / 156
4.4 Detection and Treatment of Outliers / 157
4.5 Lack of Fit of the Regression Model / 161
4.5.1 A Formal Test for Lack of Fit / 161
4.5.2 Estimation of Pure Error from Near Neighbors / 165
Problems / 170
5. TRANSFORMATIONS AND WEIGHTING

TO CORRECT MODEL INADEQUACIES 177
5.2 Variance-Stabilizing Transformations / 178
5.3 Transformations to Linearize the Model / 182
5.4 Analytical Methods for Selecting a Transformation / 188
5.4.1 Transformations on y: The Box–Cox Method / 188
5.4.2 Transformations on the Regressor Variables / 190
5.5 Generalized and Weighted Least Squares / 194
5.5.1 Generalized Least Squares / 194
5.5.2 Weighted Least Squares / 196
5.5.3 Some Practical Issues / 197
5.6 Regression Models with Random Effects / 200
5.6.1 Subsampling / 200
5.6.2 The General Situation for a Regression Model
with a Single Random Effect / 204
5.6.3 The Importance of the Mixed Model in Regression / 208
Problems / 208
6. DIAGNOSTICS FOR LEVERAGE AND INFLUENCE 217

6.1 Importance of Detecting Influential Observations / 217
6.2 Leverage / 218
6.3 Measures of Influence: Cook’s D / 221
6.4 Measures of Influence: DFFITS and DFBETAS / 223
6.5 A Measure of Model Performance / 225
ftoc.indd 7 01/30/2021 3.47.45 PM

viii CONTENTS
6.6 Detecting Groups of Influential Observations / 226

6.7 Treatment of Influential Observations / 226
Problems / 227
7. POLYNOMIAL REGRESSION MODELS 230

7.2 Polynomial Models in One Variable / 230
7.2.1 Basic Principles / 230
7.2.2 Piecewise Polynomial Fitting (Splines) / 236
7.2.3 Polynomial and Trigonometric Terms / 242
7.3 Nonparametric Regression / 243
7.3.1 Kernel Regression / 244
7.3.2 Locally Weighted Regression (Loess) / 244
7.3.3 Final Cautions / 249
7.4 Polynomial Models in Two or More Variables / 249
7.5 Orthogonal Polynomials / 255
Problems / 261
8. INDICATOR VARIABLES 268
8.1 General Concept of Indicator Variables / 268
8.2 Comments on the Use of Indicator Variables / 281
8.2.1 Indicator Variables versus Regression on Allocated
Codes / 281
8.2.2 Indicator Variables as a Substitute for a Quantitative
Regressor / 282
8.3 Regression Approach to Analysis of Variance / 283
Problems / 288
9. MULTICOLLINEARITY 293
9.2 Sources of Multicollinearity / 294
9.3 Effects of Multicollinearity / 296
9.4 Multicollinearity Diagnostics / 300
9.4.1 Examination of the Correlation Matrix / 300
9.4.2 Variance Inflation Factors / 304
9.4.3 Eigensystem Analysis of XʹX / 305
9.4.4 Other Diagnostics / 310
9.4.5 SAS and R Code for Generating Multicollinearity
Diagnostics / 311
9.5 Methods for Dealing with Multicollinearity / 311
9.5.1 Collecting Additional Data / 311
9.5.2 Model Respecification / 312
9.5.3 Ridge Regression / 312
ftoc.indd 8 01/30/2021 3.47.45 PM

CONTENTS ix
9.5.4 Principal-Component Regression / 329

9.5.5 Comparison and Evaluation of Biased Estimators / 334
9.6 Using SAS to Perform Ridge and Principal-Component
Regression / 336
Problems / 338
10. VARIABLE SELECTION AND MODEL BUILDING 342

10.1.1 Model-Building Problem / 342
10.1.2 Consequences of Model Misspecification / 344
10.1.3 Criteria for Evaluating Subset Regression Models / 347
10.2 Computational Techniques for Variable Selection / 353
10.2.1 All Possible Regressions / 353
10.2.2 Stepwise Regression Methods / 359
10.3 Strategy for Variable Selection and Model Building / 367
10.4 Case Study: Gorman and Toman Asphalt Data Using SAS / 370
Problems / 383
11. VALIDATION OF REGRESSION MODELS 388

11.2 Validation Techniques / 389
11.2.1 Analysis of Model Coefficients and Predicted Values / 389
11.2.2 Collecting Fresh Data—Confirmation Runs / 391
11.2.3 Data Splitting / 393
11.3 Data from Planned Experiments / 401
Problems / 402
12. INTRODUCTION TO NONLINEAR REGRESSION 405

12.1 Linear and Nonlinear Regression Models / 405
12.1.1 Linear Regression Models / 405
12.1.2 Nonlinear Regression Models / 406
12.2 Origins of Nonlinear Models / 407
12.3 Nonlinear Least Squares / 411
12.4 Transformation to a Linear Model / 413
12.5 Parameter Estimation in a Nonlinear System / 416
12.5.1 Linearization / 416
12.5.2 Other Parameter Estimation Methods / 423
12.5.3 Starting Values / 424
12.6 Statistical Inference in Nonlinear Regression / 425
12.7 Examples of Nonlinear Regression Models / 427
12.8 Using SAS and R / 428
Problems / 432
ftoc.indd 9 01/30/2021 3.47.45 PM

x CONTENTS
13. GENERALIZED LINEAR MODELS 440

13.2 Logistic Regression Models / 441
13.2.1 Models with a Binary Response Variable / 441
13.2.2 Estimating the Parameters in a Logistic
Regression Model / 442
13.2.3 Interpretation of the Parameters in
a Logistic Regression Model / 447
13.2.4 Statistical Inference on Model
Parameters / 449
13.2.5 Diagnostic Checking in Logistic
Regression / 459
13.2.6 Other Models for Binary
Response Data / 461
13.2.7 More Than Two Categorical Outcomes / 461
13.3 Poisson Regression / 463
13.4 The Generalized Linear Model / 469
13.4.1 Link Functions and Linear Predictors / 470
13.4.2 Parameter Estimation and Inference
in the GLM / 471
13.4.3 Prediction and Estimation with the GLM / 473
13.4.4 Residual Analysis in the GLM / 475
13.4.5 Using R to Perform GLM Analysis / 477
13.4.6 Overdispersion / 480
Problems / 481
14. REGRESSION ANALYSIS OF TIME SERIES DATA 495

14.1 Introduction to Regression Models for
Time Series Data / 495
14.2 Detecting Autocorrelation: The Durbin–Watson
Test / 496
14.3 Estimating the Parameters in Time Series Regression
Models / 501
Problems / 517
15. OTHER TOPICS IN THE USE OF REGRESSION ANALYSIS 521

15.1 Robust Regression / 521
15.1.1 Need for Robust Regression / 521
15.1.2 M-Estimators / 524
15.1.3 Properties of Robust Estimators / 531
ftoc.indd 10 01/30/2021 3.47.45 PM

CONTENTS xi
15.2 Effect of Measurement Errors in the Regressors / 532

15.2.1 Simple Linear Regression / 532
15.2.2 The Berkson Model / 534
15.3 Inverse Estimation—The Calibration Problem / 534
15.4 Bootstrapping in Regression / 538
15.4.1 Bootstrap Sampling in Regression / 539
15.4.2 Bootstrap Confidence Intervals / 540
15.5 Classification and Regression Trees (CART) / 545
15.6 Neural Networks / 547
15.7 Designed Experiments for Regression / 549
Problems / 557
APPENDIX A. STATISTICAL TABLES 561
APPENDIX B. DATA SETS FOR EXERCISES 573
APPENDIX C. SUPPLEMENTAL TECHNICAL MATERIAL 602

C.1 Background on Basic Test Statistics / 602
C.2 Background from the Theory of Linear Models / 605
C.3 Important Results on SSR and SSRes / 609
C.4 Gauss-Markov Theorem, Var(ε) = σ2I / 615
C.5 Computational Aspects of Multiple Regression / 617
C.6 Result on the Inverse of a Matrix / 618
C.7 Development of the PRESS Statistic / 619
2
C.8 Development of S(i) / 621
C.9 Outlier Test Based on R-Student / 622
C.10 Independence of Residuals and Fitted Values / 624
C.11 Gauss–Markov Theorem, Var(ε) = V / 625
C.12 Bias in MSRes When the Model Is Underspecified / 627
C.13 Computation of Influence Diagnostics / 628
C.14 Generalized Linear Models / 629
APPENDIX D. INTRODUCTION TO SAS 641

D.1 Basic Data Entry / 642
D.2 Creating Permanent SAS Data Sets / 646
D.3 Importing Data from an EXCEL File / 647
D.4 Output Command / 648
D.5 Log File / 648
D.6 Adding Variables to an Existing SAS Data Set / 650
ftoc.indd 11 01/30/2021 3.47.45 PM

xii CONTENTS
APPENDIX E. INTRODUCTION TO R TO PERFORM LINEAR

REGRESSION ANALYSIS 651
E.1 Basic Background on R / 651
E.2 Basic Data Entry / 652
E.3 Brief Comments on Other Functionality in R / 654
E.4 R Commander / 655
REFERENCES 656
INDEX 670
ftoc.indd 12 01/30/2021 3.47.45 PM

PREFACE
Regression analysis is one of the most widely used techniques for analyzing multi-
factor data. Its broad appeal and usefulness result from the conceptually logical
process of using an equation to express the relationship between a variable of inter-
est (the response) and a set of related predictor variables. Regression analysis is also
interesting theoretically because of elegant underlying mathematics and a well-
developed statistical theory. Successful use of regression requires an appreciation of
both the theory and the practical problems that typically arise when the technique
is employed with real-world data.
This book is intended as a text for a basic course in regression analysis. It contains
the standard topics for such courses and many of the newer ones as well. It blends both
theory and application so that the reader will gain an understanding of the basic prin-
ciples necessary to apply regression model-building techniques in a wide variety of
application environments. The book began as an outgrowth of notes for a course in
regression analysis taken by seniors and first-year graduate students in various fields of
engineering, the chemical and physical sciences, statistics, mathematics, and manage-
ment. We have also used the material in many seminars and industrial short courses for
professional audiences. We assume that the reader has taken a first course in statistics
and has familiarity with hypothesis tests and confidence intervals and the normal, t, χ2,
and F distributions. Some knowledge of matrix algebra is also necessary.
The computer plays a significant role in the modern application of regression.
Today even spreadsheet software has the capability to fit regression equations by
least squares. Consequently, we have integrated many aspects of computer usage
into the text, including displays of both tabular and graphical output, and general
discussions of capabilities of some software packages. We use Minitab®, JMP®,
SAS®, and R for various problems and examples in the text. We selected these
packages because they are widely used both in practice and in teaching regression
and they have good regression. Many of the homework problems require software
for their solution. All data sets in the book are available in electronic form from the
xiii
fpref.indd 13 01/30/2021 3.47.33 PM

xiv PREFACE
publisher. The ftp site ftp://ftp.wiley.com/public/sci_tech_med/introduction_linear_

regression hosts the data, problem solutions, PowerPoint files, and other material
related to the book.
CHANGES IN THE SIXTH EDITION
We have made a number of changes in this edition of the book. This includes the
reorganization of text material, new examples, new exercises, and new material on
a variety of topics. Our objective was to make the book more useful as both a text
and a reference and to update our treatment of certain topics.
Chapter 1 is a general introduction to regression modeling and describes some typical
applications of regression. Chapters 2 and 3 provide the standard results for least-
squares model fitting in simple and multiple regression, along with basic inference
procedures (tests of hypotheses, confidence and prediction intervals). Chapter 4 dis-
cusses some introductory aspects of model adequacy checking, including residual analy-
sis and a strong emphasis on residual plots, detection and treatment of outliers, the
PRESS statistic, and testing for lack of fit. Chapter 5 discusses how transformations and
weighted least squares can be used to resolve problems of model inadequacy or to deal
with violations of the basic regression assumptions. Both the Box–Cox and Box–Tidwell
techniques for analytically specifying the form of a transformation are introduced. Influ-
ence diagnostics are presented in Chapter 6, along with an introductory discussion of
how to deal with influential observations. Polynomial regression models and their varia-
tions are discussed in Chapter 7. Topics include the basic procedures for fitting and
inference for polynomials and discussion of centering in polynomials, hierarchy, piece-
wise polynomials, models with both polynomial and trigonometric terms, orthogonal
polynomials, an overview of response surfaces, and an introduction to nonparametric
and smoothing regression techniques. Chapter 8 introduces indicator variables and also
makes the connection between regression and analysis-of-variance models. Chapter 9
focuses on the multicollinearity problem. Included are discussions of the sources of
multicollinearity, its harmful effects, diagnostics, and various remedial measures. We
introduce biased estimation, including ridge regression and some of its variations and
principal-component regression.Variable selection and model-building techniques are
developed in Chapter 10, including stepwise procedures and all-possible-regressions. We
also discuss and illustrate several criteria for the evaluation of subset regression models.
Chapter 11 presents a collection of techniques useful for regression model validation.
The first 11 chapters are the nucleus of the book. Many of the concepts and
examples flow across these chapters. The remaining four chapters cover a variety of
topics that are important to the practitioner of regression, and they can be read
independently. Chapter 12 in introduces nonlinear regression, and Chapter 13 is a
basic treatment of generalized linear models. While these are perhaps not standard
topics for a linear regression textbook, they are so important to students and profes-
sionals in engineering and the sciences that we would have been seriously remiss
without giving an introduction to them. Chapter 14 covers regression models for
time series data. Chapter 15 includes a survey of several important topics, including
robust regression, the effect of measurement errors in the regressors, the inverse
estimation or calibration problem, bootstrapping regression estimates, classification
and regression trees, neural networks, and designed experiments for regression.
In addition to the text material, Appendix C contains brief presentations of some
additional topics of a more technical or theoretical nature. Some of these topics will
fpref.indd 14 01/30/2021 3.47.33 PM

PREFACE xv
be of interest to specialists in regression or to instructors teaching a more advanced

course from the book. Computing plays an important role in many regression
courses. Mintab, JMP, SAS, and R are widely used in regression courses. Outputs
from all of these packages are provided in the text. Appendix D is an introduction
to using SAS for regression problems. Appendix E is an introduction to R.
USING THE BOOK AS A TEXT
Because of the broad scope of topics, this book has great flexibility as a text. For a
first course in regression, we would recommend covering Chapters 1 through 10 in
detail and then selecting topics that are of specific interest to the audience. For
example, one of the authors (D.C.M.) regularly teaches a course in regression to an
engineering audience. Topics for that audience include nonlinear regression (because
mechanistic models that are almost always nonlinear occur often in engineering), a
discussion of neural networks, and regression model validation. Other topics that
we would recommend for consideration are multicollinearity (because the problem
occurs so often) and an introduction to generalized linear models focusing mostly
on logistic regression. G.G.V. has taught a regression course for graduate students
in statistics that makes extensive use of the Appendix C material.
We believe the computer should be directly integrated into the course. In recent
years, we have taken a notebook computer and computer projector to most classes
and illustrated the techniques as they are introduced in the lecture. We have found
that this greatly facilitates student understanding and appreciation of the tech-
niques. We also require that the students use regression software for solving the
homework problems. In most cases, the problems use real data or are based on
real-world settings that represent typical applications of regression.
There is an instructor’s manual that contains solutions to all exercises, electronic
versions of all data sets, and questions/problems that might be suitable for use on
examinations.
ACKNOWLEDGMENTS
We would like to thank all the individuals who provided helpful feedback and
assistance in the preparation of this book. Dr. Scott M. Kowalski, Dr. Ronald G.
Askin, Dr. Mary Sue Younger, Dr. Russell G. Heikes, Dr. John A. Cornell, Dr. André
I. Khuri, Dr. George C. Runger, Dr. Marie Gaudard, Dr. James W. Wisnowski, Dr.
Ray Hill, and Dr. James R. Simpson made many suggestions that greatly improved
both earlier editions and this fifth edition of the book. We particularly appreciate
the many graduate students and professional practitioners who provided feedback,
often in the form of penetrating questions, that led to rewriting or expansion of
material in the book. We are also indebted to John Wiley & Sons, the American
Statistical Association, and the Biometrika Trustees for permission to use copy-
righted material.
Douglas C. Montgomery
Elizabeth A. Peck
G. Geoffrey Vining
fpref.indd 15 01/30/2021 3.47.33 PM

ABOUT THE COMPANION WEBSITE
This book is accompanied by an instructor companion website and a student

companion website:
www.wiley.com/go/montgomery/introlinearregression6e
The instructor site includes PowerPoint slides to facilitate instructional use of the
book.
The student site includes data sets.
xvi
flast.indd 16 01/30/2021 3.47.21 PM

CHAPTER 1
INTRODUCTION
1.1 REGRESSION AND MODEL BUILDING
Regression analysis is a statistical technique for investigating and modeling the

relationship between variables. Applications of regression are numerous and occur
in almost every field, including engineering, the physical and chemical sciences,
economics, management, life and biological sciences, and the social sciences. Regres-
sion analysis is used extensively in data mining and is a basic tool of data science
and analytics. Because of its wide applicability to a range of problems, regression
analysis may be the most widely used statistical technique.
As an example of a problem in which regression analysis may be helpful, suppose
that an industrial engineer employed by a soft drink beverage bottler is analyzing
the product delivery and service operations for vending machines. He suspects that
the time required by a route deliveryman to load and service a machine is related
to the number of cases of product delivered. The engineer visits 25 randomly chosen
retail outlets having vending machines, and the in-outlet delivery time (in minutes)
and the volume of product delivered (in cases) are observed for each. The 25 obser-
vations are plotted in Figure 1.1a. This graph is called a scatter diagram. This display
clearly suggests a relationship between delivery time and delivery volume; in fact,
the impression is that the data points generally, but not exactly, fall along a straight
line. Figure 1.1b illustrates this straight-line relationship.
If we let y represent delivery time and x represent delivery volume, then the
equation of a straight line relating these two variables is
y 0 1x (1.1)
Introduction to Linear Regression Analysis, Sixth Edition. Douglas C. Montgomery, Elizabeth A. Peck,
and G. Geoffrey Vining.
© 2021 John Wiley & Sons, Inc. Published 2021 by John Wiley & Sons, Inc.
Companion website: www.wiley.com/go/montgomery/introlinearregression6e
c01.indd 1 30-01-2021 14:20:27

2 INTRODUCTION
80 80
70 70
60 60
Delivery time, y
Delivery time, y
50 50
40 40
30 30
20 20
10 10
0 0
0 10 20 30 0 10 20 30
Delivery volume, x Delivery volume, x
(a) (b)
Figure 1.1 (a) Scatter diagram for delivery volume. (b) Straight-line relationship between
delivery time and delivery volume.
where β0 is the intercept and β1 is the slope. Now the data points do not fall
exactly on a straight line, so Eq. (1.1) should be modified to account for this. Let the
difference between the observed value of y and the straight line (β0 + β1x) be an
error ε. It is convenient to think of ε as a statistical error; that is, it is a random vari-
able that accounts for the failure of the model to fit the data exactly. The error may
be made up of the effects of other variables on delivery time, measurement errors,
and so forth. Thus, a more plausible model for the delivery time data is
y 0 1x (1.2)
Equation (1.2) is called a linear regression model. Customarily x is called the inde-
pendent variable and y is called the dependent variable. However, this often causes
confusion with the concept of statistical independence, so we refer to x as the pre-
dictor or regressor variable and y as the response variable. Because Eq. (1.2) involves
only one regressor variable, it is called a simple linear regression model.
To gain some additional insight into the linear regression model, suppose that we
can fix the value of the regressor variable x and observe the corresponding value of
the response y. Now if x is fixed, the random component ε on the right-hand side of
Eq. (1.2) determines the properties of y. Suppose that the mean and variance of ε are
0 and σ2, respectively. Then the mean response at any value of the regressor variable is
E y|x y| x E 0 1x 0 1x
Notice that this is the same relationship that we initially wrote down following inspec-
tion of the scatter diagram in Figure 1.1a. The variance of y given any value of x is
2 2
Var y | x y| x Var 0 1x
Thus, the true regression model μy|x = β0 + β1x is a line of mean values, that is, the
height of the regression line at any value of x is just the expected value of y for that
c01.indd 2 30-01-2021 14:20:32

Regression and Model Building 3
Observed values of y
for a given x
are sampled from Observed value
these distributions of y
N (0, σ2 = 2) μy|x Straight–line

approximation
y
True relationship
N (0, σ2 = 2)
Observed value of y
10 20 30 x x
Figure 1.2 How observations are generated Figure 1.3 Linear regression
in linear regression. approximation of a complex relationship.
x. The slope, β1 can be interpreted as the change in the mean of y for a unit change in
x. Furthermore, the variability of y at a particular value of x is determined by the vari-
ance of the error component of the model, σ2. This implies that there is a distribution
of y values at each x and that the variance of this distribution is the same at each x.
For example, suppose that the true regression model relating delivery time to
delivery volume is μy|x = 3.5 + 2x, and suppose that the variance is σ 2 = 2. Figure 1.2
illustrates this situation. Notice that we have used a normal distribution to describe
the random variation in ε. Since y is the sum of a constant β0 + β1x (the mean) and
a normally distributed random variable, y is a normally distributed random variable.
For example, if x = 10 cases, then delivery time y has a normal distribution with
mean 3.5 + 2(10) = 23.5 minutes and variance 2. The variance σ 2 determines the
amount of variability or noise in the observations y on delivery time. When σ 2 is
small, the observed values of delivery time will fall close to the line, and when σ 2 is
large, the observed values of delivery time may deviate considerably from the line.
In almost all applications of regression, the regression equation is only an approx-
imation to the true functional relationship between the variables of interest. These
functional relationships are often based on physical, chemical, or other engineering
or scientific theory, that is, knowledge of the underlying mechanism. Consequently,
these types of models are often called mechanistic models. For example, the familiar
physics equation momentum = mass × velocity is a mechanistic model.
Regression models, on the other hand, are thought of as empirical models. Figure
1.3 illustrates a situation where the true relationship between y and x is relatively
complex, yet it may be approximated quite well by a linear regression equation.
Sometimes the underlying mechanism is more complex, resulting in the need for a
more complex approximating function, as in Figure 1.4, where a “piecewise linear”
regression function is used to approximate the true relationship between y and x.
Generally regression equations are valid only over the region of the regressor
variables contained in the observed data. For example, consider Figure 1.5. Suppose
that data on y and x were collected in the interval x1 ≤ x ≤ x2. Over this interval the
c01.indd 3 30-01-2021 14:20:33

4 INTRODUCTION
True relationship
y
Piecewise linear
regression
y approximation
x1 x2 x3
x x
Figure 1.4 Piecewise linear Figure 1.5 The danger of extrapolation
approximation of a complex relationship. in regression.
linear regression equation shown in Figure 1.5 is a good approximation of the true
relationship. However, suppose this equation were used to predict values of y for
values of the regressor variable in the region x2 ≤ x ≤ x3. Clearly the linear regres-
sion model is not going to perform well over this range of x because of model error
or equation error.
In general, the response variable y may be related to k regressors, x1, x2, . . . , xk,
so that
y 0 1 x1 2 x2  k xk (1.3)
This is called a multiple linear regression model because more than one regressor
is involved. The adjective linear is employed to indicate that the model is linear in
the parameters β0, β1, . . . , βk, not because y is a linear function of the x’s. We shall
see subsequently that many models in which y is related to the x’s in a nonlinear
fashion can still be treated as linear regression models as long as the equation is
linear in the β’s.
An important objective of regression analysis is to estimate the unknown param-
eters in the regression model. This process is also called fitting the model to the data.
We study several parameter estimation techniques in this book. One of these tech-
mques is the method of least squares (introduced in Chapter 2). For example, the
least-squares fit to the delivery time data is
yˆ 3.321 2.1762 x
where ŷ is the fitted or estimated value of delivery time corresponding to a delivery

volume of x cases. This fitted equation is plotted in Figure 1.1b.
The next phase of a regression analysis is called model adequacy checking, in
which the appropriateness of the model is studied and the quality of the fit ascer-
tained. Through such analyses the usefulness of the regression model may be deter-
mined. The outcome of adequacy checking may indicate either that the model is
reasonable or that the original fit must be modified. Thus, regression analysis is an
iterative procedure, in which data lead to a model and a fit of the model to the data
is produced. The quality of the fit is then investigated, leading either to modification
c01.indd 4 30-01-2021 14:20:38

Data Collection 5
of the model or the fit or to adoption of the model. This process is illustrated several
times in subsequent chapters.
A regression model does not imply a cause-and-effect relationship between the
variables. Even though a strong empirical relationship may exist between two or
more variables, this cannot be considered evidence that the regressor variables and
the response are related in a cause-and-effect manner. To establish causality, the
relationship between the regressors and the response must have a basis outside the
sample data—for example, the relationship may be suggested by theoretical consid-
erations. Regression analysis can aid in confirming a cause-and-effect relationship,
but it cannot be the sole basis of such a claim.
Finally it is important to remember that regression analysis is part of a broader
data-analytic approach to problem solving. That is, the regression equation itself
may not be the primary objective of the study. It is usually more important to gain
insight and understanding concerning the system generating the data.
1.2 DATA COLLECTION
An essential aspect of regression analysis is data collection. Any regression analysis

is only as good as the data on which it is based. Three basic methods for collecting
data are as follows:
• A retrospective study based on historical data

• An observational study
• A designed experiment
A good data collection scheme can ensure a simplified and a generally more appli-
cable model. A poor data collection scheme can result in serious problems for the
analysis and its interpretation. The following example illustrates these three methods.
Example 1.1
Consider the acetone–butyl alcohol distillation column shown in Figure 1.6. The
operating personnel are interested in the concentration of acetone in the distillate
(product) stream. Factors that may influence this are the reboil temperature, the
condensate temperature, and the reflux rate. For this column, operating personnel
maintain and archive the following records:
• The concentration of acetone in a test sample taken every hour from the
product stream
• The reboil temperature controller log, which is a plot of the reboil
temperature
• The condenser temperature controller log
• The nominal reflux rate each hour
The nominal reflux rate is supposed to be constant for this process. Only infre-
quently does production change this rate. We now discuss how the three different
data collection strategies listed above could be applied to this process. ■
c01.indd 5 30-01-2021 14:20:38

6 INTRODUCTION
Condenser
Coolant
50
45 Distillate
Trays Reflux
40
35
30
25
Sidedraw
20
Feed 15
10
Trays
5
Reboiler
Steam
Bottoms
Figure 1.6 Acetone–butyl alcohol distillation column.
Retrospective Study We could pursue a retrospective study that would use either
all or a sample of the historical process data over some period of time to determine
the relationships among the two temperatures and the reflux rate on the acetone
concentration in the product stream. In so doing, we take advantage of previously
collected data and minimize the cost of the study. However, there are several
problems:
1. We really cannot see the effect of reflux on the concentration since we must
assume that it did not vary much over the historical period.
2. The data relating the two temperatures to the acetone concentration do not
correspond directly. Constructing an approximate correspondence usually
requires a great deal of effort.
3. Production controls temperatures as tightly as possible to specific target values
through the use of automatic controllers. Since the two temperatures vary so
little over time, we will have a great deal of difficulty seeing their real impact
on the concentration.
4. Within the narrow ranges that they do vary, the condensate temperature tends
to increase with the reboil temperature. As a result, we will have a great deal
c01.indd 6 30-01-2021 14:20:38

Data Collection 7
of difficulty separating out the individual effects of the two temperatures. This
leads to the problem of collinearity or multicollinearity, which we discuss in
Chapter 9.
Retrospective studies often offer limited amounts of useful information. In

general, their primary disadvantages are as follows:
• Some of the relevant data often are missing.

• The reliability and quality of the data are often highly questionable.
• The nature of the data often may not allow us to address the problem at hand.
• The analyst often tries to use the data in ways they were never intended to be
used.
• Logs, notebooks, and memories may not explain interesting phenomena identi-
fied by the data analysis.
Using historical data always involves the risk that, for whatever reason, some of
the data were not recorded or were lost. Typically, historical data consist of informa-
tion considered critical and of information that is convenient to collect. The conve-
nient information is often collected with great care and accuracy. The essential
information often is not. Consequently, historical data often suffer from transcrip-
tion errors and other problems with data quality. These errors make historical data
prone to outliers, or observations that are very different from the bulk of the data.
A regression analysis is only as reliable as the data on which it is based.
Just because data are convenient to collect does not mean that these data are
particularly useful. Often, data not considered essential for routine process monitor-
ing and not convenient to collect do have a significant impact on the process. His-
torical data cannot provide this information since they were never collected. For
example, the ambient temperature may impact the heat losses from our distillation
column. On cold days, the column loses more heat to the environment than during
very warm days. The production logs for this acetone–butyl alcohol column do not
record the ambient temperature. As a result, historical data do not allow the analyst
to include this factor in the analysis even though it may have some importance.
In some cases, we try to use data that were collected as surrogates for what we
really needed to collect. The resulting analysis is informative only to the extent that
these surrogates really reflect what they represent. For example, the nature of the
inlet mixture of acetone and butyl alcohol can significantly affect the column’s per-
formance. The column was designed for the feed to be a saturated liquid (at the
mixture’s boiling point). The production logs record the feed temperature but do
not record the specific concentrations of acetone and butyl alcohol in the feed
stream. Those concentrations are too hard to obtain on a regular basis. In this case,
inlet temperature is a surrogate for the nature of the inlet mixture. It is perfectly
possible for the feed to be at the correct specific temperature and the inlet feed to
be either a subcooled liquid or a mixture of liquid and vapor.
In some cases, the data collected most casually, and thus with the lowest quality,
the least accuracy, and the least reliability, turn out to be very influential for explain-
ing our response. This influence may be real, or it may be an artifact related to the
inaccuracies in the data. Too many analyses reach invalid conclusions because they
c01.indd 7 30-01-2021 14:20:38

8 INTRODUCTION
lend too much credence to data that were never meant to be used for the strict
purposes of analysis.
Finally, the primary purpose of many analyses is to isolate the root causes under-
lying interesting phenomena. With historical data, these interesting phenomena may
have occurred months or years before. Logs and notebooks often provide no sig-
nificant insights into these root causes, and memories clearly begin to fade over time.
Too often, analyses based on historical data identify interesting phenomena that go
unexplained.
Observational Study We could use an observational study to collect data for this
problem. As the name implies, an observational study simply observes the process
or population. We interact or disturb the process only as much as is required to
obtain relevant data. With proper planning, these studies can ensure accurate, com-
plete, and reliable data. On the other hand, these studies often provide very limited
information about specific relationships among the data.
In this example, we would set up a data collection form that would allow the
production personnel to record the two temperatures and the actual reflux rate at
specified times corresponding to the observed concentration of acetone in the
product stream. The data collection form should provide the ability to add com-
ments in order to record any interesting phenomena that may occur. Such a proce-
dure would ensure accurate and reliable data collection and would take care of
problems 1 and 2 above. This approach also minimizes the chances of observing an
outlier related to some error in the data. Unfortunately, an observational study
cannot address problems 3 and 4. As a result, observational studies can lend them-
selves to problems with collinearity.
Designed Experiment The best data collection strategy for this problem uses a
designed experiment where we would manipulate the two temperatures and the
reflux ratio, which we would call the factors, according to a well-defined strategy,
called the experimental design. This strategy must ensure that we can separate out
the effects on the acetone concentration related to each factor. In the process, we
eliminate any collinearity problems. The specified values of the factors used in the
experiment are called the levels. Typically, we use a small number of levels for each
factor, such as two or three. For the distillation column example, suppose we use a
“high” or +1 and a “low” or −1 level for each of the factors. We thus would use
two levels for each of the three factors. A treatment combination is a specific com-
bination of the levels of each factor. Each time we carry out a treatment combina-
tion is an experimental run or setting. The experimental design or plan consists of
a series of runs.
For the distillation example, a very reasonable experimental strategy uses
every possible treatment combination to form a basic experiment with eight differ-
ent settings for the process. Table 1.1 presents these combinations of high and low
levels. This experimental arrangement is called a factorial design.
Figure 1.7 illustrates that this factorial design forms a cube in terms of these high
and low levels. With each setting of the process conditions, we allow the column to
reach equilibrium, take a sample of the product stream, and determine the acetone
concentration. We then can draw specific inferences about the effect of these factors.
Such an approach allows us to proactively study a population or process.
c01.indd 8 30-01-2021 14:20:38

Uses of Regression 9
TABLE 1.1 Designed Experiment

for the Distillation Column
Reboil Condensate Reflux
+1 te
Temperature Temperature Rate sa
Reflux
en p. +1
Rate
d
n em
−1 −1 −1 Co–1 T
–1
+1 −1 −1
−1 +1 −1 –1 Reboil +1
+1 +1 −1 Temp.
−1 −1 +1
+1 −1 +1 Figure 1.7 The designed experiment for the
−1 +1 +1 distillation column.
+1 +1 +1
1.3 USES OF REGRESSION
Regression models are used for several purposes, including the following:
1. Data description
2. Parameter estimation
3. Prediction and estimation
4. Control
Engineers and scientists frequently use equations to summarize or describe a set of

data. Regression analysis is helpful in developing such equations. For example, we
may collect a considerable amount of delivery time and delivery volume data, and
a regression model would probably be a much more convenient and useful summary
of those data than a table or even a graph.
Sometimes parameter estimation problems can be solved by regression methods.
For example, chemical engineers use the Michaelis–Menten equation y = β1x/
(x + β2) + ε to describe the relationship between the velocity of reaction y and
concentration x. Now in this model, β1 is the asymptotic velocity of the reaction, that
is, the maximum velocity as the concentration gets large. If a sample of observed
values of velocity at different concentrations is available, then the engineer can use
regression analysis to fit this model to the data, producing an estimate of the
maximum velocity. We show how to fit regression models of this type in Chapter 12.
Many applications of regression involve prediction of the response variable. For
example, we may wish to predict delivery time for a specified number of cases of
soft drinks to be delivered. These predictions may be helpful in planning delivery
activities such as routing and scheduling or in evaluating the productivity of delivery
operations. The dangers of extrapolation when using a regression model for predic-
tion because of model or equation error have been discussed previously (see Figure
1.5). However, even when the model form is correct, poor estimates of the model
parameters may still cause poor prediction performance.
Regression models may be used for control purposes. For example, a chemical
engineer could use regression analysis to develop a model relating the tensile
strength of paper to the hardwood concentration in the pulp. This equation could
c01.indd 9 30-01-2021 14:20:39

10 INTRODUCTION
then be used to control the strength to suitable values by varying the level of hard-
wood concentration. When a regression equation is used for control purposes, it is
important that the variables be related in a causal manner. Note that a cause-and-
effect relationship may not be necessary if the equation is to be used only for pre-
diction. In this case it is only necessary that the relationships that existed in the
original data used to build the regression equation are still valid. For example, the
daily electricity consumption during August in Atlanta, Georgia, may be a good
predictor for the maximum daily temperature in August. However, any attempt to
reduce the maximum temperature by curtailing electricity consumption is clearly
doomed to failure.
1.4 ROLE OF THE COMPUTER
Building a regression model is an iterative process. The model-building process is

illustrated in Figure 1.8. It begins by using any theoretical knowledge of the process
that is being studied and available data to specify an initial regression model.
Graphical data displays are often very useful in specifying the initial model. Then
the parameters of the model are estimated, typically by either least squares or
maximum likelihood. These procedures are discussed extensively in the text. Then
model adequacy must be evaluated. This consists of looking for potential misspecifi-
cation of the model form, failure to include important variables, including unneces-
sary variables, or unusual/inappropriate data. If the model is inadequate, then
adjustments must be made and the parameters estimated again. This process may
be repeated several times until an adequate model is obtained. Finally, model valida-
tion should be carried out to ensure that the model will produce results that are
acceptable in the final application.
A good regression computer program is a necessary tool in the model-building
process. However, the routine application of standard regression computer pro-
grams often does not lead to successful results. The computer is not a substitute
for creative thinking about the problem. Regression analysis requires the intelligent
and artful use of the computer. We must learn how to interpret what the computer
is telling us and how to incorporate that information in subsequent models. Gener-
ally, regression computer programs are part of more general statistics software
packages, such as Minitab, SAS, JMP, and R. We discuss and illustrate the use of
No No
Data
Yes Model Yes Model Yes Model

Model Parameter adequacy
specification estimation validation use
checking
Theory
Figure 1.8 Regression model-building process.
c01.indd 10 30-01-2021 14:20:39

Role of the Computer 11
these packages throughout the book. Appendix D contains details of the SAS pro-
cedures typically used in regression modeling along with basic instructions for their
use. Appendix E provides a brief introduction to the R statistical software package.
We present R code for doing analyses throughout the text. Without these skills, it
is virtually impossible to successfully build a regression model.
c01.indd 11 30-01-2021 14:20:39

CHAPTER 2
SIMPLE LINEAR REGRESSION
2.1 SIMPLE LINEAR REGRESSION MODEL
This chapter considers the simple linear regression model, that is, a model with a
single regressor x that has a relationship with a response y that is a straight line. This
simple linear regression model is
y 0 1x (2.1)
where the intercept β0 and the slope β1 are unknown constants and ε is a random
error component. The errors are assumed to have mean zero and unknown variance
σ 2. Additionally we usually assume that the errors are uncorrelated. This means that
the value of one error does not depend on the value of any other error.
It is convenient to view the regressor x as controlled by the data analyst and
measured with negligible error, while the response y is a random variable. That is,
there is a probability distribution for y at each possible value for x. The mean of
this distribution is
E yx 0 1x (2.2a)
and the variance is

2
Var y x Var 0 1x (2.2b)
Introduction to Linear Regression Analysis, Sixth Edition. Douglas C. Montgomery, Elizabeth A. Peck,
and G. Geoffrey Vining.
© 2021 John Wiley & Sons, Inc. Published 2021 by John Wiley & Sons, Inc.
Companion website: www.wiley.com/go/montgomery/introlinearregression6e
12
c02.indd 12 30-01-2021 14:25:29

Least-Squares Estimation of the Parameters 13
Thus, the mean of y is a linear function of x although the variance of y does not
depend on the value of x. Furthermore, because the errors are uncorrelated, the
responses are also uncorrelated.
The parameters β0 and β1 are usually called regression coefficients. These coef-
ficients have a simple and often useful interpretation. The slope β1 is the change in
the mean of the distribution of y produced by a unit change in x. If the range of
data on x includes x = 0, then the intercept β0 is the mean of the distribution of the
response y when x = 0. If the range of x does not include zero, then β0 has no practi-
cal interpretation.
2.2 LEAST-SQUARES ESTIMATION OF THE PARAMETERS
The parameters β0 and β1 are unknown and must be estimated using sample data.
Suppose that we have n pairs of data, say (y1, x1), (y2, x2), . . . , (yn, xn). As noted in
Chapter 1, these data may result either from a controlled experiment designed
specifically to collect the data, from an observational study, or from existing histori-
cal records (a retrospective study).
2.2.1 Estimation of β0 and β1

The method of least squares is used to estimate β0 and β1. That is, we estimate β0
and β1 so that the sum of the squares of the differences between the observations
yi and the straight line is a minimum. From Eq. (2.1) we may write
yi 0 1 xi i, i 1, 2,  , n (2.3)
Equation (2.1) maybe viewed as a population regression model while Eq.

(2.3) is a sample regression model, written in terms of the n pairs of data (yi, xi)
(i = 1, 2, . . . , n). Thus, the least-squares criterion is
n
2
S 0, 1 yi 0 1 xi (2.4)
i 1
The least-squares estimators of β0 and β1, say ˆ0 and ˆ1, must satisfy
n
S ˆ0 ˆ1 xi
2 yi 0
0 ˆ0, ˆ1 i 1
and
n
S ˆ0 ˆ1 xi xi
2 yi 0
1 ˆ0, ˆ1 i 1
c02.indd 13 30-01-2021 14:25:40

14 SIMPLE LINEAR REGRESSION
Simplifying these two equations yields

n n
n ˆ0 ˆ1 xi yi
i 1 i 1
n n n
ˆ0 xi ˆ1 xi2 yi xi (2.5)
i 1 i 1 i 1
Equations (2.5) are called the least-squares normal equations. The solution to the
normal equations is
ˆ0 y ˆ1 x (2.6)
and
n n
n
yi xi
i 1 i 1
yi xi
ˆ1 i 1 n (2.7)
n 2
n
xi
i 1
xi2
i 1 n
where
1 n 1 n
y yi and x xi
ni 1 ni 1
are the averages of yi and xi, respectively. Therefore, ˆ0 and ˆ1 in Eqs. (2.6) and (2.7)
are the least-squares estimators of the intercept and slope, respectively. The fitted
simple linear regression model is then
yˆ ˆ0 ˆ1 x (2.8)
Equation (2.8) gives a point estimate of the mean of y for a particular x.

Since the denominator of Eq. (2.7) is the corrected sum of squares of the xi and
the numerator is the corrected sum of cross products of xi and yi, we may write these
quantities in a more compact notation as
n 2
n
xi n
i 1 2
Sxx xi2 xi x (2.9)
i 1 n i 1
and
n n
n
yi xi n
i 1 i 1
Sxy yi xi yi xi x (2.10)
i 1 n i 1
c02.indd 14 30-01-2021 14:25:53

Thus, a convenient way to write Eq. (2.7) is
ˆ1 Sxy
(2.11)
Sxx
The difference between the observed value yi and the corresponding fitted value
ŷi is a residual. Mathematically the ith residual is
ei yi yˆ i yi ˆ0 ˆ1 xi , i 1, 2,  , n (2.12)
Residuals play an important role in investigating model adequacy and in

detecting departures from the underlying assumptions. This topic is discussed in
subsequent chapters.
Example 2.1 The Rocket Propellant Data
A rocket motor is manufactured by bonding an igniter propellant and a sustainer

propellant together inside a metal housing. The shear strength of the bond between
the two types of propellant is an important quality characteristic. It is suspected that
shear strength is related to the age in weeks of the batch of sustainer propellant.
Twenty observations on shear strength and the age of the corresponding batch of
propellant have been collected and are shown in Table 2.1. The scatter diagram,
shown in Figure 2.1, suggests that there is a strong statistical relationship between
shear strength and propellant age, and the tentative assumption of the straight-line
model y = β0 + β1x + ε appears to be reasonable.
TABLE 2.1 Data for Example 2.1

Shear Strength, Age of Propellant,
Observation, i yi (psi) xi (weeks)
1 2158.70 15.50
2 1678.15 23.75
3 2316.00 8.00
4 2061.30 17.00
5 2207.50 5.50
6 1708.30 19.00
7 1784.70 24.00
8 2575.00 2.50
9 2357.90 7.50
10 2256.70 11.00
11 2165.20 13.00
12 2399.55 3.75
13 1779.80 25.00
14 2336.75 9.75
15 1765.30 22.00
16 2053.50 18.00
17 2414.40 6.00
18 2200.50 12.50
19 2654.20 2.00
20 1753.70 21.50
c02.indd 15 30-01-2021 14:25:58

2700
2600
2500
2400
2300
Shear strength
2200
2100
2000
1900
1800
1700
1600
0 2 4 6 8 10 12 14 16 18 20 22 24 26
Age of propellant
Figure 2.1 Scatter diagram of shear strength versus propellant age, Example 2.1.
To estimate the model parameters, first calculate
n 2
n
xi
i 1 71, 422.56
Sxx xi2 4677.69 1106.56
i 1 n 20
and
n n
xi yi
n
i 1 i 1
267.25 42, 627.15
Sxy xi yi 528, 492.64 41, 112.65
i 1 n 20
Therefore, from Eqs. (2.11) and (2.6), we find that
ˆ1 Sxy 41, 112.65

37.15
Sxx 1106.56
and
ˆ0 y ˆ1 x 2131.3575 37.15 13.3625 2627.82
c02.indd 16 30-01-2021 14:26:06

TABLE 2.2 Data, Fitted Values, and Residuals for Example 2.1
Observed Value, yi Fitted Value, ŷi Residual, ei
2158.70 2051.94 106.76
1678.15 1745.42 −67.27
2316.00 2330.59 −14.59
2061.30 1996.21 65.09
2207.50 2423.48 −215.98
1708.30 1921.90 −213.60
1784.70 1736.14 48.56
2575.00 2534.94 40.06
2357.90 2349.17 8.73
2256.70 2219.13 37.57
2165.20 2144.83 20.37
2399.55 2488.50 −88.95
1799.80 1698.98 80.82
2336.75 2265.58 71.17
1765.30 1810.44 −45.14
2053.50 1959.06 94.44
2414.40 2404.90 9.50
2200.50 2163.40 37.10
2654.20 2553.52 100.68
1753.70 1829.02 −75.32
yi 42, 627.15 yˆ i 42, 627.15 ei 0.00
The least-squares fit is
yˆ 2627.82 37.15 x
We may interpret the slope −37.15 as the average weekly decrease in propellant
shear strength due to the age of the propellant. Since the lower limit of the x’s is
near the origin, the intercept 2627.82 represents the shear strength in a batch of
propellant immediately following manufacture. Table 2.2 displays the observed
values yi, the fitted values ŷi, and the residuals. ■
After obtaining the least-squares fit, a number of interesting questions come
to mind:
1. How well does this equation fit the data?

2. Is the model likely to be useful as a predictor?
3. Are any of the basic assumptions (such as constant variance and uncorrelated
errors) violated, and if so, how serious is this?
All of these issues must be investigated before the model is finally adopted for use.
As noted previously, the residuals play a key role in evaluating model adequacy.
Residuals can be viewed as realizations of the model errors εi. Thus, to check the
constant variance and uncorrelated errors assumption, we must ask ourselves if the
residuals look like a random sample from a distribution with these properties. We
c02.indd 17 30-01-2021 14:26:16

TABLE 2.3 Minitab Regression Output for Example 2.1

Regression Analysis
The regression equation is
Strength = 2628- 37.2 Age
Predictor Coef StDev T P
Constant 2627.82 44.18 59.47 0.000
Age -37.154 2.889 -12.86 0.000
S = 96.11 R-Sq = 90.2% R-Sq(adj) = 89.6%
Analysis of Variance
Source DF SS MS F P
Regression 1 1527483 1527483 165.38 0.000
Error 18 166255 9236
Total 19 1693738
return to these questions in Chapter 4, where the use of residuals in model adequacy
checking is explored.
Computer Output Computer software packages are used extensively in fitting

regression models. Regression routines are found in both network and PC-based
statistical software, as well as in many popular spreadsheet packages. Table 2.3
presents the output from Minitab, a widely used PC-based statistics package, for the
rocket propellant data in Example 2.1. The upper portion of the table contains the
fitted regression model. Notice that before rounding the regression coefficients
agree with those we calculated manually. Table 2.3 also contains other information
about the regression model. We return to this output and explain these quantities
in subsequent sections.
2.2.2 Properties of the Least-Squares Estimators

and the Fitted Regression Model
The least-squares estimators ˆ0 and ˆ1 have several important properties. First, note
from Eqs. (2.6) and (2.7) that ˆ0 and ˆ1 are linear combinations of the observations
yi. For example,
Sxy n
ˆ1 ci yi
Sxx i 1
where ci xi x Sxx for i = 1, 2, . . . , n.

The least-squares estimators ˆ0 and ˆ1 are unbiased estimators of the model
parameters β0 and β1. To show this for ˆ1, consider
n n
E ˆ1 E ci yi ci E yi
i 1 i 1
n n n
ci 0 1 xi 0 ci 1 ci xi
i 1 i 1 i 1
c02.indd 18 30-01-2021 14:26:26

n
since E(εi) = 0 by assumption. Now we can show directly that i 1 ci 0 and
n
i 1 ci xi 1, so
E ˆ1 1
That is, if we assume that the model is correct [E(yi) = β0 + β1xi], then ˆ1 is an
unbiased estimator of β1. Similarly we may show that ˆ0 is an unbiased estimator
of β0, or
E ˆ0 0
The variance of ˆ1 is found as
n n
Var ˆ1 Var ci yi ci2 Var yi (2.13)
i 1 i 1
because the observations yi are uncorrelated, and so the variance of the sum is just
the sum of the variances. The variance of each term in the sum is ci2 Var yi , and we
have assumed that Var(yi) = σ 2; consequently,
n
2 2
n
xi x 2
Var ˆ1 2
ci2 i 1
2
(2.14)
i 1 Sxx Sxx
The variance of ˆ0 is
Var ˆ0 Var y ˆ1 x
Var y x 2 Var ˆ1 2 xCov y, ˆ1
Now the variance of y is just Var y 2

n , and the covariance between y and ˆ1
can be shown to be zero (see Problem 2.25). Thus,
1 x2
Var ˆ0 Var y x 2 Var ˆ1 2
(2.15)
n Sxx
Another important result concerning the quality of the least-squares estimators
ˆ0 and ˆ1 is the Gauss-Markov theorem, which states that for the regression model
(2.1) with the assumptions E(ε) = 0, Var(ε) = σ 2, and uncorrelated errors, the least-
squares estimators are unbiased and have minimum variance when compared with
all other unbiased estimators that are linear combinations of the yi. We often say
that the least-squares estimators are best linear unbiased estimators, where “best”
implies minimum variance. Appendix C.4 proves the Gauss-Markov theorem for the
more general multiple linear regression situation, of which simple linear regression
is a special case.
c02.indd 19 30-01-2021 14:26:47

There are several other useful properties of the least-squares fit:
1. The sum of the residuals in any regression model that contains an intercept β0
is always zero, that is,
n n
yi yˆ i ei 0
i 1 i 1
This property follows directly from the first normal equation in Eqs. (2.5) and
is demonstrated in Table 2.2 for the residuals from Example 2.1. Rounding
errors may affect the sum.
2. The sum of the observed values yi equals the sum of the fitted values ŷi, or
n n
yi yˆ i
i 1 i 1
Table 2.2 demonstrates this result for Example 2.1.

3. The least-squares regression line always passes through the centroid [the point
(y, x)] of the data.
4. The sum of the residuals weighted by the corresponding value of the regressor
variable always equals zero, that is,
n
xi ei 0
i 1
5. The sum of the residuals weighted by the corresponding fitted value always
equals zero, that is,
n
ŷi ei 0
i 1
2.2.3 Estimation of σ 2
In addition to estimating β0 and β1, an estimate of σ 2 is required to test hypotheses
and construct interval estimates pertinent to the regression model. Ideally we would
like this estimate not to depend on the adequacy of the fitted model. This is only
possible when there are several observations on y for at least one value of x (see
Section 4.5) or when prior information concerning σ 2 is available. When this approach
cannot be used, the estimate of σ 2 is obtained from the residual or error sum of
squares,
n n
2
SSRes ei2 yi yˆ i (2.16)
i 1 i 1
A convenient computing formula for SSRes may be found by substituting yˆ i ˆ0 ˆ1 xi

into Eq. (2.16) and simplifying, yielding
c02.indd 20 30-01-2021 14:26:59

n
SSRes yi2 ny 2 ˆ1Sxy (2.17)
i 1
But
n n
2
yi2 ny 2 yi y SST
i 1 i 1
is just the corrected sum of squares of the response observations, so
SSRes SST ˆ1Sxy (2.18)
The residual sum of squares has n − 2 degrees of freedom, because two degrees
of freedom are associated with the estimates ˆ0 and ˆ1 involved in obtaining ŷi.
Section C.3 shows that the expected value of SSRes is E(SSRes) = (n − 2)σ 2, so an
unbiased estimator of σ 2 is
SSRes
ˆ2 MSRes (2.19)
n 2
The quantity MSRes is called the residual mean square. The square root of ˆ 2 is
sometimes called the standard error of regression, and it has the same units as the
response variable y.
Because ˆ 2 depends on the residual sum of squares, any violation of the assump-
tions on the model errors or any misspecification of the model form may seriously
damage the usefulness of ˆ 2 as an estimate of σ 2. Because ˆ 2 is computed from the
regression model residuals, we say that it is a model-dependent estimate of σ 2.
To estimate σ 2 for the rocket propellant data in Example 2.1, first find
n 2
n n
yi
i 1
SST yi2 ny 2
yi2
i 1 i 1 n
2
42, 627.15
92, 547, 433.45 1, 693, 737.60
20
From Eq. (2.18) the residual sum of squares is
SSRes SST ˆ1Sxy

1, 693, 737.60 37.15 41, 112.65 166, 402..65
Therefore, the estimate of σ 2 is computed from Eq. (2.19) as
SSRes 166, 402.65

ˆ2 9244.59
n 2 18
c02.indd 21 30-01-2021 14:27:17

Remember that this estimate of σ 2 is model dependent. Note that this differs slightly
from the value given in the Minitab output (Table 2.3) because of rounding. ■
2.2.4 Alternate Form of the Model
There is an alternate form of the simple linear regression model that is occasionally
useful. Suppose that we redefine the regressor variable xi as the deviation from its
own average, say xi x . The regression model then becomes
yi 0 1 xi x 1x i
0 1x 1 xi x i
0 1 xi x i (2.20)
Note that redefining the regressor variable in Eq. (2.20) has shifted the origin of the
x’s from zero to x. In order to keep the fitted values the same in both the original
and transformed models, it is necessary to modify the original intercept. The rela-
tionship between the original and transformed intercept is
0 0 1x (2.21)
It is easy to show that the least-squares estimator of the transformed intercept

is ˆ0 y. The estimator of the slope is unaffected by the transformation. This alter-
nate form of the model has some advantages. First, the least-squares estimators
ˆ0 y and ˆ1 Sxy Sxx are uncorrelated, that is, Cov ˆ , ˆ 0. This will make some
0 1
applications of the model easier, such as finding confidence intervals on the mean
of y (see Section 2.4.2). Finally, the fitted model is
yˆ y ˆ1 x x (2.22)
Although Eqs. (2.22) and (2.8) are equivalent (they both produce the same value
of ŷ for the same value of x), Eq. (2.22) directly reminds the analyst that the regres-
sion model is only valid over the range of x in the original data. This region is
centered at x.
2.3 HYPOTHESIS TESTING ON THE SLOPE AND INTERCEPT
We are often interested in testing hypotheses and constructing confidence intervals

about the model parameters. Hypothesis testing is discussed in this section, and
Section 2.4 deals with confidence intervals. These procedures require that we make
the additional assumption that the model errors εi are normally distributed. Thus,
the complete assumptions are that the errors are normally and independently dis-
tributed with mean 0 and variance σ 2, abbreviated NID(0, σ 2). In Chapter 4 we
discuss how these assumptions can be checked through residual analysis.
2.3.1 Use of t Tests

Suppose that we wish to test the hypothesis that the slope equals a constant, say β10.
The appropriate hypotheses are
c02.indd 22 30-01-2021 14:27:29

Hypothesis Testing on the Slope and Intercept 23
H 0: 1 10 , H1: 1 10 (2.23)
where we have specified a two-sided alternative. Since the errors εi are NID(0, σ 2),
the observations yi are NID(β0 + β1xi, σ 2). Now ˆ1 is a linear combination of the
observations, so ˆ1 is normally distributed with mean β1 and variance σ 2/Sxx using
the mean and variance of ˆ1 found in Section 2.2.2. Therefore, the statistic
ˆ1 10
Z0
2
Sxx
is distributed N(0, 1) if the null hypothesis H0: β1 = β10 is true. If σ 2 were known, we
could use Z0 to test the hypotheses (2.23). Typically, σ 2 is unknown. We have already
seen that MSRes is an unbiased estimator of σ 2. Appendix C.3 establishes that
(n − 2)MSRes/σ 2 follows a n 2 distribution and that MSRes and ˆ1 are independent.
2
By the definition of a t statistic given in Section C.1,

ˆ1 10
t0 (2.24)
MSRes Sxx
follows a tn−2 distribution if the null hypothesis H0: β1 = β10 is true. The degrees of
freedom associated with t0 are the number of degrees of freedom associated with
MSRes. Thus, the ratio t0 is the test statistic used to test H0: β1 = β10. The test procedure
computes t0 and compares the observed value of t0 from Eq. (2.24) with the upper
α/2 percentage point of the tn−2 distribution (tα/2,n−2). This procedure rejects the null
hypothesis if
t0 t 2, n 2 (2.25)
Alternatively, a P-value approach could also be used for decision making.

The denominator of the test statistic, t0, in Eq. (2.24) is often called the estimated
standard error, or more simply, the standard error of the slope. That is,
MSRes
se ˆ1 (2.26)
Sxx
Therefore, we often see t0 written as
ˆ1 10
t0 (2.27)
se ˆ1
A similar procedure can be used to test hypotheses about the intercept. To test
H 0: 0 00 , H1: 0 00 (2.28)
we would use the test statistic

ˆ0 00
ˆ0 00
t0 (2.29)
MSRes 1 n x 2
Sxx se ˆ0
c02.indd 23 30-01-2021 14:27:46

where se ˆ0 MSRes 1 n x 2 Sx is the standard error of the intercept. We reject

the null hypothesis H0: β0 = β00 if |t0| > tα/2,n−2.
2.3.2 Testing Significance of Regression

A very important special case of the hypotheses in Eq. (2.23) is
H 0: 1 0, H1: 1 0 (2.30)
These hypotheses relate to the significance of regression. Failing to reject H0: β1 = 0

implies that there is no linear relationship between x and y. This situation is illus-
trated in Figure 2.2. Note that this may imply either that x is of little value in explain-
ing the variation in y and that the best estimator of y for any x is ŷ y (Figure 2.2a)
or that the true relationship between x and y is not linear (Figure 2.2b). Therefore,
failing to reject H0: β1 = 0 is equivalent to saying that there is no linear relationship
between y and x.
Alternatively, if H0: β1 = 0 is rejected, this implies that x is of value in explaining
the variability in y. This is illustrated in Figure 2.3. However, rejecting H0: β1 = 0
could mean either that the straight-line model is adequate (Figure 2.3a) or that even
y y
x x
(a) (b)
Figure 2.2 Situations where the hypothesis H0: β1 = 0 is not rejected.
y y
x x
(a) (b)
Figure 2.3 Situations where the hypothesis H0: β1 = 0 is rejected.
c02.indd 24 30-01-2021 14:27:51

though there is a linear effect of x, better results could be obtained with the addition
of higher order polynomial terms in x (Figure 2.3b).
The test procedure for H0: β1 = 0 may be developed from two approaches. The
first approach simply makes use of the t statistic in Eq. (2.27) with β10 = 0, or
ˆ1
t0
se ˆ1
The null hypothesis of significance of regression would be rejected if |t0| > tα/2,n−2.
We test for significance of regression in the rocket propellant regression model

of Example 2.1. The estimate of the slope is ˆ1 37.15, and in Example 2.2, we
computed the estimate of σ 2 to be MSRes ˆ 2 9244.59. The standard error of the
slope is
MSRes 9244.59
se ˆ1 2.89
Sxx 1106.56
Therefore, the test statistic is

ˆ1 37.15
t0 12.85
se ˆ1 2.89
If we choose α = 0.05, the critical value of t is t0.025,18 = 2.101. Thus, we would reject
H0: β1 = 0 and conclude that there is a linear relationship between shear strength
and the age of the propellant. ■
Minitab Output The Minitab output in Table 2.3 gives the standard errors of the
slope and intercept (called “StDev” in the table) along with the t statistic for testing
H0: β1 = 0 and H0: β0 = 0. Notice that the results shown in this table for the slope
essentially agree with the manual calculations in Example 2.3. Like most computer
software, Minitab uses the P-value approach to hypothesis testing. The P value for
the test for significance of regression is reported as P = 0.000 (this is a rounded
value; the actual P value is 1.64 × 10−10). Clearly there is strong evidence that
strength is linearly related to the age of the propellant. The test statistic for
H0: β0 = 0 is reported as t0 = 59.47 with P = 0.000. One would feel very confident
in claiming that the intercept is not zero in this model.
2.3.3 Analysis of Variance

We may also use an analysis-of-variance approach to test significance of regression.
The analysis of variance is based on a partitioning of total variability in the response
variable y. To obtain this partitioning, begin with the identity
yi y yˆ i y yi yˆ i (2.31)
c02.indd 25 30-01-2021 14:28:03

Squaring both sides of Eq. (2.31) and summing over all n observations
produces
n n n n
2 2 2
yi y yˆ i y yi yˆ i 2 yˆ i y yi yˆ i
i 1 i 1 i 1 i 1
Note that the third term on the right-hand side of this expression can be
rewritten as
n n n
2 yˆ i y yi yˆ i 2 yˆ i yi yˆ i 2y yi yˆ i
i 1 i 1 i 1
n n
2 yˆ i ei 2y ei 0
i 1 i 1
since the sum of the residuals is always zero (property 1, Section 2.2.2) and the sum
of the residuals weighted by the corresponding fitted value ŷi is also zero (property
5, Section 2.2.2). Therefore,
n n n
2 2 2
yi y yˆ i y yi yˆ i (2.32)
i 1 i 1 i 1
The left-hand side of Eq. (2.32) is the corrected sum of squares of the observa-
tions, SST, which measures the total variability in the observations. The two compo-
nents of SST measure, respectively, the amount of variability in the observations yi
accounted for by the regression line and the residual variation left unexplained by
n 2
the regression line. We recognize SSRes i 1 yi yˆ i as the residual or error sum
n 2
of squares from Eq. (2.16). It is customary to call i 1 yˆ i y the regression or
model sum of squares.
Equation (2.32) is the fundamental analysis-of-variance identity for a regression
model. Symbolically, we usually write
SST SSR SSRes (2.33)
Comparing Eq. (2.33) with Eq. (2.18) we see that the regression sum of squares may
be computed as
SSR ˆ1Sxy (2.34)
The degree-of-freedom breakdown is determined as follows. The total sum of

squares, SST, has dfT = n − 1 degrees of freedom because one degree of freedom is
lost as a result of the constraint in 1 yi y on the deviations yi y . The model or
regression sum of squares, SSR, has dfR = 1 degree of freedom because SSR is
completely determined by one parameter, namely, ˆ1 [see Eq. (2.34)]. Finally, we
noted previously that SSR has dfRes = n − 2 degrees of freedom because two
c02.indd 26 30-01-2021 14:28:21

constraints are imposed on the deviations yi yˆ i as a result of estimating ˆ0 and ˆ1.

Note that the degrees of freedom have an additive property:
dfT dfR dfRes

n 1 1 n 2 (2.35)
We can use the usual analysis-of-variance F test to test the hypothesis H0: β1 = 0.
2
Appendix C.3 shows that (1) SSRes = (n − 2)MSRes/σ 2 follows a n 2 distribution;
2
(2) if the null hypothesis H0: β1 = 0 is true, then SSR/σ 2 follows a 1 distribution; and
(3) SSRes and SSR are independent. By the definition of an F statistic given in
Appendix C.1,
SSR dfR SSR 1 MSR

F0 (2.36)
SSRes dfRes SSRes n 2 MSRes
follows the F1,n−2 distribution. Appendix C.3 also shows that the expected values of
these mean squares are
2 2 2
E MSRes , E MSR 1 Sxx
These expected mean squares indicate that if the observed value of F0 is large, then
it is likely that the slope β1 ≠ 0. Appendix C.3 also shows that if β1 ≠ 0, then F0
follows a noncentral F distribution with 1 and n − 2 degrees of freedom and a non-
centrality parameter of
2
1 Sxx
2
This noncentrality parameter also indicates that the observed value of F0 should be
large if β1 ≠ 0. Therefore, to test the hypothesis H0: β1 = 0, compute the test statistic
F0 and reject H0 if
F0 F ,1, n 2
The test procedure is summarized in Table 2.4.
TABLE 2.4 Analysis of Variance for Testing Significance of Regression

Source of Degrees of
Variation Sum of Squares Freedom Mean Square F0
Regression SSR ˆ1Sxy 1 MSR MSR/MSRes
Residual SSRes SST ˆ1Sxy n−2 MSRes
Total SST n−1
c02.indd 27 30-01-2021 14:28:37

We will test for significance of regression in the model developed in Example

2.1 for the rocket propellant data. The fitted model is yˆ 2627.82 37.15 x,
SST = 1,693,737.60, and Sxy = −41,112.65. The regression sum of squares is computed
from Eq. (2.34) as
SSR ˆ1Sxy 37.15 41, 112.65 1, 527, 334.95
The analysis of variance is summarized in Table 2.5. The computed value of F0 is

165.21, and from Table A.4, F0.01,1,18 = 8.29. The P value for this test is 1.66 × 10−10.
Consequently, we reject H0: β1 = 0. ■
Minitab Output The Minitab output in Table 2.3 also presents the analysis-of-
variance test significance of regression. Comparing Tables 2.3 and 2.5, we note that
there are some slight differences between the manual calculations and those per-
formed by computer for the sums of squares. This is due to rounding the manual
calculations to two decimal places. The computed values of the test statistics essen-
tially agree.
More About the t Test We noted in Section 2.3.2 that the t statistic
ˆ1 ˆ1
t0 (2.37)
se ˆ1 MSRes Sxx
could be used for testing for significance of regression. However, note that on squar-
ing both sides of Eq. (2.37), we obtain
ˆ12 Sxx ˆ1Sxy MSR

t02 (2.38)
MSRes MSRes MSRes
Thus, t02 in Eq. (2.38) is identical to F0 of the analysis-of-variance approach in

Eq. (2.36). For example; in the rocket propellant example t0 = −12.5, so
2
t02 12.5 165.12  F0 165.21. In general, the square of a t random variable
with f degrees of freedom is an F random variable with one and f degrees of freedom
in the numerator and denominator, respectively. Although the t test for H0: β1 = 0
is equivalent to the F test in simple linear regression, the t test is somewhat more
adaptable, as it could be used for one-sided alternative hypotheses (either H1: β1 < 0
or H1: β1 > 0), while the F test considers only the two-sided alternative. Regression
TABLE 2.5 Analysis-of-Variance Table for the Rocket Propellant Regression Model
Source of Sum of Degrees of
Variation Squares Freedom Mean Square F0 P value
Regression 1,527,334.95 1 1,527,334.95 165.21 1.66 × 10−10
Residual 166,402.65 18 9,244.59
Total 1,693,737.60 19
c02.indd 28 30-01-2021 14:28:48

Another random document with
no related content on Scribd:
country was thrown on the defensive. On the east the possession of
Montreal or Kingston would cut off all Canada above from support
by the sea, which would be equivalent to ensuring its fall. “I shall
continue to exert myself to the utmost to overcome every difficulty,”
wrote Brock, who gave such emphatic proof of energetic and
sagacious exertion in his subsequent course. “Should, however, the
communication between Montreal and Kingston be cut off, the fate
of the troops in this part of the province will be decided.”[93] “The
Montreal frontier,” said the officer selected by the Duke of
Wellington to report on the defenses of Canada, “is the most
important, and at present [1826] confessedly most vulnerable and
accessible part of Canada.”[94] There, then, was the direction for
offensive operations by the United States; preferably against
Montreal, for, if successful, a much larger region would be isolated
and reduced. Montreal gone, Kingston could receive no help from
without; and, even if capable of temporary resistance, its surrender
would be but a question of time. Coincidently with this military
advance, naval development for the control of the lakes should have
proceeded, as a discreet precaution; although, after the fall of
Kingston and Montreal, there could have been little use of an inland
navy, for the British local resources would then have been
inadequate to maintain an opposing force.
Results of the Northern Campaign
[While control was more vital and the forces stronger on Lake
Ontario than on either Erie or Champlain, no naval action of
consequence occurred there in 1813 or in fact throughout the war.
Yeo, the British commander, was enjoined by Admiralty orders to
take no risks; and the American Commodore Chauncey, with no such
justification, adopted a similar policy. Hence the important fleet
actions of the war were in other waters—Perry’s victory of September
10, 1813, on Lake Erie, and Macdonough’s victory on Lake
Champlain a year later. The first sentence in the paragraph following
refers to a raid on Buffalo, December 30, 1813.—Editor.]
With this may be said to have terminated the northern campaign
of 1813. The British had regained full control of the Niagara
peninsula, and they continued to hold Fort Niagara, in the State of
New York, till peace was concluded. The only substantial gain on the
whole frontier, from the extreme east to the extreme west, was the
destruction of the British fleet on Lake Erie, and the consequent
transfer of power in the west to the United States. This was the left
flank of the American position. Had the same result been
accomplished on the right flank,—as it might have been,—at
Montreal, or even at Kingston, the center and left must have fallen
also. For the misdirection of effort to Niagara, the local commanders,
Dearborn and Chauncey, are primarily responsible; for Armstrong[95]
yielded his own correct perceptions to the representations of the first
as to the enemy’s force, supported by the arguments of the naval
officer favoring the diversion of effort from Kingston to Toronto.
Whether Chauncey ever formally admitted to himself this
fundamental mistake, which wrecked the summer’s work upon Lake
Ontario, does not appear; but that he had learned from experience is
shown by a letter to the Secretary of the Navy,[96] when the squadrons
had been laid up. In this he recognized the uselessness of the heavy
sailing schooners when once a cruising force of ships for war had
been created, thereby condemning much of his individual
management of the campaign; and he added: “If it is determined to
prosecute the war offensively, and secure our conquests in Upper
Canada, Kingston ought unquestionably to be the first object of
attack, and that so early in the spring as to prevent the enemy from
using the whole of the naval force that he is preparing.”
In the three chapters which here end, the Ontario operations have
been narrated consecutively and at length, without interruption by
other issues,—except the immediately related Lake Erie campaign,—
because upon them turned, and upon them by the dispositions of the
government this year were wrecked the fortunes of the war. The year
1813, from the opening of the spring to the closing in of winter, was
for several reasons the period when conditions were most propitious
to the American cause. In 1812 war was not begun until June, and
then with little antecedent preparation; and it was waged half-
heartedly, both governments desiring to nip hostilities. In 1814, on
the other hand, when the season opened, Napoleon had fallen, and
the United States no longer had an informal ally to divert the efforts
of Great Britain. But in the intervening year, 1813, although the
pressure upon the seaboard, the defensive frontier, was undoubtedly
greater than before, and much vexation and harassment was
inflicted, no serious injury was done beyond the suppression of
commerce, inevitable in any event. In the north, on the lakes
frontier, the offensive and the initiative continued in the hands of the
United States. No substantial reinforcements reached Canada until
long after the ice broke up, and then in insufficient numbers. British
naval preparations had been on an inadequate scale, receiving no
proper professional supervision. The American Government, on the
contrary, had had the whole winter to prepare, and the services of a
very competent naval organizer. It had also the same period to get
ready its land forces; while incompetent Secretaries of War and of
the Navy gave place in January to capable men in both situations.
With all this in its favor, and despite certain gratifying successes,
the general outcome was a complete failure, the full measure of
which could be realized only when the downfall of Napoleon revealed
what disaster may result from neglect to seize opportunity while it
exists. The tide then ebbed, and never again flowed. For this many
causes may be alleged. The imbecile ideas concerning military and
naval preparation which had prevailed since the opening of the
century doubtless counted for much. The entrusting of chief
command to broken-down men like Dearborn and Wilkinson was
enough to ruin the best conceived schemes. But, despite these very
serious drawbacks, the strategic misdirection of effort was the most
fatal cause of failure.
There is a simple but very fruitful remark of a Swiss military
writer, that every military line may be conceived as having three
parts, the middle and the two ends, or flanks. As sound principle
requires that military effort should not be distributed along the
whole of an enemy’s position,—unless in the unusual case of
overwhelming superiority,—but that distinctly superior numbers
should be concentrated upon a limited portion of it, this idea of a
threefold division aids materially in considering any given situation.
One third, or two thirds, of an enemy’s line may be assailed, but very
seldom the whole; and everything may depend upon the choice made
for attack. Now the British frontier, which the United States was to
assail, extended from Montreal on the east to Detroit on the west. Its
three parts were: Montreal and the St. Lawrence on the east, or left
flank; Ontario in the middle, centering at Kingston; and Erie on the
right; the strength of the British position in the last-named section
being at Detroit and Malden, because they commanded the straits
upon which the Indian tribes depended for access to the east. Over
against the British positions named lay those of the United States.
Given in the same order, these were: Lake Champlain, and the shores
of Ontario and of Erie, centering respectively in the naval stations at
Sackett’s Harbor and Presque Isle.
Accepting these definitions, which are too obvious to admit of
dispute, what considerations should have dictated to the United
States the direction of attack; the one, or two, parts out of the three,
on which effort should be concentrated? The reply, as a matter of
abstract, accepted, military principle, is certain. Unless very urgent
reasons to the contrary exist, strike at one end rather than at the
middle, because both ends can come up to help the middle against
you quicker than one end can get to help the other; and, as between
the two ends, strike at the one upon which the enemy most depends
for reinforcements and supplies to maintain his strength. Sometimes
this decision presents difficulties. Before Waterloo, Wellington had
his own army as a center of interest; on his right flank the sea,
whence came supplies and reinforcements from England; on his left
the Prussian army, support by which was imminently necessary. On
which flank would Napoleon throw the weight of his attack?
Wellington reasoned, perhaps through national bias, intensified by
years of official dependence upon sea support, that the blow would
fall upon his right, and he strengthened it with a body of men sorely
needed when the enemy came upon his left, in overwhelming
numbers, seeking to separate him from the Prussians.
No such doubt was possible as to Canada in 1813. It depended
wholly upon the sea, and it touched the sea at Montreal. The United
States, with its combined naval and military strength, crude as the
latter was, was at the beginning of 1813 quite able in material power
to grapple two out of the three parts,—Montreal and Kingston. Had
they been gained, Lake Erie would have fallen; as is demonstrated by
the fact that the whole Erie region went down like a house of cards
the moment Perry triumphed on the lake. His victory was decisive,
simply because it destroyed the communications of Malden with the
sea. The same result would have been achieved, with effect over a far
wider region, by a similar success in the east.
27. Lessons of the War with Spain[97]
The Possibilities of a “Fleet in Being”
[Admiral Cervera left the Cape Verde Islands on April 29, 1898. After
touching at Martinique on May 11, he coaled at Curaçao on the 15th,
and entered Santiago on the 19th.
On news of Cervera’s arrival at Martinique, Sampson’s squadron
from Porto Rico and Schley’s Flying Squadron from Hampton Roads
converged on Key West. Sampson had his full strength in the
approaches to Havana by the 21st and Schley was off Cienfuegos, the
chief southern port of Cuba, on the 22d.
“We cannot,” writes Admiral Mahan, “expect ever again to have an
enemy so entirely inapt as Spain showed herself to be; yet, even so,
Cervera’s division reached Santiago on the 19th of May, two days
before our divisions appeared in the full force they could muster
before Havana and Cienfuegos.”[98]—Editor.]
As was before said, the disparity between the armored fleets of the
two nations was nominally inconsiderable; and the Spaniards
possessed one extremely valuable—and by us unrivalled—advantage
in a nearly homogeneous group of five[99] armored cruisers, very fast,
and very similar both in nautical qualities and in armament. It is
difficult to estimate too highly the possibilities open to such a body of
ships, regarded as a “fleet in being,” to use an expression that many
of our readers may have seen, but perhaps scarcely fully understood.
The phrase “fleet in being,” having within recent years gained
much currency in naval writing, demands—like the word “jingo”—
preciseness of definition; and this, in general acceptance, it has not
yet attained. It remains, therefore, somewhat vague, and so
occasions misunderstandings between men whose opinions perhaps
do not materially differ. The writer will not attempt to define, but a
brief explanation of the term and its origin may not be amiss. It was
first used, in 1690, by the British admiral Lord Torrington, when
defending his course in declining to engage decisively, with an
inferior force, a French fleet, then dominating in the Channel, and
under cover of which it was expected that a descent upon the English
coast would be made by a great French army. “Had I fought
otherwise,” he said, “our fleet had been totally lost, and the kingdom
had lain open to invasion. As it was, most men were in fear that the
French would invade; but I was always of another opinion, for I
always said that whilst we had a fleet in being, they would not dare to
make an attempt.”
A “fleet in being,” therefore, is one the existence and maintenance
of which, although inferior, on or near the scene of operations, is a
perpetual menace to the various more or less exposed interests of the
enemy, who cannot tell when a blow may fall, and who is therefore
compelled to restrict his operations, otherwise possible, until that
fleet can be destroyed or neutralized. It corresponds very closely to “a
position on the flank and rear” of an enemy, where the presence of a
smaller force, as every military student knows, harasses, and may
even paralyze, offensive movements. When such a force is extremely
mobile, as a fleet of armored cruisers may be, its power of mischief is
very great; potentially, it is forever on the flank and rear, threatening
the lines of communications. It is indeed as a threat to
communications that the “fleet in being” is chiefly formidable.
The theory received concrete and convincing illustration during
the recent hostilities, from the effect exerted—and justly exerted—
upon our plans and movements by Cervera’s squadron, until there
had been assembled before Santiago a force at once so strong and so
numerous as to make his escape very improbable. Even so, when a
telegram was received from a capable officer that he had identified
by night, off the north coast of Cuba, an armored cruiser,—which, if
of that class, was most probably an enemy,—the sailing of Shafter’s
expedition was stopped until the report could be verified. So much
for the positive, material influence—in the judgment of the writer,
the reasonable influence—of a “fleet in being.” As regards the moral
effect, the effect upon the imagination, it is scarcely necessary more
than to allude to the extraordinary play of the fancy, the
kaleidoscopic effects elicited from our own people, and from some
foreign critics, in propounding dangers for ourselves and ubiquity for
Cervera. Against the infection of such tremors it is one of the tasks of
those in responsibility to guard themselves and, if possible, their
people. “Don’t make pictures for yourself,” was Napoleon’s warning
to his generals. “Every naval operation since I became head of the
government has failed, because my admirals see double and have
learned—where I don’t know—that war can be made without running
risks.”
The probable value of a “fleet in being” has, in the opinion of the
writer, been much overstated; for, even at the best, the game of
evasion, which this is, if persisted in, can have but one issue. The
superior force will in the end run the inferior to earth. In the
meanwhile, however, vital time may have been lost. It is conceivable,
for instance, that Cervera’s squadron, if thoroughly effective, might,
by swift and well-concealed movements, have detained our fleet in
the West Indies until the hurricane of September, 1898, swept over
the Caribbean. We had then no reserve to replace armored ships lost
or damaged. But, for such persistence of action, there is needed in
each unit of the “fleet in being” an efficiency rarely attainable, and
liable to be lost by unforeseen accident at a critical moment. Where
effect, nay, safety, depends upon mere celerity of movement, as in
retreat, a crippled ship means a lost ship; or a lost fleet, if the body
sticks to its disabled member. Such efficiency it is probable Cervera’s
division never possessed. The length of its passage across the
Atlantic, however increased by the embarrassment of frequently
recoaling the torpedo destroyers, so far overpassed the extreme
calculations of our naval authorities, that ready credence was given
to an apparently authentic report that it had returned to Spain; the
more so that such concentration was strategically correct, and it was
incorrect to adventure an important detachment so far from home,
without the reinforcement it might have received in Cadiz. This
delay, in ships whose individual speed had originally been very high,
has been commonly attributed in our service to the inefficiency of the
engine-room force; and this opinion is confirmed by a Spanish
officer writing in their “Revista de la Marina.” “The Americans,” he
says, “keep their ships cruising constantly, in every sea, and therefore
have a large and qualified engine-room force. We have but few
machinists, and are almost destitute of firemen.” This inequality,
however, is fundamentally due to the essential differences of
mechanical capacity and development in the two nations. An
amusing story was told the writer some years ago by one of our
consuls in Cuba. Making a rather rough passage between two ports,
he saw an elderly Cuban or Spanish gentleman peering frequently
into the engine-room, with evident uneasiness. When asked the
cause of his concern, the reply was, “I don’t feel comfortable unless
the man in charge of the engines talks English to them.”
When to the need of constant and sustained ability to move at high
speed is added the necessity of frequent recoaling, allowing the
hostile navy time to come up, it is evident that the active use of a
“fleet in being,” however perplexing to the enemy, must be both
anxious and precarious to its own commander. The contest is one of
strategic wits, and it is quite possible that the stronger, though
slower, force, centrally placed, may, in these days of cables, be able to
receive word and to corner its antagonist before the latter can fill his
bunkers. Of this fact we should probably have received a very
convincing illustration, had a satisfactory condition of our coast
defenses permitted the Flying Squadron to be off Cienfuegos, or even
off Havana, instead of in Hampton Roads. Cervera’s entrance to
Santiago was known to us within twenty-four hours. In twenty-four
more it could have been communicated off Cienfeugos by a fast
despatch boat, after which less than forty-eight would have placed
our division before Santiago. The uncertainty felt by Commodore
Schley, when he arrived off Cienfuegos, as to whether the Spanish
division was inside or no, would not have existed had his squadron
been previously blockading; and his consequent delay of over forty-
eight hours—with the rare chance thus offered to Cervera—would not
have occurred. To coal four great ships within that time was probably
beyond the resources of Santiago; whereas the speed predicted for
our own movements is rather below than above the dispositions
contemplated to ensure it.
The great end of a war fleet, however, is not to chase, nor to fly, but
to control the seas. Had Cervera escaped our pursuit at Santiago, it
would have been only to be again paralyzed at Cienfuegos or at
Havana. When speed, not force, is the reliance, destruction may be
postponed, but can be escaped only by remaining in port. Let it not,
therefore, be inferred, from the possible, though temporary, effect of
a “fleet in being,” that speed is the chief of all factors in the
battleship. This plausible, superficial notion, too easily accepted in
these days of hurry and of unreflecting dependence upon machinery
as the all in all, threatens much harm to the future efficiency of the
navy. Not speed, but power of offensive action, is the dominant
factor in war. The decisive preponderant element of great land forces
has ever been the infantry, which, it is needless to say, is also the
slowest. The homely summary of the art of war, “To get there first
with the most men,” has with strange perverseness been so distorted
in naval—and still more in popular—conception, that the second and
more important consideration has been subordinated to the former
and less essential. Force does not exist for mobility, but mobility for
force. It is of no use to get there first unless, when the enemy in turn
arrives, you have also the most men,—the greater force. This is
especially true of the sea, because there inferiority of force—of gun
power—cannot be compensated, as on land it at times may be, by
judiciously using accidents of the ground. I do not propose to fall
into an absurdity of my own by questioning the usefulness of higher
speed, provided the increase is not purchased at the expense of
strictly offensive power; but the time has come to say plainly that its
value is being exaggerated; that it is in the battleship secondary to
gun power; that a battle fleet can never attain, nor maintain, the
highest rate of any ship in it, except of that one which at the moment
is the slowest, for it is a commonplace of naval action that fleet speed
is that of the slowest ship; that not exaggerated speed, but uniform
speed—sustained speed—is the requisite of the battle fleet; that it is
not machinery, as is often affirmed, but brains and guns, that win
battles and control of the sea. The true speed of war is not headlong
precipitancy, but the unremitting energy which wastes no time.
For the reasons that have been given, the safest, though not the
most effective, disposition of an inferior “fleet in being” is to lock it
up in an impregnable port or ports, imposing upon the enemy the
intense and continuous strain of watchfulness against escape. This it
was that Torrington, the author of the phrase, proposed for the time
to do. Thus it was that Napoleon, to some extent before Trafalgar,
but afterward with set and exclusive purpose, used the French Navy,
which he was continually augmenting, and yet never, to the end of
his reign, permitted again to undertake any serious expedition. The
mere maintenance of several formidable detachments, in apparent
readiness, from the Scheldt round to Toulon, presented to the British
so many possibilities of mischief that they were compelled to keep
constantly before each of the French ports a force superior to that
within, entailing an expense and an anxiety by which the emperor
hoped to exhaust their endurance. To some extent this was Cervera’s
position and function in Santiago, whence followed logically the
advisability of a land attack upon the port, to force to a decisive issue
a situation which was endurable only if incurable. “The destruction
of Cervera’s squadron,” justly commented an Italian writer, before
the result was known, “is the only really decisive fact that can result
from the expedition to Santiago, because it will reduce to impotence
the naval power of Spain. The determination of the conflict will
depend throughout upon the destruction of the Spanish sea power,
and not upon territorial descents, although the latter may aggravate
the situation.” The American admiral from before Santiago, when
urging the expedition of a land force to make the bay untenable,
telegraphed, “The destruction of this squadron will end the war;” and
it did.
28. The Santiago Blockade[100]
Our battle fleet before Santiago was more than powerful enough to
crush the hostile squadron in a very short time if the latter attempted
a stand-up fight. The fact was so evident that it was perfectly clear
nothing of the kind would be hazarded; but, nevertheless, we could
not afford to diminish the number of armored vessels on this spot,
now become the determining center of the conflict. The possibility of
the situation was twofold. Either the enemy might succeed in an
effort at evasion, a chance which required us to maintain a distinctly
superior force of battleships in order to allow the occasional absence
of one or two for coaling or repairs, besides as many lighter cruisers
as could be mustered for purposes of lookout, or, by merely
remaining quietly at anchor, protected from attack by the lines of
torpedoes, he might protract a situation which tended not only to
wear out our ships, but also to keep them there into the hurricane
season,—a risk which was not, perhaps, adequately realized by the
people of the United States.
It is desirable at this point to present certain other elements of the
naval situation which weightily affected naval action at the moment,
and which, also, were probably overlooked by the nation at large, for
they give a concrete illustration of conditions, which ought to
influence our national policy, as regards the navy, in the present and
immediate future. We had to economize our ships because they were
too few. There was no reserve. The Navy Department had
throughout, and especially at this period, to keep in mind, not merely
the exigencies at Santiago, but the fact that we had not a battleship in
the home ports that could in six months be made ready to replace
one lost or seriously disabled, as the Massachusetts, for instance, not
long afterwards was, by running on an obstruction in New York Bay.
Surprise approaching disdain was expressed, both before and after
the destruction of Cervera’s squadron, that the battle fleet was not
sent into Santiago either to grapple the enemy’s ships there, or to
support the operations of the army, in the same way, for instance,
that Farragut crossed the torpedo lines at Mobile. The reply—and, in
the writer’s judgment, the more than adequate reason—was that the
country could not at that time, under the political conditions which
then obtained, afford to risk the loss or disablement of a single
battleship, unless the enterprise in which it was hazarded carried a
reasonable probability of equal or greater loss to the enemy, leaving
us, therefore, as strong as before relatively to the naval power which
in the course of events might yet be arrayed against us. If we lost ten
thousand men, the country could replace them; if we lost a
battleship, it could not be replaced. The issue of the war, as a whole
and in every locality to which it extended, depended upon naval
force, and it was imperative to achieve, not success only, but success
delayed no longer than necessary. A million of the best soldiers
would have been powerless in face of hostile control of the sea.
Dewey had not a battleship, but there can be no doubt that that
capable admiral thought he ought to have one or more; and so he
ought, if we had had them to spare. The two monitors would be
something, doubtless, when they arrived; but, like all their class, they
lacked mobility.
When Cámara started by way of Suez for the East, it was no more
evident than it was before that we ought to have battleships there.
That was perfectly plain from the beginning; but battleships no more
than men can be in two places at once, and until Cámara’s movement
had passed beyond the chance of turning west, the Spanish fleet in
the Peninsula had, as regarded the two fields of war, the West Indies
and the Philippines, the recognized military advantage of an interior
position. In accepting inferiority in the East, and concentrating our
available force in the West Indies, thereby ensuring a superiority
over any possible combination of Spanish vessels in the latter
quarter, the Department acted rightly and in accordance with sound
military precedent; but it must be remembered that the Spanish
Navy was not the only possibility of the day. The writer was not in a
position to know then, and does not know now, what weight the
United States Government attached to the current rumors of possible
political friction with other states whose people were notoriously
sympathizers with our enemy. The public knows as much about that
as he does; but it was clear that if a disposition to interfere did exist
anywhere, it would not be lessened by a serious naval disaster to us,
such as the loss of one of our few battleships would be. Just as in the
maintenance of a technically “effective” blockade of the Cuban ports,
so, also, in sustaining the entireness and vigor of the battle fleet, the
attitude of foreign Powers as well as the strength of the immediate
enemy had to be considered. For such reasons it was recommended
that the orders on this point to Admiral Sampson should be
peremptory; not that any doubt existed as to the discretion of that
officer, who justly characterized the proposition to throw the ships
upon the mine fields of Santiago as suicidal folly, but because it was
felt that the burden of such a decision should be assumed by a
superior authority, less liable to suffer in personal reputation from
the idle imputations of over-caution, which at times were ignorantly
made by some who ought to have known better, but did not. “The
matter is left to your discretion,” the telegram read, “except that the
United States armored vessels must not be risked.”
When Cervera’s squadron was once cornered, an intelligent
opponent would, under any state of naval preparedness, have seen
the advisability of forcing him out of the port by an attack in the rear,
which could be made only by an army. As Nelson said on one
occasion, “What is wanted now is not more ships, but troops.” Under
few conditions should such a situation be prolonged. But the reasons
adduced in the last paragraph made it doubly incumbent upon us to
bring the matter speedily to an issue, and the combined expedition
from Tampa was at once ordered. Having in view the number of
hostile troops in the country surrounding Santiago, as shown by the
subsequent returns of prisoners, and shrewdly suspected by
ourselves beforehand, it was undoubtedly desirable to employ a
larger force than was sent. The criticism made upon the inadequate
number of troops engaged in this really daring movement is
intrinsically sound, and would be wholly accurate if directed, not
against the enterprise itself, but against the national
shortsightedness which gave us so trivial an army at the outbreak of
the war. The really hazardous nature of the movement is shown by
the fact that the column of Escario, three thousand strong, from
Manzanillo, reached Santiago on July 3d; too late, it is true,
abundantly too late, to take part in the defense of San Juan and El
Caney, upon holding which the city depended for food and water; yet
not so late but that it gives a shivering suggestion how much more
arduous would have been the task of our troops had Escario come up
in time. The incident but adds another to history’s long list of
instances where desperate energy and economy of time have wrested
safety out of the jaws of imminent disaster. The occasion was one
that called upon us to take big risks; and success merely justifies
doubly an attempt which, from the obvious balance of advantages
and disadvantages, was antecedently justified by its necessity, and
would not have been fair subject for blame, even had it failed.
The Navy Department did not, however, think that even a small
chance of injury should be taken which could be avoided; and it may
be remarked that, while the man is unfit for command who, on
emergency, is unable to run a very great risk for the sake of decisive
advantage, he, on the other hand, is only less culpable who takes
even a small risk of serious harm against which reasonable
precaution can provide. It has been well said that Nelson took more
care of his topgallant masts, in ordinary cruising, than he did of his
whole fleet when the enemy was to be checked or beaten; and this
combination of qualities apparently opposed is found in all strong
military characters to the perfection of which both are necessary.
29. “Fleet in Being” and “Fortress Fleet”[101]
The Port Arthur Squadron in the Russo-Japanese War
[At the outbreak of the Russo-Japanese War in February, 1904,

Russia had three armored cruisers at Vladivostok, another at
Chemulpo, Korea, and seven battleships, six cruisers, and a torpedo
flotilla at Port Arthur. Three of the Port Arthur ships were badly
damaged by torpedo attack on February 8, and the cruiser at
Chemulpo was destroyed on the next day. Togo lost two of his six
first-class battleships by running into a mine field off Port Arthur on
May 15. In an attempt to escape to Vladivostok on August 10, the
Port Arthur squadron lost a battleship and several cruisers; the
remainder were sunk in the course of the Port Arthur siege. This
lasted from May 27 to January 1, 1905. Even before February 8,
1904, the Japanese had begun transporting their troops to Korea;
and after the fall of Port Arthur they were able to throw their full
strength against General Kuropatkin in the decisive battle of
Mukden, February 24, 1905.—Editor.]
I have been led, on an occasion not immediately connected with
Naval Strategy, to observe that errors and defeats are more obviously
illustrative of principles than successes are. It is from the records of
the beaten side that we are most surely able to draw instruction. This
is partly due to the fact that the general or admiral who is worsted
has to justify himself to his people, perhaps also to his Government.
The naval practice of court-martialing a defeated captain or admiral
has been most productive of the material which history, and the art
of war, both require for their treatment. Even failing a court-martial,
defeat cries aloud for explanation; whereas success, like charity,
covers a multitude of sins. To this day Marengo is the victory of
Napoleon, not of Desaix; and the hazardous stretching of the French
line which caused the first defeat is by most forgotten in the ultimate
triumph. The man who has failed will of his own motion bring out all
that extenuates failure, or relieves him from the imputation of it. The
victor is asked few questions; and if conscious of mistakes he need
not reveal them. More can be found to criticize Kuropatkin and
Rozhestvensky than to recognize either their difficulties or their
merits. Probably few, even in this naval audience, knew, or have
noted, that on the day preceding that on which two Japanese
battleships, the Hatsuse and Yashima, were sunk by Russian mines,
not a Japanese scout was in sight, to notice the Russian vessel
engaged in the work which resulted so disastrously to its foes. On
that day, during that operation, no Japanese vessel was visible to the
lookouts at Port Arthur.
For the reasons advanced, I turn at first, and more particularly, to
the Russian naval action for illustration of principles, whether shown
in right or wrong conduct; and here I first name two such principles,
or formulation of maxims, as having been fundamental, and in my
judgment fundamentally erroneous, in the Russian practice. These
are mental conceptions, the first of which has been explicitly stated
as controlling Russian plans, and influencing Russian military ideas;
while the second may be deduced, inferentially, as exercising much
effect. The first, under the title of “Fortress Fleet,” is distinctly
Russian; realized, that is, in Russian theory and practice, though not
without representation in the military thought of other countries.
The second is the well known “Fleet in Being;” a conception
distinctly English in statement and in origin, although, like the first,
it finds reflection in naval circles elsewhere. I shall not at this point
define this conception “Fleet in Being.” I shall attempt to do so later,
by marking its extreme expression; but to do more will require more
space than is expedient to give here, because full definition would
demand the putting forward of various shades of significance, quite
wide in their divergence, which are attributed to the expression
—“Fleet in Being”—by those who range themselves as advocates of
the theory embraced in the phrase.
It is, however, apt here to remark that, in extreme formulation, the
two theories, or principles, summed up in the phrases, “Fortress
Fleet” and “Fleet in Being,” are the antipodes of each other. They
represent naval, or military, thought polarized, so to say. The one
lays all stress on the fortress, making the fleet so far subsidiary as to
have no reason for existence save to help the fortress. The other
discards the fortress altogether, unless possibly as a momentary
refuge for the vessels of the fleet while coaling, repairing, or
refreshing. The one throws national defense for the coast lines upon
fortifications only; the other relies upon the fleet alone for actual
defense. In each case, co-operation between the two arms, fleet and
coast-works, is characterized by a supremacy of one or the other, so
marked as to be exclusive. Co-ordination of the two, which I conceive
to be the proper solution, can scarcely be said to exist. The relation is
that of subjection, rather than of co-ordination. [Here a distinction is
drawn between compromise, which implies concessions and a
middle course between divergent purposes, and the proper method
best expressed by the word adjustment, which signifies
concentration on a single purpose and co-ordination of all means to
that end.]
It is worthy of your consideration whether the word “compromise”
does not really convey to your minds an impression that, when you
come to design a ship of war, you must be prepared to concede
something on every quality, in order that each of the others may have
its share. Granting, and I am not prepared to deny, that in effect each
several quality must yield something, if only in order that its own
effectiveness be ensured, as in the case of the central defense force
just cited, is it of no consequence that you approach the problem in
the spirit of him who divided his force among several passes, rather
than of him who recognizes a central conception to which all else is
to minister? Take the armored cruiser; a fad, I admit, with myself.
She is armored, and she is a cruiser; and what have you got? A ship
to “lie in the line?” as our ancestors used to say. No, and Yes; that is
to say, she may on a pinch, and at a risk which exceeds her powers. A
cruiser? Yes, and No; for, in order to give her armor and armament
which do not fit her for the line, you have given tonnage beyond what
is needed for the speed and coal endurance proper to a cruiser. By
giving this tonnage to armor and armament you have taken it from
other uses; either from increasing her own speed and endurance, or
from providing an additional cruiser. You have in her more cruiser
than you ought to have, and less armored vessel; or else less cruiser
and more armored ship. I do not call this a combination, though it is
undoubtedly a compromise. You have put two things together, but
they remain two, have not become one; and, considering the
tonnage, you have neither as much armored ship, nor as much
cruiser, as you ought to have. I do not say you have a useless ship. I
do say you have not as useful a ship as, for the tonnage, you ought to
have. Whether this opinion of one man is right or wrong, however, is
a very small matter compared with the desirability of officers
generally considering these subjects on proper lines of thought, and
with proper instruments of expression; that is, with correct
principles and correct phraseology.
As an illustration of what I am here saying, the two expressions,
“Fortress Fleet” and “Fleet in Being,” themselves give proof in their
ultimate effect upon Russian practice and principle. Fortress Fleet
was a dominant conception in Russian military and naval thought. I
quote with some reserve, because from a daily newspaper,[102] but as
probably accurate, and certainly characteristic of Russian theory, the
following: “Before his departure from Bizerta for the Suez Canal,
Admiral Wirenius, in command of the Russian squadron, remarked
that the Russian plan was to make Port Arthur and Vladivostok the
two most important arsenals in the empire, each having a fleet of
corresponding strength,”—corresponding, that is, to the fortress,
—“depending upon it as upon a base.” The distribution would be a
division in the face of the probable enemy, Japan, centrally situated,
because the design has reference primarily to the fortress, not to
naval efficiency. The conception is not wholly erroneous; if it were,
the error would have been detected. It has an element of truth, and
therein lies its greatest danger; the danger of half or quarter truths. A
fleet can contribute to the welfare of coast fortresses; especially when
the fortress is in a foreign possession of the nation. On the other
hand, the Fleet in Being theory has also an element of truth, a very
considerable element; and it has been before the naval public,
explicitly, for so long a time that it is impossible it was not known in
Russia. It was known and was appreciated. It had a strong following.
The Russian Naval General Staff clamored for command of the sea;
but in influence upon the government, the responsible director and
formulator of national policy, it did not possess due weight. Not
having been adequately grasped,—whether from neglect, or because
the opposite factor of Fortress Fleet was already in possession of
men’s minds,—it was never able to secure expression in the national
plans. There was compromise, possibly; both things, Fleet in Being
and Fortress Fleet, were attempted; but there was not adjustment.
The fortress throughout reduced the fleet, as fleet, to insignificance
in the national conceptions. What resulted was that at Port Arthur
the country got neither a fortress fleet, for, except the guns mounted
from it, the fleet contributed nothing to the defense of the place; nor
yet a Fleet in Being, for it was never used as such.
It is interesting to observe that this predominant conception of a
fortress fleet reflects national temperament; that is, national
characteristics, national bias. For, for what does Fortress Fleet
stand? For the defensive idea. For what does Fleet in Being stand?
For the offensive. In what kind of warfare has Russia most
conspicuously distinguished herself? In defensive. She has had her
Suvarof, doubtless; but in 1812, and in the Crimea, and now again, in
1904–1905, it is to the defensive that she has inclined. In virtue of
her territorial bulk and vast population, she has, so to say, let the
enemy hammer at her, sure of survival in virtue of mass. Militarily,
Russia as a nation is not enterprising. She has an apathetic bias
towards the defensive. She has not, as a matter of national, or
governmental, decision, so grasped the idea of offense, nor, as a
people, been so gripped by that idea, as to correct the natural
propensity to defense, and to give to defense and offense their proper
adjustment in national and military policy.
In these two well-known expressions, “Fortress Fleet” and “Fleet in
Being,” both current, and comparatively recent, we find ourselves
therefore confronting the two old divisions of warfare,—defensive
and offensive. We may expect these old friends to exhibit their well-
known qualities and limitations in action; but, having recognized
them under their new garb, we will also consider them under it,
speaking not directly of offensive and defensive, but of Fortress Fleet
and Fleet in Being, and endeavoring, first, to trace their influence in
the Russian conduct....
Why then was the fleet stationed in Port Arthur? Because,
expecting the Japanese attack to fall upon Port Arthur, the purpose
of the Russian authorities was not to use the fleet offensively against
the enemy’s navy, but defensively as a fortress fleet; defending the
fortress by defensive action, awaiting attack, not making it. That is,
the function of the fortress was conceived as defensive chiefly, and
not as offensive. Later, I hope to show that the purpose, the raison
d’être, of a coast fortress is in itself offensive; because it exists chiefly
for the purpose of sheltering a fleet, and keeping it fit to act
offensively. For the present, waiving the point, it will be sufficient to
note that the conception of the fleet by the Russians, that it should
act only in defense, led necessarily to imperfect action even in that
respect. The Port Arthur division virtually never acted offensively,

Secdocument 6089

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Secdocument 6089

Uploaded by

Copyright:

Available Formats

Introduction to Linear Regression

Analysis (Wiley Series in Probability

LINEAR REGRESSION ANALYSIS

Introduction to Linear Regression Analysis skillfully blends theory and application

DOUGLAS C. MONTGOMERY, PHD, is Regents Professor of Industrial Engineering

ELIZABETH A. PECK, PHD, is Logistics Modeling Specialist at the Coca-Cola

G. GEOFFREY VINING, PHD, is Professor in the Department of Statistics at Virginia

A Companion website with additional resources is available at

ffirs.indd 1 01/30/2021 3.47.08 PM

Established by WALTER A. SHEWHART and SAMUEL S. WILKS

Editors: David J. Balding, Noel A. C. Cressie, Garrett M. Fitzmaurice,

ffirs.indd 2 01/30/2021 3.47.08 PM

ffirs.indd 3 01/30/2021 3.47.08 PM

Library of Congress Cataloging-in-Publication Data

Cover Design: Wiley

Set in 10/12pt TimesTenRoman by SPi Global, Pondicherry, India

ffirs.indd 4 01/30/2021 3.47.08 PM

2. SIMPLE LINEAR REGRESSION 12

ftoc.indd 5 01/30/2021 3.47.44 PM

2.6 Coefficient of Determination / 35

3. MULTIPLE LINEAR REGRESSION 69

ftoc.indd 6 01/30/2021 3.47.44 PM

4. MODEL ADEQUACY CHECKING 134

5. TRANSFORMATIONS AND WEIGHTING

6. DIAGNOSTICS FOR LEVERAGE AND INFLUENCE 217

ftoc.indd 7 01/30/2021 3.47.45 PM

6.6 Detecting Groups of Influential Observations / 226

7. POLYNOMIAL REGRESSION MODELS 230

ftoc.indd 8 01/30/2021 3.47.45 PM

9.5.4 Principal-Component Regression / 329

10. VARIABLE SELECTION AND MODEL BUILDING 342

11. VALIDATION OF REGRESSION MODELS 388

12. INTRODUCTION TO NONLINEAR REGRESSION 405

ftoc.indd 9 01/30/2021 3.47.45 PM

13. GENERALIZED LINEAR MODELS 440

14. REGRESSION ANALYSIS OF TIME SERIES DATA 495

15. OTHER TOPICS IN THE USE OF REGRESSION ANALYSIS 521

ftoc.indd 10 01/30/2021 3.47.45 PM

15.2 Effect of Measurement Errors in the Regressors / 532

APPENDIX A. STATISTICAL TABLES 561

APPENDIX B. DATA SETS FOR EXERCISES 573

APPENDIX C. SUPPLEMENTAL TECHNICAL MATERIAL 602

APPENDIX D. INTRODUCTION TO SAS 641

ftoc.indd 11 01/30/2021 3.47.45 PM

APPENDIX E. INTRODUCTION TO R TO PERFORM LINEAR

ftoc.indd 12 01/30/2021 3.47.45 PM

fpref.indd 13 01/30/2021 3.47.33 PM

publisher. The ftp site ftp://ftp.wiley.com/public/sci_tech_med/introduction_linear_

CHANGES IN THE SIXTH EDITION

fpref.indd 14 01/30/2021 3.47.33 PM

be of interest to specialists in regression or to instructors teaching a more advanced

USING THE BOOK AS A TEXT

fpref.indd 15 01/30/2021 3.47.33 PM

This book is accompanied by an instructor companion website and a student

flast.indd 16 01/30/2021 3.47.21 PM

1.1 REGRESSION AND MODEL BUILDING

Regression analysis is a statistical technique for investigating and modeling the

c01.indd 1 30-01-2021 14:20:27

c01.indd 2 30-01-2021 14:20:32

N (0, σ2 = 2) μy|x Straight–line

c01.indd 3 30-01-2021 14:20:33

where ŷ is the fitted or estimated value of delivery time corresponding to a delivery

c01.indd 4 30-01-2021 14:20:38