INTRODUCTION TO LINEAR REGRESSION ANALYSIS

1 INTRODUCTION TO LINEAR REGRESSION ANALYSISWILEY SERIES IN PROBABILITY AND STATISTICSE stablished by WALTER A. SHEWHART and SAMUEL S. WILKSE ditors: David J. Balding, Noel A. C. Cressie, Garrett M. Fitzmaurice,Harvey Goldstein, Iain M. Johnstone, Geert Molenberghs, David W. Scott, Adrian F. M. Smith, Ruey S. Tsay, Sanford WeisbergEditors Emeriti: Vic Barnett, J. Stuart Hunter, Joseph B. Kadane, Jozef L. TeugelsA complete list of the titles in this series appears at the end of this TO LINEAR REGRESSION ANALYSISS ixth EditionDOUGLAS C. MONTGOMERYA rizona State UniversitySchool of Computing, Informatics, and Decision Systems EngineeringTempe, AZELIZABETH A. PECKThe Coca-Cola Company (retired)Atlanta, GAG. GEOFFREY VININGV irginia TechDepartment of StatisticsBlacksburg, VAThis sixth edition first published 2021 2021 John Wiley & Sons, HistoryJohn Wiley and Sons, Inc.

2 (5e, 2012)All rights reserved. No part of this publication may be reproduced, stored in a retrieval system, or trans-mitted, in any form or by any means, electronic, mechanical, photocopying, recording or otherwise, except as permitted by law. Advice on how to obtain permission to reuse material from this title is available at right of Douglas C. Montgomery, Elizabeth A. Peck, and G. Geoffrey Vining to be identified as the authors of this work has been asserted in accordance with Office(s)John Wiley & Sons, Inc., 111 River Street, Hoboken, NJ 07030, USA Editorial Office111 River Street, Hoboken, NJ 07030, USA For details of our global editorial offices, customer services, and more information about Wiley products visit us at also publishes its books in a variety of electronic formats and by print-on-demand. Some content that appears in standard print versions of this book may not be available in other of Liability/Disclaimer of Warranty: While the publisher and authors have used their best efforts in preparing this work, they make no representations or warranties with respect to the accuracy or com-pleteness of the contents of this work and specifically disclaim all warranties, including without limitation any implied warranties of merchantability or fitness for a particular purpose.

3 No warranty may be created or extended by sales representatives, written sales materials or promotional statements for this work. The fact that an organization, website, or product is referred to in this work as a citation and/or potential source of further information does not mean that the publisher and authors endorse the information or services the organization, website, or product may provide or recommendations it may make. This work is sold with the understanding that the publisher is not engaged in rendering professional services. The advice and strategies contained herein may not be suitable for your situation. You should consult with a specialist where appropriate. Further, readers should be aware that websites listed in this work may have changed or disappeared between when this work was written and when it is read.

4 Neither the publisher nor authors shall be liable for any loss of profit or any other commercial damages, including but not limited to special, incidental, consequential, or other of Congress Cataloging-in-Publication DataNames: Montgomery, Douglas C., author. | Peck, Elizabeth A., 1953 author. | Vining, G. Geoffrey, 1954 author. Title: INTRODUCTION to LINEAR REGRESSION ANALYSIS / Douglas C. Montgomery, Elizabeth A. Peck, G. Geoffrey Vining. Description: Fifth edition. | Hoboken, New Jersey : Wiley, [2020] | Series: Wiley series in probability and statistics | Includes bibliographical references and index. Identifiers: LCCN 2020034055 (print) | LCCN 2020034056 (ebook) | ISBN 9781119578727 (hardback) | ISBN 9781119578741 (adobe pdf) | ISBN 9781119578758 (epub) Subjects: LCSH: REGRESSION ANALYSIS . Classification: LCC .M65 2020 (print) | LCC (ebook) | DDC dc23 LC record available at ebook record available at Design: Wiley Cover Images: Abstract marbled background, blue marbling wavy lines oxygen/Getty Images, LINEAR REGRESSION ANALYSIS graph Courtesy of Douglas C.

5 MontgomerySet in 10/12pt TimesTenRoman by SPi Global, Pondicherry, India10 9 8 7 6 5 4 3 2 1vCONTENTSPREFACE xiiiABOUT THE COMPANION WEBSITE xvi 1. INTRODUCTION REGRESSION and Model Building / Data Collection / Uses of REGRESSION / Role of the Computer / 10 2. SIMPLE LINEAR REGRESSION Simple LINEAR REGRESSION Model / Least-Squares Estimation of the Parameters / Estimation of 0 and 1 / Properties of the Least-Squares Estimators and the Fitted REGRESSION Model / Estimation of 2 / Alternate Form of the Model / Hypothesis Testing on the Slope and Intercept / Use of t Tests / Testing Significance of REGRESSION / ANALYSIS of Variance / Interval Estimation in Simple LINEAR REGRESSION / Confidence Intervals on 0, 1, and 2 / Interval Estimation of the Mean Response / Prediction of New Observations / 33vi Coefficient of Determination / A Service Industry Application of REGRESSION / Does Pitching Win Baseball Games?

6 / Using SAS and R for Simple LINEAR REGRESSION / Some Considerations in the Use of REGRESSION / REGRESSION Through the Origin / Estimation by Maximum Likelihood / Case Where the Regressor x is Random / x and y Jointly Distributed / x and y Jointly Normally Distributed: Correlation Model / 54 Problems / 59 3. MULTIPLE LINEAR REGRESSION Multiple REGRESSION Models / Estimation of the Model Parameters / Least-Squares Estimation of the REGRESSION Coefficients / Geometrical Interpretation of Least Squares / Properties of the Least-Squares Estimators / Estimation of 2 / Inadequacy of Scatter Diagrams in Multiple REGRESSION / Maximum-Likelihood Estimation / Hypothesis Testing in Multiple LINEAR REGRESSION / Test for Significance of REGRESSION / Tests on Individual REGRESSION Coefficients and Subsets of Coefficients / Special Case of Orthogonal Columns in X / Testing the General LINEAR Hypothesis / Confidence Intervals in Multiple REGRESSION / Confidence Intervals on the REGRESSION Coefficients / CI Estimation of the Mean Response / Simultaneous Confidence Intervals on REGRESSION

7 Coefficients / Prediction of New Observations / A Multiple REGRESSION Model for the Patient Satisfaction Data / Does Pitching and Defense Win Baseball Games? / Using SAS and R for Basic Multiple LINEAR REGRESSION / Hidden Extrapolation in Multiple REGRESSION / Standardized REGRESSION Coefficients / Multicollinearity / Why Do REGRESSION Coefficients Have the Wrong Sign? / 123 Problems / 125 CONTENTS vii 4. MODEL ADEQUACY CHECKING INTRODUCTION / Residual ANALYSIS / Definition of Residuals / Methods for Scaling Residuals / Residual Plots / Partial REGRESSION and Partial Residual Plots / Using Minitab , SAS, and R for Residual ANALYSIS / Other Residual Plotting and ANALYSIS Methods / PRESS Statistic / Detection and Treatment of Outliers / Lack of Fit of the REGRESSION Model / A Formal Test for Lack of Fit / Estimation of Pure Error from Near Neighbors / 165 Problems / 170 5.

8 TRANSFORMATIONS AND WEIGHTING TO CORRECT MODEL INADEQUACIES INTRODUCTION / Variance-Stabilizing Transformations / Transformations to Linearize the Model / Analytical Methods for Selecting a Transformation / Transformations on y: The Box Cox Method / Transformations on the Regressor Variables / Generalized and Weighted Least Squares / Generalized Least Squares / Weighted Least Squares / Some Practical Issues / REGRESSION Models with Random Effects / Subsampling / The General Situation for a REGRESSION Model with a Single Random Effect / The Importance of the Mixed Model in REGRESSION / 208 Problems / 208 6. DIAGNOSTICS FOR LEVERAGE AND INFLUENCE Importance of detecting Influential Observations / Leverage / Measures of Influence: Cook s D / Measures of Influence: DFFITS and DFBETAS / A Measure of Model Performance / 225viii detecting Groups of Influential Observations / Treatment of Influential Observations / 226 Problems / 227 7.

9 POLYNOMIAL REGRESSION MODELS INTRODUCTION / Polynomial Models in One Variable / Basic Principles / Piecewise Polynomial Fitting (Splines) / Polynomial and Trigonometric Terms / Nonparametric REGRESSION / Kernel REGRESSION / Locally Weighted REGRESSION (Loess) / Final Cautions / Polynomial Models in Two or More Variables / Orthogonal Polynomials / 255 Problems / 261 8. INDICATOR VARIABLES General Concept of Indicator Variables / Comments on the Use of Indicator Variables / Indicator Variables versus REGRESSION on Allocated Codes / Indicator Variables as a Substitute for a Quantitative Regressor / REGRESSION Approach to ANALYSIS of Variance / 283 Problems / 288 9. MULTICOLLINEARITY INTRODUCTION / Sources of Multicollinearity / Effects of Multicollinearity / Multicollinearity Diagnostics / Examination of the Correlation Matrix / Variance Inflation Factors / Eigensystem ANALYSIS of X X / Other Diagnostics / SAS and R Code for Generating Multicollinearity Diagnostics / Methods for Dealing with Multicollinearity / Collecting Additional Data / Model Respecification / Ridge REGRESSION / 312 CONTENTS Principal-Component REGRESSION / Comparison and Evaluation of Biased Estimators / Using SAS to Perform Ridge and Principal-Component REGRESSION / 336 Problems / 33810.

10 VARIABLE SELECTION AND MODEL BUILDING INTRODUCTION / Model-Building Problem / Consequences of Model Misspecification / Criteria for Evaluating Subset REGRESSION Models / Computational Techniques for Variable Selection / All Possible Regressions / Stepwise REGRESSION Methods / Strategy for Variable Selection and Model Building / Case Study: Gorman and Toman Asphalt Data Using SAS / 370 Problems / 38311. VALIDATION OF REGRESSION MODELS INTRODUCTION / Validation Techniques / ANALYSIS of Model Coefficients and Predicted Values / Collecting Fresh Data Confirmation Runs / Data Splitting / Data from Planned Experiments / 401 Problems / 40212. INTRODUCTION TO NONLINEAR REGRESSION LINEAR and Nonlinear REGRESSION Models / LINEAR REGRESSION Models / Nonlinear REGRESSION Models / Origins of Nonlinear Models / Nonlinear Least Squares / Transformation to a LINEAR Model / Parameter Estimation in a Nonlinear System / Linearization / Other Parameter Estimation Methods / Starting Values / Statistical Inference in Nonlinear REGRESSION / Examples of Nonlinear REGRESSION Models / Using SAS and R / 428 Problems / 432x CONTENTS13.