Introduction to Building a Linear Regression Model

1 Introduction to Building a Linear Regression ModelLeslie A. ChristensenThe Goodyear Tire & Rubber Company, Akron OhioAbstractThis paper will explain the steps necessary to builda Linear Regression Model using the SAS System . The process will start with testing the assumptionsrequired for Linear modeling and end with testing thefit of a Linear Model . This paper is intended foranalysts who have limited exposure to buildinglinear models. This paper uses the REG, GLM,CORR, UNIVARIATE, and PLOT following topics will be covered in this paper:1. assumptions regarding Linear regression2. examing data prior to modeling3. creating the model4.

2 Testing for assumption validation5. writing the equation6. testing for multicollinearity7. testing for auto correlation8. testing for effects of outliers9. testing the fit10 modeling without Linear Model has the form Y = b0 + b1X + . Theconstant b0 is called the intercept and the coefficientb1 is the parameter estimate for the variable X. The is the error term. is the residual that can not beexplained by the variables in the Model . Most of theassumptions and diagnostics of Linear regressionfocus on the assumptions of . The followingassumptions must hold when Building a linearregression The dependent variable must be continuous.

3 Ifyou are trying to predict a categorical variable, Linear Regression is not the correct method. Youcan investigate discrim, logistic, or some othercategorical The data you are modeling meets the "iid"criterion. That means the error terms, , are:a. independent from one another andb. identically assumption 2a does not hold, you need toinvestigate time series or some other type ofmethod. If assumption 2b does not hold, youneed to investigate methods that do not assumenormality such as non-parametric The error term is normally distributed with amean of zero and a standard deviation of 2,N(0, 2).Although not an actual assumption of linearregression, it is good practice to ensure the data youare modeling came from a random sample or someother sampling frame that will be valid for theconclusions you wish to make based on DatasetWe will use the dataset that isprovided with the SAS System for PCs Thedataset has the following variables: PRICE, BATHS,BEDROOMS, SQFEET and STYLE.

4 STYLE is acategorical variable with four levels. We will Examination Prior toModelingBefore you begin modeling, it isrecommended that you plot yourdata. By examining these initialplots, you can quickly assesswhether the data have linearrelationships or interactions code below will produce threeplots. PROC PLOT DATA=HOUSES; PLOT PRICE*(BATHS BEDROOMS SQFEET); RUN;An X variable ( SQFEET) thathas a Linear relationship with Y(PRICE) will produce a plot thatresembles a straight line. (NoteFigure 1.) Here are someexceptions you may come across inyour own your data look like Figure 2,consider transforming the X variablein your modeling to log10X or 1 Figure 2 Figure 3 Figure 42If your data look like Figure 3, consider transformingthe X variable in your modeling to X2 or exp(X).

5 If your data look like Figure 4, consider transformingthe X variable in your modeling to 1/X or exp(-X)This SAS code can be used to visually inspect forinteractions between two variables. PROC PLOT DATA=HOUSES; PLOT BATHS*BEDROOMS; RUN;Additionally, running correlations among theindependent variables is helpful. These correlationswill help prevent multicollinearity problems later. PROC CORR DATA=HOUSES; VAR BATHS BEDROOMS SQFEET; RUN;In our example, the output of the correlationanalysis will contain the following. Correlation Analysis 3 'VAR' Variables: BATHS BEDROOMSSQFEETP earson Correlation Coefficients / Prob > |R|under Ho: Rho=0 / N = 15 BATHS BEDROOMS SQFEET BATHS BEDROOMS SQFEET In the above example, the correlation coefficentsare in bold.

6 The correlation of betweenBATHS and BEDROOMS indicates that thesevariables are highly correlated. A decision shouldbe made to include only one of them in the Model . You might also argue that is high. For ourexample we will keep it in the the ModelAs you read, learn and become experienced withlinear Regression you will find there is no one correctway to build a Model . The method suggested hereis to help you better understand the decisionsrequired without having to learn a lot of SASprogramming. The REG procedure can be used to build and testthe assumptions of the data we propose to , PROC REG has some limitations as tohow the variables in your Model must be set up.

7 REG can not handle interactions such asBEDROOMS*SQFEET or categorical variables withmore than two levels. As such you need to use aDATA step to manipulate your variables. Let's sayyou have two continous variables (BEDROOMS andSQFEET) and a categorical variable with four levels(STYLE) and you want all of the variables plus aninteraction term in the first pass of the Model . Youwould have to have a DATA step to prepare yourdata as such: DATA HOUSES; SET HOUSES; BEDSQFT = BEDROOMS*SQFEET; IF STYLE='CONDO' THEN DO; S1=0; S2=0; S3=0; END; ELSE IF STYLE='RANCH' THEN DO; S1=1; S2=0; S3=0; END; ELSE IF STYLE='SPLIT' THEN DO; S1=0; S2=1; S3=0; END; ELSE DO; S1=0; S2=0; S3=1; END; RUN;When creating a categorical term in your Model , youwill need to create dummy variables for "the numberof levels minus 1".

8 That is, if you have three levels,you will need to create two dummy variables and the variables are correctly prepared for REGwe can run the procedure to get an initial look atour Model . PROC REG DATA=HOUSES; Model PRICE = BEDROOMS SQFEET S1 S2 S3 BEDSQFT ; RUN;The GLM procedure can also be used to create alinear Regression Model . The GLM procedure is thesafer procedure to use for your final modelingbecause it does not assume your data are balanced. That is with respect to categorical variables, it doesnot assume you have equal sample sizes for eachlevel of each category. GLM also allows you towrite interaction terms and categorical variables withmore than two levels directly into the Model statement.

9 (These categorical variables can evenbe character variables.) Thus using GLM eliminatessome DATA step , the SAS system does not provide thesame statistics in REG and GLM. Thus you maywant to test some basic assumptions with REG andthen move on to using GLM for final modeling. Using GLM we can run the Model as:3 PROC GLM DATA=HOUSES; CLASS STYLE; Model PRICE = BEDROOMS SQFEET STYLE BEDROOMS*SQFEET; RUN;The output from this initial modeling attempt willcontain the following statistics:General Linear Models ProcedureDependent Variable: PRICE Asking price Sum of Mean F Pr>Source DF Squares Square Value FModel 6 7895558723 1315926453 8 25635276 3204409 CorrectedTotal 14 7921194000 R-Square Root MSE PRICE Mean Type III Mean F Pr>Source DF SS Square Value FBEDROOMS 1 245191 245191 1 823358866 823358866 3 5982918 1994306 *SQFEET 1 712995 712995 Building a Model you may wonder whichstatistic tells whether the Model is good.

10 There is noone correct answer. Here are some approaches ofstatistics that are found in both REG and and Adj-RsqYou want these numbers to be as high aspossible. If your Model has a lot of variables,use Adj-Rsq because a Model with morevariables will have a higher R-square than a similar Model with fewer variables. Adj-Rsqtakes the number of variables in your Model intoaccount. An R-square or or higher isgenerally accepted as MSEYou want this number to be small compared toother models. The value of Root MSE will bedependent on the values of the Y variable youare modeling. Thus, you can only compareRoot MSE against other models that aremodeling the same dependent III SS Pr>FAs a guideline, you want the value for each ofthe variables in your Model to have a Type IIISS p-value of or less.