MATH 3P82 REGRESSION ANALYSIS Lecture Notes

1 MATH 3P82 REGRESSION ANALYSISL ecture Notesc Jan Vrbik23 Contents1 PREVIEW52 USINGMAPLE7 Basics .. 9 Procedures .. 10 MatrixAlgebra .. 10 Otherusefulcommands: .. 113 SIMPLE REGRESSION13 MaximumLikelihoodMethod .. 15 ConfidenceIntervals .. 17 REGRESSION coefficients .. 18 Residual 19 Hypotheses 20 ModelAdequacy(Lack-of-FitTest).. 20 Weighted REGRESSION .. 22 Correlation .. 24 Large -Sample 26 Confidence interval for the correlation 274 MULTIVARIATE (LINEAR) REGRESSION29 MultivariateNormalDistribution.

2 29 Partial correlation 30 Multiple REGRESSION - Main 33 Weighted-case 35 SearchingforOptimalModel .. 37 CoefficientofCorrelation(Determination) .. 384 Polynomial REGRESSION .. 39 Dummy(Indicator)Variables .. 425 NONLINEARREGRESSION436 ROBUSTREGRESSION47 Laplace distribution .. 47 CauchyCase .. 53 YuleModel .. 555 Chapter 1 PREVIEWR egression is a procedure which selects, from a certain class of functions, the onewhich bestfits a given set of empirical data (usually presented as a table ofxandyvalues with, inevitably, some random component).

3 The independent variablexisusually called theregressor(there may be one or more of these), the dependent variableyis The random components (calledresiduals)are usually assumednormallydistributed, with the same and independent of class from which the functions are selected (themodel) is usually one ofthe following types:1. a linear function ofx( +bx)- simple (univariate) linear REGRESSION ,2. a linear function ofx1,x2,..xk- multiple (multivariate) linear REGRESSION ,3. a polynomial function ofx- polynomial REGRESSION ,4.

4 Any other type of function, with one or more parameters ( )-nonlinear coefficients (parameters) of these models are calledregression coeffi-cients(parameters). Our main task is going to be tofind goodestimatorsofthe REGRESSION coefficients (they should have correct expected values and variancesas small as possible), to be used forpredictingvalues ofywhen new observationsare of the related issues are:1. How do know (can we test) whether the relationship (betweenyandx)is truly linear? What if it is not (we have switch to either polynomial ornonlinear model).

5 2. Similarly, are the residuals truly normal and independent of each other? Howdo wefix the procedure if the answer is Even when they are normal and independent, what if their variance changeswithx(here, we have to do the so calledweighted REGRESSION ).4. Even when all the assumptions are properly met: In the multivariate casewith many independent variables, do we really need them all to make a goodprediction abouty?And, if itispossible to reduce them (usually substan-tially) to a smaller subset, how do we do it ( selecting thebestfive,say).

6 ?67 Chapter 2 USING MAPLEB asicsTyping anexpression(following Maple s>prompt) results in evaluating the expression contains only integers (no decimal point), one gets the exact(rational) answer, as soon as at least one number in the expression is real (with adecimal point), the result is real (rounded offto 10 significant digits). The symbols ,/and facilitate multiplication, division and exponentiation, respectively. Notethat each line of your input has to end with a semicolon:>4 5 3/(5 + 2) + 2 ( 3) ;110356 The result of any computation can be stored under a name (which you make up,rather arbitrarily), and used in any subsequent expression.

7 Maple then remembersthe value, until the end of your session, or till you deliberately replace it with anew value. Note that this (giving a name to a result) is achieved by typing thename, followed by a colon and the equal sign (a group of two symbols, representinga single operation), followed by the actual expression to be stored:>a:= ( +4) (2 6) + 2/3 4/5;a:= 333>a/7+9; 238>a:= 14/6;a:=73;>a/7+9;a:=283;(from now on, we will omit the>prompt from our examples, showing onlywhatwehave to type).Maple can also handle the usual functions such assin, cos, tan, arcsin,arccos, arctan, exp, ln, sqrt, etc.

8 All angles are always measured in (3.);sqrt(8) ;.14112 000812 2We can alsodefineour own functions by:f:=x >x 2;8f:=x x2f(3);9wherefis an arbitrary and LoopsMaple can store, under a single name, a wholelistof values, thus:a:= [3/2,5,sqrt(3),7] ;a:= [32,5, 3,7]The individual elements of a list can be referenced by indexing (and used incomputing another expression):a[2] 4;20 One can add elements of a list by the followingcommand(as Maple callsthem):sum( a[i] , i = ) ;272+ 3 One can convert the last answer to its decimal form by:evalf(%).

9 081 Note that the%symbol always refers to the previous tosum, one can also computeproductof elements of a subtract say3from each element of the lista,redefiningacorrespondingly,canbeac hievedby:forifrom1to4doa[i]:=a[i] 3end do:Note that terminating a statement by:instead of the usual;will preventMaple from printing the four results computed in the process (we may not needto see them individually). Also note that, upon completion of this statement,iwill have the value of5(any informationihad contained previously will have beendestroyed)!

10 We can easily verify that the individual elements of ouralist have been updatedaccordingly:a[2] ;2We may also create a list using the following approach:b:= [seq(2 i, i= )] ;b:= [2,4,8,16,32,64 ] ;9 Variables and PolynomialsIf a symbol, such as for examplex,has not been assigned a specificvalue,Mapleconsiders it a variable. We may then defineato be apolynomialinx,thus:a:= 3 2 x+4 x 2 ;a:= 3 2x+4x2A polynomial can be differentiateddiff(a, x); 2+8xintegrated from, say,0to3int(a, x= ) ;36or plotted, for a certain range ofxvaluesplot(a, x= ) ;We can also evaluate it, substituting a specificnumberforx(there are actuallytwo ways of doing this):subs(x=3,a);eval(a, x=3).