Regression Models - БГЭУ

1 Chapter 1 Regression IntroductionRegression modelsform the core of the discipline of econometrics. Althougheconometricians routinely estimate a wide variety of statistical Models , usingmany different types of data, the vast majority of these are either regressionmodels or close relatives of them. In this chapter, we introduce the concept ofa Regression model , discuss several varieties of them, and introduce the estima-tion method that is most commonly used with Regression Models , namely,leastsquares. This estimation method is derived by using themethod of moments,which is a very general principle of estimation that has many applications most elementary type of Regression model is thesimple linear regressionmodel, which can be expressed by the following equation:yt= 1+ 2Xt+ut.( )The subscripttis used to index theobservationsof asample. The total num-ber of observations, also called thesample size, will be denoted byn.

2 Thus,for a sample of sizen, the subscripttruns from 1 ton. Each observationcomprises an observation on adependent variable, written asytfor observa-tiont, and an observation on a singleexplanatory variable, orindependentvariable, written relation ( ) links the observations on the dependent and the explana-tory variables for each observation in terms of two unknownparameters, 1and 2, and an unobservederror term,ut. Thus, of the five quantities thatappear in ( ), two,ytandXt, are observed, and three, 1, 2, andut, arenot. Three of them,yt,Xt, andut, are specific to observationt, while theother two, the parameters, are common to is a simple example of how a Regression model like ( ) could arise ineconomics. Suppose that the indextis a time index, as the notation value oftcould represent a year, for instance. Thenytcould be house-hold consumption as measured in yeart, andXtcould be measured disposableincome of households in the same year.

3 In that case, ( ) would representwhat in elementary macroeconomics is called aconsumption 1999, Russell Davidson and James G. MacKinnon34 Regression ModelsIf for the moment we ignore the presence of the error terms, 2is themarginalpropensity to consumeout of disposable income, and 1is what is sometimescalledautonomous consumption. As is true of a great many econometric mod-els, the parameters in this example can be seen to have a direct interpretationin terms of economic theory. The variables, income and consumption, do in-deed vary in value from year to year, as the term variables suggests. Incontrast, the parameters reflect aspects of the economy that do not vary, buttake on the same values each purpose of formulating the model ( ) is to try to explain the observedvalues of the dependent variable in terms of those of the explanatory to ( ), for eacht, the value ofytis given by a linear functionofXt, plus what we have called the error term,ut.

4 The linear (strictly speak-ing, affine1) function, which in this case is 1+ 2Xt, is called theregressionfunction. At this stage we should note that, as long as we say nothing aboutthe unobserved quantityut, ( ) does not tell us anything. In fact, we canallow the parameters 1and 2to be quite arbitrary, since, for any given 1and 2, ( ) can always be made to be true by we wish to make sense of the Regression model ( ), then, we must makesome assumptions about the properties of the error termut. Precisely whatthose assumptions are will vary from case to case. In all cases, though, it isassumed thatutis arandom variable. Most commonly, it is assumed that,whatever the value ofXt, the expectation of the random variableutis assumption usually serves toidentifythe unknown parameters 1and 2, in the sense that, under the assumption, ( ) can be true only for specificvalues of those presence of error terms in Regression Models means that the explanationsthese Models provide are at best partial.

5 This would not be so if the errorterms could be directly observed as economic variables, for thenutcould betreated as a further explanatory variable. In that case, ( ) would be arelation linkingyttoXtandutin a completely unambiguous fashion. GivenXtandut,ytwould be completely explained without course, error terms are not observed in the real world. They are includedin Regression Models because we are not able to specify all of the real-worldfactors that determineyt. When we set up our Models withutas a ran-dom variable, what we are really doing is using the mathematical concept ofrandomness to model ourignoranceof the details of economic we are doing when we suppose that the mean of an error term is zero issupposing that the factors determiningytthat we ignore are just as likely tomakeytbigger than it would have been if those factors were absent as theyare to makeytsmaller.

6 Thus we are assuming that, on average, the effectsof the neglected determinants tend to cancel out. This does not mean that1A functiong(x) is said to beaffineif it takes the formg(x) =a+bxfor tworeal 1999, Russell Davidson and James G. Distributions, Densities, and Moments5those effects are necessarily small. The proportion of the variation inytthatis accounted for by the error term will depend on the nature of the data andthe extent of our ignorance. Even if this proportion is large, as it will be insome cases, Regression Models like ( ) can be useful if they allow us to seehowytis related to the variables, likeXt, that we can actually of the literature in econometrics, and therefore much of this book, isconcerned with how to estimate, and test hypotheses about, the parametersof Regression Models . In the case of ( ), these parameters are theconstantterm, orintercept, 1, and theslope coefficient, 2.

7 Although we will beginour discussion of estimation in this chapter, most of it will be postponed untillater chapters. In this chapter, we are primarily concerned with understandingregression Models as statistical Models , rather than with estimating them ortesting hypotheses about the next section, we review some elementary concepts from probabilitytheory, including random variables and their expectations. Many readers willalready be familiar with these concepts. They will be useful in Section ,where we discuss the meaning of Regression Models and some of the formsthat such Models can take. In Section , we review some topics from matrixalgebra and show how multiple Regression Models can be written using matrixnotation. Finally, in Section , we introduce the method of moments andshow how it leads to ordinary least squares as a way of estimating Distributions, Densities, and MomentsThe variables that appear in an econometric model are treated as what statis-ticians callrandom variables.

8 In order to characterize a random variable, wemust first specify the set of all the possible values that the random variablecan take on. The simplest case is ascalar random variable, orscalar of possible values for a scalar may be the real line or a subset of thereal line, such as the set of nonnegative real numbers. It may also be the setof integers or a subset of the set of integers, such as the numbers 1, 2, and a random variable is a collection of possibilities, random variables cannotbe observed as such. What we do observe arerealizationsof random variables,a realization being one value out of the set of possible values. For a scalarrandom variable, each realization is therefore a single real any random variable,probabilitiescan be assigned to subsets of thefull set of possibilities of values forX, in some cases to each point in thatset.

9 Such subsets are calledevents, and their probabilities are assigned by aprobability distribution, according to a few general 1999, Russell Davidson and James G. MacKinnon6 Regression ModelsDiscrete and Continuous Random VariablesThe easiest sort of probability distribution to consider arises whenXis adiscrete random variable, which can take on a finite, or perhaps a countablyinfinite number of values, which we may denote asx1, x2, .. The probabilitydistribution simply assigns probabilities, that is, numbers between 0 and 1,to each of these values, in such a way that the probabilities sum to 1: i=1p(xi) = 1,wherep(xi) is the probability assigned toxi. Any assignment of nonnega-tive probabilities that sum to one automatically respects all the general rulesalluded to the context of econometrics, the most commonly encountered discrete ran-dom variables occur in the context ofbinary data, which can take on thevalues 0 and 1, and in the context ofcount data, which can take on the values0, 1, 2.

10 ; see Chapter possibility is thatXmay be acontinuous random variable, which, forthe case of a scalar , can take on any value in some continuous subset of thereal line, or possibly the whole real line. The dependent variable in a regressionmodel is normally a continuous For a continuous , the probabilitydistribution can be represented by acumulative distribution function, function, which is often denotedF(x), is defined on the real line. Itsvalue is Pr(X x), the probability of the event thatXis equal to or lessthan some valuex. In general, the notation Pr(A) signifies the probabilityassigned to the eventA, a subset of the full set of possibilities. SinceXiscontinuous, it does not really matter whether we define the CDF as Pr(X x)or as Pr(X < x) here, but it is conventional to use the former that, in the preceding paragraph, we usedXto denote a randomvariable andxto denote a realization ofX, that is, a particular value that therandom variableXmay take on.