Lecture 1: Stationary Time Series

1 Lecture 1: Stationary Time Series 1 IntroductionIf a random variableXis indexed to time, usually denoted byt, the observations{Xt, t T}iscalled a time Series , whereTis a time index set (for example,T=Z, the integer set).Time Series data are very common in empirical economic studies. Figure 1 plots some frequentlyused variables. The upper left figure plots the quarterly GDP from 1947 to 2001; the upper rightfigure plots the the residuals after linear-detrending the logarithm of GDP; the lower left figureplots the monthly S&P 500 index data from 1990 to 2001; and the lower right figure plots the logdifference of the monthly S&P. As you could see, these four Series display quite different patternsover time. Investigating and modeling these different patterns is an important part of this this course, you will find that many of the techniques (estimation methods, inference proce-dures, etc) you have learned in your general econometrics course are still applicable in time seriesanalysis.

2 However, there are something special of time Series data compared to cross sectional example, when working with cross-sectional data, it usually makes sense to assume that theobservations are independent from each other, however, time Series data are very likely to displaysome degree of dependence over time. More importantly, for time Series data, we could observeonly one history of the realizations of this variable. For example, suppose you obtain a Series ofUS weekly stock index data for the last 50 years. This sample can be said to be large in terms ofsample size, however, it is still one data point, as it is only one of the many possible Autocovariance FunctionsIn modeling finite number of random variables, a covariance matrix is usually computed to sum-marize the dependence between these variables. For a time Series {Xt} t= , we need to modelthe dependence over infinite number of random variables. The autocovariance and autocorrelationfunctions provide us a tool for this 1(Autocovariance function).

3 The autocovariance function of a time Series {Xt}withV ar(Xt)< is defined by X(s, t) =Cov(Xs, Xt) =E[(Xs EXs)(Xt EXt)].Example 1(Moving average process) Let t (0,1), andXt= t+ t 1 Copyright 2002-2006 by Ling Log(GDP)1990199219941996199820002002050010001500 TimeMonthly S&P 500 Index1990199219941996199820002002 S&P 500 Index ReturnsFigure 1: Plots of some economic variables2thenE(Xt) = 0 and X(s, t) =E(XsXt). Lets t. Whens=t, X(t, t) =E(X2t) = ,whent=s+ 1, X(t, t+ 1) =E[( t+ t 1)( t+1+ t)] = ,whent s >1, X(s, t) = Stationarity and Strict StationarityWith autocovariance functions, we can define the covariance stationarity, or weak stationarity. Inthe literature, usually stationarity means weak stationarity, unless otherwise 2(Stationarity or weak stationarity) The time Series {Xt, t Z}(whereZis theinteger set) is said to be Stationary if(I)E(X2t)< t Z.(II)EXt= t Z.(III) X(s, t) = X(s+h, t+h) s, t, h other words, a Stationary time Series {Xt}must have three features: finite variation, constantfirst moment, and that the second moment X(s, t) only depends on (t s) and not depends onsort.

4 In light of the last point, we can rewrite the autocovariance function of a Stationary processas X(h) =Cov(Xt, Xt+h) fort, h , whenXtis Stationary , we must have X(h) = X( h).Whenh= 0, X(0) =Cov(Xt, Xt) is the variance ofXt, so the autocorrelation function for astationary time Series {Xt}is defined to be X(h) = X(h) X(0).Example 1 (continued): In example 1, we see thatE(Xt) = 0,E(X2t) = , and the autoco-variance functions does not depend onsort. Actually we have X(0) = , X(1) = , and x(h) = 0 forh >1. Therefore,{Xt}is a Stationary 2(Random walk) LetStbe a random walkSt= ts=0 XswithS0= 0 andXtisindependent and identically distributed with mean zero and variance 2. Then forh >0, S(t, t+h) =Cov(St, St+h)=Cov t i=1Xi,t+h j=1Xj =V ar(t i=1Xi)sinceCov(Xi, Xj) = 0 fori6=j=t 23In this case, the autocovariance function depends on timet, therefore the random walk processStis not 3(Process with linear trend): Let t iid(0, 2) andXt= t+ (Xt) = t, which depends ont, therefore a process with linear trend is not Stationary processes, there is simple type of process that is widely used in constructingmore complicated 4(White noise): The time Series tis said to be a white noise with mean zero andvariance 2 , written as W N(0, 2 )if and only if thas zero mean and covariance function as (h) ={ 2 ifh= 00 ifh6= 0}It is clear that a white noise process is Stationary .

5 Note that white noise assumption is weakerthan identically independent distributed tell if a process is covariance Stationary , we compute the unconditional first two moments,therefore, processes with conditional heteroskedasticity may still be 5(ARCH model) LetXt= twithE( t) = 0,E( 2t) = 2>0, andE( t s) = 0 fort6=s. Assume the following process for 2t, 2t=c+ 2t 1+utwhere 0< <1 andut W N(0,1).In this example, the conditional variance ofXtis time varying, asEt 1(X2t) =Et 1( 2t) =Et 1(c+ 2t 1+ut) =c+ 2t , the unconditional variance ofXtis constant, which is 2=c/(1 ). Therefore, thisprocess is still 3(Strict stationarity) The time Series {Xt, t Z}is said to be strict Stationary if thejoint distribution of(Xt1, Xt2, .. , Xtk)is the same as that of(Xt1+h, Xt2+h, .. , Xtk+h).In other words, strict stationarity means that the joint distribution only depends on the dif-ference h, not the time (t1, .. , tk).Remarks: First note that finite variance is not assumed in the definition of strong stationarity,therefore, strict stationarity does not necessarily imply weak stationarity.

6 For example, processeslike Cauchy is strictly Stationary but not weak Stationary . Second, a nonlinear function ofa strict Stationary variable is still strictly Stationary , but this is not true for weak Stationary . Forexample, the square of a covariance Stationary process may not have finite variance. Finally, weak401002003009001000110012001300140015 00S&P 500 index in year 19990100200300 500 returns in year 1999010020030090010001100120013001400150 0S&P 500 index in year 20010100200300 500 returns in year 2001 Figure 2: Plots of S&P index and returns in year 1999 and 2001stationarity usually does not imply strict stationarity as higher moments of the process may dependon timet. However, if process{Xt}is a Gaussian time Series , which means that the distributionfunctions of{Xt}are all multivariate Gaussian, the joint density offXt,Xt+j1,..,Xt+jk(xt, xt+j1, .. , xt+jk)is Gaussian for anyj1, j2, .. , jk, weak Stationary also implies strict Stationary .

7 This is because amultivariate Gaussian distribution is fully characterized by its first two example, a white noise is Stationary but may not be strict Stationary , but a Gaussianwhite noise is strict Stationary . Also, general white noise only implies uncorrelation while Gaussianwhite noise also implies independence. Because if a process is Gaussian, uncorrelation impliesindependence. Therefore, a Gaussian white noise is (0, 2). Stationary and nonstationary processes are very different in their properties, and they requiredifferent inference procedures. We will discuss this in much details through this course. At thispoint, note that a simple and useful method to tell if a process is Stationary in empirical studies isto plot the data. Loosely speaking, if a Series does not seem to have a constant mean or variance,then very likely, it is not Stationary . For example, Figure 2 plots the daily S&P 500 index in year1999 and 2001. The upper left figure plots the index in 1999, upper right figure plots the returnsin 1999, lower left figure plots the index in 2001, and lower right figure plots the returns in that the index level are very different in 1999 and 2001.

8 In year 1999, it is wandering ata higher level and the market rises. In year 2001, the level is much lower and the market comparison, we did not see much difference in the returns in year 1999 and 2001 (althoughthe returns in 2001 seem to have thicker tails). Actually, only judging from the return data, itis very hard to tell which figure plots the market in booms, and which figure plots the market incrashes. Therefore, people usually treat stock price data as nonstationary and stock return data ErgodicityRecall that Kolmogorov s law of large number (LLN) tells that ifXi ( , 2) fori= 1, .. , n,then we have the following limit for the ensemble average Xn=n 1n i=1Xi .In time Series , we have time Series average, not ensemble average. To explain the differencesbetween ensemble average and time Series average, consider the following experiment. Suppose wewant to track the movements of some particles and draw inference about their expected position(suppose that these particles move on the real line).

9 If we have a group of particles (group sizen),then we could track down the position of each particle and plot a distribution of their mean of this sample is called ensemble average. If all these particles are , LLN tells thatthis average converges to its expectation asn . However, as we remarked earlier, with timeseries observations, we only have one history. That means, in this experiment, we only have oneparticle. Then instead of collectingnparticles, we can only track this single particle and record itsposition, sayxt, fort= 1,2, .. , T. The mean we computed by averaging over time,T 1 Tt=1xtis called time Series the time Series average converges to the same limit as the ensemble average? The answeris yes ifXtis Stationary and ergodic. IfXtis Stationary and ergodic withE(Xt) = , then thetime Series average has the same limit as ensemble average, XT=T 1T t=1Xt .This result is given as ergodic theorem, and we will discuss it later in our Lecture 4 on asymp-totic theory.

10 Note that this result require both stationarity and ergodicity. We have explainedstationarity and we see that stationarity allows time Series dependence. Ergodicity requires aver-age asymptotic independence . Note that stationarity itself does not guarantee ergodicity (page 47in Hamilton and Lecture 4).Readings:Hamilton, Ch. and Davis, Page 1-29 Hayashi, Page 97-1026