Lecture 13 Time Series: Stationarity, AR(p) & MA(q)

1 RS EC2 - Lecture 1311 Lecture 13 Time Series: Stationarity, AR(p) & MA(q) Time Series: Introduction In the early 1970 s, it was discovered that simple time series models performed better than the complicated multivarate, then popular, 1960s macro models (FRB-MIT-Penn). See, Nelson (1972). The tools? Simple univariate (ARIMA) models, popularized by the textbook of Box & Jenkins (1970).Q: What is a time series? A time series ytis a process observed in sequence over time, t = 1,.., T Yt={y1, y2 ,y3, .., yT} Main Issue with Time Series: the sequential nature of Yt, we expect yt& yt-1to be dependent. Depending on assumptions, classical results (based on LLN & CLT) may not be EC2 - Lecture 132 Usually, time series models are separated into two categories: Univariate (yt R, it is scalar) primary model: Autoregressions (ARs).

2 Multivariate (yt Rm, it vector-valued). primary model: Vectot autoregressions (VARs). In time series, {.., y1, y2 ,y3, .., yT} are jointly RV. We want to model the conditional expectation: E[yt| Ft-1]where Ft-1= {yt-1, yt-2 ,yt-3, ..} is the past history of the Series: Introduction Two popular models for E[yt|Ft-1]: An autoregressive (AR) process models E[yt|Ft-1] with lagged dependent variables. A moving average (MA) process models E[yt|Ft-1] with lagged errors. Usually, E[yt|Ft-1] has been modeled as a linear process. But, recently, non-linearities have become more common. In general, we assume the error term, t, is uncorrelated with anything, with mean 0 and constant variance, 2. We call a process like this a white noise (WN)process. We denote it as t~ WN(0, 2)Time Series: IntroductionRS EC2 - Lecture 133 We want to select an appropriate time series model to forecast yt.

3 In this class, the choices are AR(p), MA(q) or ARMA(p, q). Steps for forecasting:(1) Identify the appropriate model. That is, determine p, q.(2) Estimate the model.(3) Test the model.(4) Forecast. In this Lecture , we go over the statistical theory (stationarity, ergodicity and MDS CLT), the main models (AR, MA & ARMA) and tools that will help us describe and identify a proper modelTime Series: IntroductionCLM Revisited: Time Series With autocorrelated data, we get dependent observations. Recall, t= t-1+ utThe independence assumption (A2 ) is violated. The LLN and the CLT cannot be easily applied, in this context. We need new tools and will introduce the concepts of stationarity and ergodicity. The ergodic theorem will give us a counterpart to the get asymptotic distributions, we also need a CLT for dependent variables, using the concept of mixing and stationarity.

4 Or we can rely on the martingale CLT. RS EC2 - Lecture 134 Consider the joint probability distribution of the collection of RVs:Then, we say that a process is ),..,(),..,(221121nnntttttttttzZzZzZPzzz F 1storder stationaryif ktanyforzFzFktt,)()(111 Nth-order stationaryifkttanyforzzFzzFktkttt,,),(), (212121 kttanyforzzFzzFnktktttnn,,)..()..(111 Definition. A process is strongly(strictly) stationaryif it is a Nth-order stationary process for any stationaryifTime Series Stationarity 2221212122221221121)(),()()])([(),()()() ()()()(tttttttttttttttttttttttttttZZEZZC ovdzzfZZEZV ardzzfZZE Time Series Moments The moments describe a distribution. We calculate the moments as usual: Note: (t1-t2) is called the auto- covariance function think of it as a function of k = t1 t2.

5 (0) is the variance. Stationarity requires all these moments to be independent of time. If the moments are time dependent, we say the series is EC2 - Lecture 135 For strictly stationary process:22and ttbecause kttkttzFzF1111)()(provided thatkktktttktktttktttktttttkttktktttzzzz zzFzzF ),(),(),( then, and let ),(),(),cov(),cov(),(),(2121212121212121 The correlation between any two RVs depends on the time difference. )(,)(2ttZEZEThen,Time Series Moments A process is said to be N-order weakly stationaryif all its joint moments up to orderNexist and are time invariant. A covariance stationaryprocess (or 2nd order weakly stationary ) has:- constant mean- constant variance- covariance function depends on time difference between is,Ztis covariance stationary if:Time Series Weak Stationarity )()()])([(),(constant)(constant)(2121221 121ttfttZZEZZCovZVarZEtttttttt RS EC2 - Lecture 136 Examples: For all assume t~ WN(0, 2)1) yt= yt-1+ tE[yt] = 0(assuming 1)Var[yt] = 2/(1- 2)(assuming | |<11)E[ytyt-1] = E[yt-12] stationary , not time dependent2) yt= + yt-1+ t yt= t + j=0 to t-1 t-j+ y0E[yt] = t + y0 Var[yt] = j=0 to t-1 2 = 2t non- stationary , time dependentTime Series Weak Stationarity stationary Series 13.

6 ~ Examples:% Changes in USD/GBP (1978:I-2011:IV) 9 17 25 33 41 49 57 65 73 81 89 97 105 113 121 129 Time% changeRS EC2 - Lecture 137 Non- stationary SeriesExamples:driftRW with ~12211ttttttttyyWNyyty US CPI Prices (1978:I-2011:IV) 10192837465564738291100109118127136US CPI We want to allow as much dependence as the LLN allows us to do it. But, stationarity is not enough, as the following example shows: Example: Let {Ut} be a sequence of uniformly distributed on [0, 1] and let Z be N(0,1) independent of {Ut}.Define Yt= Z+Ut . Then Ytis stationary (why?), butThe problem is that there is too much dependencein the sequence {Yt} (because of Z). In fact the correlation between Y1and Ytis always positive for any value of )(11 pntnonttnZYYEYnYTime Series Ergodicity RS EC2 - Lecture 138 We want to estimate the mean of the process {Zt}, (Zt).

7 But, we need to distinguishing between ensemble averageand time average:- Ensemble Average- Time Series AverageQ: Which estimator is the most appropriate? A: Ensemble Average. But, it is impossible to calculate. We only observe one Zt. Q: Under which circumstances we can use the time average (only one realization of {Zt})? Is the time average an unbiased and consistent estimator of the mean? The Ergodic Theoremgives us the 1nZzntt 1 Time Series Ergodicity of the Mean Recall the sufficient conditions for consistency of an estimator: the estimator is asymptotically unbiased and its variance asymptotically collapses to Q: Is the time average is asymptotically unbiased? Yes. tttnZEnzE 1)(1)(2. Q: Is the variance going to zero as T grows? It depends. )]( )()[( )(),cov(1)var(0)2()1(2101110201212011201 12 nnnnntntttntnsstntnsstnnnZZnzTime Series Ergodicity of the Mean RS EC2 - Lecture 1390)1(lim)var(lim)1()()var(?

8 01)1(020 kknnnnkkkknknznknknnz If the Ztwere uncorrelated, the variance of the time average would be O(n-1). Since independent random variables are necessarily uncorrelated (but not vice versa), we have just recovered a form of the LLN for independent data. Q: How can we make the remaining part, the sum over the upper triangle of the covariance matrix, go to zero as well? A: We need to impose conditions on k. Conditions weaker than "they are all zero;" but, strong enough to exclude the sequence of identical copies. Time Series Ergodicity of the Mean Definition: A covariance - stationary process is ergodicfor the mean if )(limtZEzpErgodicity Theorem: Then, a sufficient condition for ergodicity for the mean is k as 0k 111111111kknttnkknttnkknt We use two inequalities to put upper bounds on the variance of the time average: Covariances can be negative, so we upper-bound the sum of the actual covariances by the sum of their magnitudes.

9 Then, we extend the inner sum so it covers alllags. This might of course be infinite (sequence-of-identical-copies). Time Series Ergodicity of the Mean RS EC2 - Lecture 1310 Ergodicity under Gaussian DistributionIf {Zt}is a stationary Gaussian process,is sufficient to ensure ergodicity for all : Recall that only the first two moments are needed to describe the normal distribution. kk A sufficient condition to ensure ergodicity for second moments is:A process which is ergodic in the first and second moments is usually referred as ergodic in the wide sense. kk Time Series Ergodicity of 2ndMoments We state two essential theorems to the analysis of stationary time series. Difficult to prove in general. Theorem IIf ytis strictly stationary and ergodic and xt= f(yt, yt-1, yt-2.)

10 Is a RV, then xtis strictly stationary and II (Ergodic Theorem)If ytis strictly stationary and ergodic and E[yt] < ; then as T ; These results allow us to consistently estimate parameters using time-series Series Ergodicity Theorems ][1tpttyEyT RS EC2 - Lecture 1311 Definition: tis a martingale difference sequence (MDS) ifE[ t| Ft-1]=0. Regression errors are naturally a MDS. Some time-series processes may be a MDS as a consequence of optimizing behaviour. For example, most asset pricing models imply that asset returns should be the sum of a constant plus a MDS. Useful property: tis uncorrelated with any function of the lagged information Ft-1. Then, for k> 0 E[yt-k t] = Series - MDST heorem(MDS CLT)If utis a strictly stationary and ergodic MDS and E(utut ) = < ; then as T ; Application: Let xt ={yt-1, yt-2.}