Introduction to Time Series Analysis. Lecture 3.

1 Introduction to Time Series Analysis. Lecture Bartlett1. Review: Autocovariance, linear processes2. Sample autocorrelation function3. ACF and prediction4. Properties of the ACF1 Mean, Autocovariance, StationarityA time Series {Xt}hasmean function t=E[Xt]andautocovariance function X(t+h,t) =Cov(Xt+h,Xt)=E[(Xt+h t+h)(Xt t)].It isstationaryif both are independent we write X(h) = X(h,0).Theautocorrelation function (ACF)is X(h) = X(h) X(0)=Corr(Xt+h,Xt).2 Linear ProcessesAn important class of stationary time Series :Xt= + Xj= jWt jwhere{Wt} WN(0, 2w)and , jare parameters satisfying Xj= | j|<.

2 3 Linear ProcessesXt= + Xj= jWt jExamples: White noise: 0= 1. MA(1): 0= 1, 1= . AR(1): 0= 1, 1= , 2= 2, ..4 Estimating the ACF: Sample ACFR ecall:Suppose that{Xt}is a stationary time =E[Xt].Itsautocovariance functionis (h) =Cov(Xt+h,Xt)=E[(Xt+h )(Xt )].Itsautocorrelation functionis (h) = (h) (0).5 Estimating the ACF: Sample ACFFor observationsx1,..,xnof a time Series ,thesample meanis x=1nnXt= autocovariance functionis (h) =1nn |h|Xt=1(xt+|h| x)(xt x),for n < h < autocorrelation functionis (h) = (h) (0).6 Estimating the ACF: Sample ACFS ample autocovariance function: (h) =1nn |h|Xt=1(xt+|h| x)(xt x).

3 The sample covariance of(x1,xh+1),..,(xn h,xn), except that we normalize byninstead ofn h, and we subtract the full sample ACF for white Gaussian (hence ) noise 20 15 10 505101520 lines= ACFWe can recognize the sample autocorrelation functions of many non-white(even non-stationary) time Series :Sample ACF:WhitezeroTrendSlow decayPeriodicPeriodicMA(q)Zero for|h|> qAR(p)Decays to zero exponentially9 Sample ACF: Trend0102030405060708090100 4 3 2 10123410 Sample ACF: Trend 60 40 200204060 (why?)11 Sample ACFTime Series :Sample ACF:WhitezeroTrendSlow decayPeriodicPeriodicMA(q)Zero for|h|> qAR(p)Decays to zero exponentially12 Sample ACF: Periodic0102030405060708090100 4 3 2 1012345613 Sample ACF: Periodic0102030405060708090100 4 3 2 10123456signalsignal plus noise14 Sample ACF: Periodic 100 80 60 40 20020406080100 (why?)

4 15 Sample ACFTime Series :Sample ACF:WhitezeroTrendSlow decayPeriodicPeriodicMA(q)Zero for|h|> qAR(p)Decays to zero exponentially16 ACF: MA(1) 10 8 6 4 /(1+ 2)MA(1): Xt = Zt + Zt 117 Sample ACF: MA(1) 10 8 6 4 20246810 ACF18 Sample ACFTime Series :Sample ACF:WhitezeroTrendSlow decayPeriodicPeriodicMA(q)Zero for|h|> qAR(p)Decays to zero exponentially19 ACF: AR(1) 10 8 6 4 |h|AR(1): Xt = Xt 1 + Zt20 Sample ACF: AR(1) 10 8 6 4 20246810 ACF21 Introduction to Time Series Analysis. Lecture Sample autocorrelation function2. ACF and prediction3. Properties of the ACF22 ACF and prediction02468101214161820 3 2 1012white noiseMA(1) 10 8 6 4 20246810 ACF23 ACF of a MA(1) process 505 505lag 0 505 505lag 1 505 505lag 2 505 505lag 324 ACF and least squares predictionBest least squares estimate ofYis EY:mincE(Y c)2=E(Y EY) least squares estimate ofYgivenXis E[Y|X]:minfE(Y f(X))2= minfE E[(Y f(X))2|X] =E E[(Y E[Y|X])2|X] = var[Y|X].

5 Similarly, the best least squares estimate ofXn+hgivenXnisf(Xn) =E[Xn+h|Xn].25 ACF and least squares predictionSuppose thatX= (X1,..,Xn+h)is jointly Gaussian:fX(x) =1(2 )n/2| |1/2exp 12(x ) 1(x ) .Then the joint distribution of(Xn,Xn+h)isN n n+h , 2n n n+h n n+h 2n+h ,and the conditional distribution ofXn+hgivenXnisN n+h+ n+h n(xn n), 2n+h(1 2) .26 ACF and least squares predictionSo for Gaussian and stationary{Xt}, the best estimate ofXn+hgivenXn=xnisf(xn) = + (h)(xn ),and the mean squared error isE(Xn+h f(Xn))2= 2(1 (h)2).Notice: Prediction accuracy improves as| (h)| 1.

6 Predictor is linear:f(x) = (1 (h)) + (h) and least squares linear predictionConsider alinear predictorofXn+hgivenXn=xn. Assume first that{Xt}is stationary with EXn= 0, and predictXn+hwithf(xn) = best linear predictor minimizesE(Xn+h aXn)2=E X2n+h E(2aXn+hXn) +E a2X2n = 2 2a (h) +a2 2,and this is minimized whena= (h), that is,f(xn) = (h) this optimal linear predictor, the mean squared error isE(Xn+h f(Xn))2= 2 2 (h) (h) + (h)2 2= 2(1 (h)2).28 ACF and least squares linear predictionConsider the followinglinear predictorofXn+hgivenXn=xn, when{Xn}is stationary and EXn= :f(xn) =a(xn ) + linear predictor that minimizesE(Xn+h (a(Xn ) +b))2hasa= (h),b= , that is,f(xn) = (h)(Xn ) +.

7 For this optimal linear predictor, the mean squared error isagainE(Xn+h f(Xn))2= 2(1 (h)2).29 Least squares prediction ofXn+hgivenXnf(Xn) = + (h)(Xn ).E(f(Xn) Xn+h)2= 2(1 (h)2). If{Xt}is stationary,fis theoptimal linear predictor. If{Xt}is also Gaussian,fis theoptimal predictor. Linear prediction is optimal for Gaussian time Series . Over all stationary processes with that value of (h)and 2, the optimalmean squared error is maximized by the Gaussian process. Linear prediction needs only second order statistics. Extends to longer histories,(Xn,Xn 1,..).30 Introduction to Time Series Analysis.

8 Lecture Sample autocorrelation function2. ACF and prediction3. Properties of the ACF31 Properties of the autocovariance functionFor the autocovariance function of a stationary time Series {Xt},1. (0) 0,(variance is non-negative)2.| (h)| (0),(from Cauchy-Schwarz)3. (h) = ( h),(from stationarity)4. is positive , any function :Z Rthat satisfies (3) and (4) is theautocovariance of some stationary time of the autocovariance functionA functionf:Z Rispositive semidefiniteif for alln, the matrixFn,with entries(Fn)i,j=f(i j), is positive matrixFn Rn nis positive semidefinite if, for all vectorsa Rn,a Fa see that is psd, consider the variance of(X1.)

9 ,Xn) of the autocovariance functionFor the autocovariance function of a stationary time Series {Xt},1. (0) 0,2.| (h)| (0),3. (h) = ( h),4. is positive , any function :Z Rthat satisfies (3) and (4) is theautocovariance of some stationary time Series (in particular, a Gaussianprocess). : (1) and (2) follow from (4).34 Introduction to Time Series Analysis. Lecture Sample autocorrelation function2. ACF and prediction3. Properties of the ACF35