AR, MA and ARMA models - Hedibert

1 StationarityACFL jung-BoxtestWhite noiseAR modelsExamplePACFAIC/BICF orecastingMA modelsSummaryAR, MA and ARMA models1 Stationarity2 ACF3 Ljung-Box test4 White noise5AR models6 Example7 PACF8 AIC/BIC9 Forecasting10MA models11 Summary1 / 40 StationarityACFL jung-BoxtestWhite noiseAR modelsExamplePACFAIC/BICF orecastingMA modelsSummaryLinear Time Series Analysisand Its Applications1 For basic concepts of linear time series analysis see Box, Jenkins, and Reinsel (1994, Chapters 2-3), and Brockwell and Davis (1996, Chapters 1-3)The theories of linear time series discussed include stationarity dynamic dependence autocorrelation function modeling forecasting1 Tsay (2010), Chapter / 40 StationarityACFL jung-BoxtestWhite noiseAR modelsExamplePACFAIC/BICF orecastingMA modelsSummaryThe econometric models introduced include(a) simple autoregressive models ,(b) simple moving-average models ,(b) mixed autoregressive moving-average models ,(c) seasonal models ,(d) unit-root nonstationarity,(e) regression models with time series errors, and(f) fractionally differenced models for / 40 StationarityACFL jung-BoxtestWhite noiseAR modelsExamplePACFAIC/BICF orecastingMA modelsSummaryStrict stationarityThe foundation of time series analysis is time series{rt}is said to bestrictly stationaryif the jointdistribution of (rt1.)

2 ,rtk) is identical to that of(rt1+t,..,rtk+t) for allt, wherekis an arbitrary positiveinteger and (t1,..,tk) is a collection ofkpositive joint distribution of (rt1,..,rtk) is invariant under is a very strong condition that is hard to / 40 StationarityACFL jung-BoxtestWhite noiseAR modelsExamplePACFAIC/BICF orecastingMA modelsSummaryWeak stationarityA time series{rt}isweakly stationaryif both the mean ofrtand the covariance betweenrtandrt lare time invariant,wherelis an arbitrary specifically,{rt}is weakly stationary if(a)E(rt) = , which is a constant, and(b)Cov(rt,rt l) = l, which only depends practice, suppose that we have observedTdata points{rt|t= 1,..,T}.

3 The weak stationarity implies that thetime plot of the data would show that theTvalues fluctuatewith constant variation around a fixed applications, weak stationarity enables one to makeinference concerning future observations ( , prediction).5 / 40 StationarityACFL jung-BoxtestWhite noiseAR modelsExamplePACFAIC/BICF orecastingMA modelsSummaryPropertiesThe covariance =Cov(rt,rt l)is called the lag-lautocovariance has two important properties:(a) 0=V ar(rt), and (b) l= second property holds becauseCov(rt,rt ( l)) =Cov(rt ( l),rt)=Cov(rt+l,rt)=Cov(rt1,rt1 l),wheret1=t+ / 40 StationarityACFL jung-BoxtestWhite noiseAR modelsExamplePACFAIC/BICF orecastingMA modelsSummaryAutocorrelation functionThe autocorrelation function of laglis l=Cov(rt,rt l) V ar(rt)V ar(rt l)=Cov(rt,rt l)V ar(rt)= l 0where the propertyV ar(rt) =V ar(rt l) for a weaklystationary series is general, the lag-lsample autocorrelation ofrtis defined as l= Tt=l+1(rt r)(rt l r) Tt=1(rt r)

4 20 l < T / 40 StationarityACFL jung-BoxtestWhite noiseAR modelsExamplePACFAIC/BICF orecastingMA modelsSummaryPortmanteau testBox and Pierce (1970) propose the Portmanteau statisticQ (m) =Tm l=1 2las a test statistic for the null hypothesisH0: 1= = m= 0against the alternative hypothesisHa: i6= 0 for somei {1,..,m}.Under the assumption that{rt}is an iid sequence withcertain moment conditions,Q (m) is asymptotically / 40 StationarityACFL jung-BoxtestWhite noiseAR modelsExamplePACFAIC/BICF orecastingMA modelsSummaryLjung and Box (1978)Ljung and Box (1978) modify theQ (m) statistic as belowto increase the power of the test in finite samples,Q(m) =T(T+ 2)m l=1 2lT decision rule is to rejectH0ifQ(m)> q2 , whereq2 denotes the 100(1 )thpercentile of a / 40 StationarityACFL jung-BoxtestWhite noiseAR modelsExamplePACFAIC/BICF orecastingMA modelsSummary# Load datada = (" ",header=TRUE)# IBM simple returns and squared returnssibm=da[,2]sibm2 = sibm^2# ACFpar(mfrow=c(1,2))acf(sibm)

5 Acf(sibm2)# Ljung-Box statistic Q(30) (sibm,lag=30,type="Ljung") (sibm2,lag=30,type="Ljung")> (sibm,lag=30,type="Ljung")Box-Ljung testdata: sibmX-squared = , df = 30, p-value = > (sibm2,lag=30,type="Ljung")Box-Ljung testdata: sibm2X-squared = , df = 30, p-value < / 40 StationarityACFL jung-BoxtestWhite noiseAR modelsExamplePACFAIC/BICF orecastingMA modelsSummaryWhite noiseA time seriesrtis called a white noise if{rt}is a sequenceof independent and identically distributed random variableswith finite mean and the ACFs are normally distributed with mean zero and variance 2,the series is called aGaussian white / 40 StationarityACFL jung-BoxtestWhite noiseAR modelsExamplePACFAIC/BICF orecastingMA modelsSummaryLinear Time SeriesA time seriesrtis said to be linear if it can be written asrt= + i=0 iat i,where is the mean ofrt, 0= 1.

6 And{at}is white the new information at timetof the time seriesand is often referred to as theinnovationorshockat weakly stationary , we can obtain its mean andvariance easily by using the independence of{at}asE(rt) = , V(rt) = 2a i=0 2i,where 2ais the variance / 40 StationarityACFL jung-BoxtestWhite noiseAR modelsExamplePACFAIC/BICF orecastingMA modelsSummaryThe lag-laucovariance ofrtis l=Cov(rt,rt l)=E ( i=0 iat i) j=0 jat l j =E i,j=0 i jat iat l j = i=0 j+l jE(a2t l j) = 2a j=0 j j+l,so l= l 0= i=0 i i+l1 + i=1 2i13 / 40 StationarityACFL jung-BoxtestWhite noiseAR modelsExamplePACFAIC/BICF orecastingMA modelsSummaryAR(1)Linear time series models are econometric and statisticalmodels used to describe the pattern of the weights instance, an stationary AR(1) model can be written asrt = 1(rt 1 ) +atwhere{at}is white noise.

7 It is easy to see thatrt = i=0 i1at i,andV(rt) = 2a1 21,provided that 21<1. In other words, the weak stationarityof an AR(1) model implies that| 1|< / 40 StationarityACFL jung-BoxtestWhite noiseAR modelsExamplePACFAIC/BICF orecastingMA modelsSummaryUsing 0= (1 1) , one can rewrite a stationary AR(1) model asrt= 0+ 1rt 1+at,such that 1measures the persistence of the dynamicdependence of an AR(1) time ACF of the AR(1) is l= 1 l 1l >0,where 0= 1 1+ 2aand l= , l= l1, , the ACF of a weakly stationary AR(1) series decaysexponentially with rate 1and starting value 0= / 40 StationarityACFL jung-BoxtestWhite noiseAR modelsExamplePACFAIC/BICF orecastingMA modelsSummaryAR(2)An AR(2) model assumes the formrt= 0+ 1rt 1+ 2rt 2+at,whereE(rt)

8 = = 01 1 2,provided that 1+ 26= is easy to see that l= 1 l 1+ 2 l 2,forl >0,and that l= 1 l 1+ 2 l 2, l 2,with 1= 1/(1 2).16 / 40 StationarityACFL jung-BoxtestWhite noiseAR modelsExamplePACFAIC/BICF orecastingMA modelsSummaryUS real GNPAs an illustration, consider the quarterly growth rate of gross national product (GNP), seasonally adjusted,from the second quarter of 1947 to the first quarter of we simply employ an AR(3) model for the the growth rate byrtthe fitted model isrt= + 1+ 2 3+at,with a= / 40 StationarityACFL jung-BoxtestWhite noiseAR modelsExamplePACFAIC/BICF orecastingMA modelsSummaryAlternatively,rt 1 2+ 3= +at,with the corresponding third-order difference equation(1 + ) = 0or(1 + )(1 + ) = 0 The first factor(1 + )

9 Shows an exponentially decaying feature of the / 40 StationarityACFL jung-BoxtestWhite noiseAR modelsExamplePACFAIC/BICF orecastingMA modelsSummaryBusiness cyclesThe second factor (1 + ) confirms the existenceof stochastic business cycles. For an AR(2) model with a pair ofcomplex characteristic roots, the average length of the stochasticcycles isk=2 cos 1[ 1/(2 2)]ork= quarters, which is about 3 :If one uses a nonlinear model to separate economy into expansion and contraction periods, the data show that theaverage duration of contraction periods is about 3 quarters andthat of expansion periods is about 12 average duration of quarters is a compromise betweenthe two separate durations.

10 The periodic feature obtained here iscommon among growth rates of national economies. For example,similar features can be found for many OECD / 40 StationarityACFL jung-BoxtestWhite noiseAR modelsExamplePACFAIC/BICF orecastingMA modelsSummaryRcodegnp=scan(file=" ")# To create a time-series objectgnp1=ts(gnp,frequency=4,start=c(19 47,2))par(mfrow=c(1,1))plot(gnp1)points( gnp1,pch="*")# Find the AR orderm1=ar(gnp,method="mle")m1$orderm2=a rima(gnp,order=c(3,0,0))m2# In R, intercept denotes the mean of the series.# Therefore, the constant term is obtained below:( +.1423)* # Residual standard errorsqrt(m2$sigma2)# Characteristic equation and solutionsp1=c(1,-m2$coef[1:3])roots = polyroot(p1)# Compute the absolute values of the solutionsMod(roots)[1] # To compute average length of business cycles:k=2*pi/acos( )20 / 40 StationarityACFL jung-BoxtestWhite noiseAR modelsExamplePACFAIC/BICF orecastingMA modelsSummaryAR(p)The results of the AR(1) and AR(2) models can readily begeneralized to the general AR(p) model .