Lecture 2: ARMA(p,q) models (part 3)

1 Lecture 2: ARMA(p,q) models (part 3)Florian PelgrinUniversity of Lausanne, Ecole des HECD epartment of mathematics (IMEA-Nice)Sept. 2011 - Jan. 2012 Florian Pelgrin (HEC)Univariate time seriesSept. 2011 - Jan. 20121 / 32 IntroductionMotivationCharacterize the main properties of ARMA(p,q) of ARMA(p,q) modelsFlorian Pelgrin (HEC)Univariate time seriesSept. 2011 - Jan. 20122 / 32 IntroductionRoad map1 ARMA(1,1) modelDefinition and conditionsMomentsEstimation2 ARMA(p,q) modelDefinition and conditionsMomentsEstimation3 Application4 AppendixFlorian Pelgrin (HEC)Univariate time seriesSept. 2011 - Jan. 20123 / 32 ARMA(1,1) modelDefinition and conditions1. ARMA(1,1) Definition and conditionsDefinitionA stochastic process (Xt)t Zis said to be a mixture autoregressive movingaverage model of order 1, ARMA(1,1), if it satisfies the followingequation :Xt= + Xt 1+ t+ t 1 t (L)Xt= + (L) twhere 6= 0, 6= 0, is a constant term, ( t)t Zis a weak white noiseprocess with expectation zero and variance 2 ( t WN(0, 2 )), (L) = 1 Land (L) = 1 + Pelgrin (HEC)Univariate time seriesSept.

2 2011 - Jan. 20124 / 32 ARMA(1,1) modelDefinition and conditions0100200300-4-2024 Simulation of an ARMA(1,1) with ( ; )0100200300-10-50510 Simulation of an ARMA(1,1) with ( ; )0100200300-10-50510 Simulation of an ARMA(1,1) with ( ; )0100200300-4-2024 Simulation of an ARMA(1,1) with ( ; ) Florian Pelgrin (HEC)Univariate time seriesSept. 2011 - Jan. 20125 / 32 ARMA(1,1) modelDefinition and conditionsThe properties of an ARMA(1,1) process are a mixture of those of anAR(1) and MA(1) processes :The (stability) stationarity condition is the one of an AR(1) process (orARMA(1,0) process) :| |< invertibility condition is the one of a MA(1) process (orARMA(0,1) process) :| |< representation of an ARMA(1,1) process is fundamental or causalif :| |<1 and| |< representation of an ARMA(1,1) process is said to be minimal andcausal if :| |<1,| |<1 and 6= .Florian Pelgrin (HEC)Univariate time seriesSept.

3 2011 - Jan. 20126 / 32 ARMA(1,1) modelDefinition and conditionsIf (Xt) is stable and thus weakly stationary, then (Xt) has an infinitemoving average representation (MA( )) :Xt= 1 + (1 L) 1(1 + L) t= 1 + k=0ak t kwhere :a0= 1ak= k+ k Pelgrin (HEC)Univariate time seriesSept. 2011 - Jan. 20127 / 32 ARMA(1,1) modelDefinition and conditionsIf (Xt) is invertible, then (Xt) has an infinite autoregressiverepresentation (AR( )) :(1 L) 1(1 L)Xt= 1 + = 1 + k=1bkXt k+ twhere = , and :bk= k k 1 .Florian Pelgrin (HEC)Univariate time seriesSept. 2011 - Jan. 20128 / 32 ARMA(1,1) MomentsDefinitionLet (Xt) denote a stationary stochastic process that has a fundamentalARMA(1,1) representation,Xt= + Xt 1+ t+ t 1. Then :E[Xt] = 1 m X(0) V(Xt) =1 + 2 + 21 2 2 X(1) Cov[Xt,Xt 1] =( + )(1 + )1 2 2 X(h) = X(h 1) for|h|> : See Appendix Pelgrin (HEC)Univariate time seriesSept.

4 2011 - Jan. 20129 / 32 ARMA(1,1) modelMomentsDefinitionThe autocorrelation function of an ARMA(1,1) process satisfies : X(h) = 1 ifh= 0( + )(1+ )1+2 + 2if|h|= 1 X(h 1) if|h|> Pelgrin (HEC)Univariate time seriesSept. 2011 - Jan. 201210 / 32 ARMA(1,1) modelMomentsThe autocorrelation function of an ARMA(1,1) process exhibitsexponential decay towards zero : it does not cut off but gradually diesout autocorrelation function of an ARMA(1,1) process displays theshape of that of an AR(1) process for|h|> Pelgrin (HEC)Univariate time seriesSept. 2011 - Jan. 201211 / 32 ARMA(1,1) modelMomentsPartial Autocorrelation :The partial autocorrelation function of an ARMA(1,1) process willgradually die out (the same property as a moving average model).Florian Pelgrin (HEC)Univariate time seriesSept. 2011 - Jan. 201212 / 32 ARMA(1,1) modelEstimationEstimationSame techniques as before, especially those of MA estimator : the extended Yule-Walker equations could beused in principe to estimate the AR coefficients but the MAcoefficients need to be estimated by other the presence of moving average components, the least squaresestimator becomes nonlinear and the corresponding estimator is theconditional nonlinear least squares estimator (see estimation ofMA(q) models ).

5 It has to be solved with numerical explicit distributional assumption for the error term, theconditional or exact maximum likelihood estimator can be computed(using also numerical or optimization methods).Other methods are also available : the Kalman filter, the generalizedmethod of moments, Pelgrin (HEC)Univariate time seriesSept. 2011 - Jan. 201213 / 32 ARMA(1,1) modelEstimationEstimation of ARMA(p,q) models (true DGP: = 0, = , and = ) Coefficient Std. Error t-Statistic p-value. Akaike info criterion Schwarz criterion ARMA(1,1) C AR(1) MA(1) ARMA(2,2) C AR(1) AR(2) MA(1) MA(2) ARMA(2,1) AR(1) AR(2) MA(1) AR(2) C AR(1) AR(2) AR(1) C AR(1) MA(2) C MA(1) MA(2) MA(4) C MA(1) MA(2) MA(3) MA(4) Note: C, AR(j), and MA(j) are respectively the estimate of the constant term, the jth autogressive term, and the jth moving average term.

6 Florian Pelgrin (HEC)Univariate time seriesSept. 2011 - Jan. 201214 / 32 ARMA(p,q) modelDefinition and conditions2. ARMA(p,q) Definition and conditionsDefinitionA stochastic process (Xt)t Zis said to be a mixture autoregressive movingaverage model of orderpandq, ARMA(p,q), if it satisfies the followingequation :Xt= + 1Xt 1+ + pXt p+ t+ 1 t 1+ + q t q t (L)Xt= + (L) twhere q6= 0, p6= 0, is a constant term, ( t)t Zis a weak white noiseprocess with expectation zero and variance 2 ( t WN(0, 2 )), (L) = 1 1L pLpand (L) = 1 + 1L+ + Pelgrin (HEC)Univariate time seriesSept. 2011 - Jan. 201215 / 32 ARMA(p,q) modelDefinition and conditionsMain idea of ARMA(p,q) modelsApproximate Wold form of stationary time series by parsimoniousparametric modelsAR and MA models can be cumbersome because one may need ahigh-order model with many parameters to adequately describe thedata dynamics (see the effective Fed fund rate application)By mixing AR and MA models into a more compact form, the numberof parameters is kept Pelgrin (HEC)Univariate time seriesSept.

7 2011 - Jan. 201216 / 32 ARMA(p,q) modelDefinition and conditionsThe properties of an ARMA(p,q) process are a mixture of those of anAR(p) and MA(q) processes :The (stability) stationarity conditions are those of an AR(p) process (orARMA(p,0) process) :zp (z 1) = 0 zp 1zp 1 p= 0 |zi|< 1, , invertibility conditions are those of an MA(q) process (orARMA(0,q) process) :zq (z 1) = 0 zq+ 1zq 1+ + q= 0 | zi|< 1, , representation of an ARMA(p,q) process is fundamental or causalif it is stable and invertibleThe representation of an ARMA(1,1) process is said to be minimal andcausal if it is stable, invertible and the characteristic polynomialszp (z 1) andzq (z 1) have no common Pelgrin (HEC)Univariate time seriesSept. 2011 - Jan. 201217 / 32 ARMA(p,q) modelDefinition and conditionsDefinitionThe representation of a mixture autoregressive moving average process of orderpandqdefined by :Xt= + 1Xt 1+ + pXt p+ t+ 1 t 1+ + q t q,is said to be a minimal causal (fundamental) representation ( t) is the innovationprocess if :(i) All the roots of the characteristic equation associated to zp 1zp 1 p= 0 are of modulus less than one,|zi|<1 fori= 1, ,p;(ii) All the roots of the characteristic equation associated to zq+ 1zq 1+ + q= 0 are of modulus less than one,| zi|<1 fori= 1, ,q;(iii) The characteristic polynomialszp (z 1) andzq (z 1) have no common Pelgrin (HEC)Univariate time seriesSept.

8 2011 - Jan. 201218 / 32 ARMA(p,q) modelDefinition and conditionsIf (Xt) is stable and thus weakly stationary, then (Xt) has an infinitemoving average representation (MA( )) :Xt= 1 pk=1 k+ (L) 1(1 + L) t= 1 pk=1 k+ k=0ak t kwhere :a0= 1 k=0|ak|< Florian Pelgrin (HEC)Univariate time seriesSept. 2011 - Jan. 201219 / 32 ARMA(p,q) modelDefinition and conditionsIf (Xt) is invertible, then (Xt) has an infinite autoregressiverepresentation (AR( )) : (L) 1 (L)Xt= 1 qk=1 k+ = 1 qk=1 q+ k=1bkXt k+ twhere k= Pelgrin (HEC)Univariate time seriesSept. 2011 - Jan. 201220 / 32 ARMA(p,q) Moments of an ARMA(p,q) The properties of the moments of an ARMA(p,q) are also a mixtureof those of an AR(1) and MA(1) mean is the same as the one of an AR(p) model (with a constantterm) :E(Xt) = 1 pk=1 k Pelgrin (HEC)Univariate time seriesSept. 2011 - Jan.

9 201221 / 32 ARMA(p,q) modelMomentsAutocorrelation :The autocorrelation function of an ARMA(p,q) process exhibitsexponential decay towards zero : it does not cut off but gradually diesout ashincreases (possibly damped autocorrelation function of an ARMA(p,q) process displays theshape of that of an AR(p) process for|h|>max(p,q+ 1).Partial Autocorrelation : The partial autocorrelation function of anARMA(p,q) process will gradually die out (the same property as aMA(q) model).Florian Pelgrin (HEC)Univariate time seriesSept. 2011 - Jan. 201222 / 32 ARMA(p,q) EstimationSame techniques as in previous least squares method2 Maximum likelihood estimator (conditional or exact)3 Generalized method of moments4 EtcFlorian Pelgrin (HEC)Univariate time seriesSept. 2011 - Jan. 201223 / 32 Application3. ApplicationEffective Fed fund rate : 1970 :01-2010 :01 (monthly observations)An ARMA(1,2) captures better the dynamics of the effective Fedfund estimation of the effective Fed fund rate : ARMA(1,2)CoefficientsEstimatesStd.)

10 ErrorP-value Pelgrin (HEC)Univariate time seriesSept. 2011 - Jan. 201224 / 32 ApplicationEffective Fed fund rate: ARMA(1,2) specification -8-6-4-202405101520197019751980198519901 99520002005 ResidualActualFitted Florian Pelgrin (HEC)Univariate time seriesSept. 2011 - Jan. 201225 / 32 ApplicationEffective Fed fund rate: diagnostics of the ARMA(1,2) specification autocorrelation Florian Pelgrin (HEC)Univariate time seriesSept. 2011 - Jan. 201226 / 32 ApplicationEffective Fed fund rate: Impulse response function of the estimated ARMA(1,2) specification Response 2 Response 2 Florian Pelgrin (HEC)Univariate time seriesSept. 2011 - Jan. 201227 / 32 Appendix4. Appendix1. Moments of an ARMA(1,1).Florian Pelgrin (HEC)Univariate time seriesSept. 2011 - Jan. 201228 / 32 Appendix1. Moments of an ARMA(1,1)The properties of the moments of an ARMA(1,1) are a mixture ofthose of an AR(1) and MA(1) mean is the same as the one of an AR(1) model (with a constantterm) :E(Xt) =E( + Xt 1+ t+ t 1)= + E(Xt 1) +E( t) + E( t 1)= + E(Xt)sinceE(Xt) =E(Xt j) for allj(stationarity property) andE( t j) = 0 for allj(white noise).