Time Series Forecasting - R in Finance

1 406080100120406080mmComputational Finance and Risk ManagementTime Series Forecastingwith State Space ModelsEric ZivotUniversity of WashingtonGuy YollinUniversity of WashingtonZivot & Yollin (Copyright 2012) time Series with State Space ModelsR/ Finance 20121 / 90406080100120406080mmOutline1 Introduction to state space models and the dlm package2 DLM estimation and Forecasting examples3 Structural time Series models and StructTS4 Exponential smoothing models and the forecast package5 time Series cross validation6 SummaryZivot & Yollin (Copyright 2012) time Series with State Space ModelsR/ Finance 20122 / 90406080100120406080mmLecture referencesG. Petris, S. Petrone, and P. CampagnoliDynamic Linear Models with , Durbin and S. J. KoopmanTime Series Analysis by State Space University Press, J. F. Commandeur and S. J. KoopmanAn Introduction to State Space time Series University Press, HyndmanForecasting with Exponential Smoothing: The State , 2008 Zivot & Yollin (Copyright 2012) time Series with State Space ModelsR/ Finance 20123 / 90406080100120406080mmOutline1 Introduction to state space models and the dlm package2 DLM estimation and Forecasting examples3 Structural time Series models and StructTS4 Exponential smoothing models and the forecast package5 time Series cross validation6 SummaryZivot & Yollin (Copyright 2012) time Series with State Space ModelsR/ Finance 20124 / 90406080100120406080mmLinear-Gaussian State Space ModelsLinear-Gaussian state space modelA linear-Gaussianstate space modelfor anm dimensional time seriesytconsists of ameasurement equationrelating the observed data to anp dimensional state vector t,and a Markovian transition equation thatdescribes the evolution of the state vector over equationhas the formytm 1=Ft(m p) t(p 1)+vtm 1,vt iidN(0,Vt)Thetransition equationfor the state vector tis the first order Markovprocess tp 1=Gt(p p)

2 T 1(p 1)+wt(p 1)wt iidN(0,Wt)E[vtw s] =0for alls,t= 1,..,TZivot & Yollin (Copyright 2012) time Series with State Space ModelsR/ Finance 20125 / 90406080100120406080mmLinear-Gaussian State Space ModelsThe matricesFt,Vt,Gt,andWtare called thesystem matrices, andcontain non-random these matrices do not depend deterministically ontthe state spacesystem is calledtime : Ifytis covariance stationary, then the state space system willbe time & Yollin (Copyright 2012) time Series with State Space ModelsR/ Finance 20126 / 90406080100120406080mmSpecification of Initial State Distribution 0 N(m0,C0)E[vt 0] =0,E[wt 0] =0fort= 1,..,TIf some or all of the elements of tare covariance stationary, then wecan typically solve for the corresponding elements ofm0andC0analytically from the elements of the system matricesFor deterministic elements of ( , mean of a Series ), thecorresponding element ofC0is defined to be non-stationary elements of ,it is customary to set thecorresponding element ofC0to a very large positive number, say & Yollin (Copyright 2012) time Series with State Space ModelsR/ Finance 20127 / 90406080100120406080mmMean-zero covariance stationary AR(2) modelME :yt=ctTE.

3 Ct= 1ct 1+ 2ct 2+ t, t N(0, 2 )The state vector is t= (ct, 2ct 1) and the transition equation is(ct 2ct 1)=( 11 20)(ct 1 2ct 2)+( t0)The transition equation system matrices areG=( 11 20),W=( 2 000),wt=( t0)Zivot & Yollin (Copyright 2012) time Series with State Space ModelsR/ Finance 20128 / 90406080100120406080mmMean-zero covariance stationary AR(2) modelThemeasurement equationisyt= (1,0) twhich has system matricesFt= (1,0),V= 0 vt= 0 Initial state distribution 0 N(m0,C0)Since t= (ct, 2ct 1) is stationary, we findm0using 0=E[ t] =GE[ t 1] +E[wt] =GE[ t] E[ t](I2 G) =0 m0=E[ t] =0 Zivot & Yollin (Copyright 2012) time Series with State Space ModelsR/ Finance 20129 / 90406080100120406080mmMean-zero covariance stationary AR(2) modelFor the state variance, stationarity of t=G t 1+wtimplies that for alltvar( t) =Gvar( t)G +var(wt) C0=GC0G +WStacking columns via thevec( ) operator then givesvec(C0) = (G G)vec(C0) +vec(W) vec(C0) = (I4 G G) 1vec(W)Zivot & Yollin (Copyright 2012) time Series with State Space ModelsR/ Finance 201210 / 90406080100120406080mmMean zero ARMA(1,1) modelTE :yt=ctME :ct= ct 1+ t+ t 1, t N(0, 2 )Define t= (ct, t) and writeyt= (1 0) t(ct t)=( 100)(ct 1 t 1)+( t t)so that the system matrices areF= (1,0),V= 0G=( 100),W= 2 (1 2)Zivot & Yollin (Copyright 2012) time Series with State Space ModelsR/ Finance 201211 / 90406080100120406080mmLinear regression with time varying parametersME :yt= t+ txt+vt,vt N(0, 2v)TE : t= t 1+w ,t,w ,t N(0, 2 )TE.

4 T= t 1+w ,t,w ,t N(0, 2 )Define t= ( t, t) yt= (1xt) t+vt( t t)=(1 00 1)( t 1 t 1)+(w ,tw ,t)Zivot & Yollin (Copyright 2012) time Series with State Space ModelsR/ Finance 201212 / 90406080100120406080mmLinear regression with time varying parametersThe system matrices areFt= (1xt),Vt= 2v,G=I2,W=( 2 00 2 )Notice thatFtis time the tis non-stationary the initial distribution is 0 N(m0,C0)m0=0,C0=k I2,k= 107 Zivot & Yollin (Copyright 2012) time Series with State Space ModelsR/ Finance 201213 / 90406080100120406080mmLog-Normal Stochastic Volatility Modelrt= tut,ut N(0,1)ln t= + ln t 1+ t, t N(0, 2 ),| |<1 Notice that|rt|= t|ut|so thatln|rt|= ln t+ ln|ut|E[|ut|] = ,var(|ut|) = 2/8 Henceln|rt|= + ln t+vt,vt (0, 2/8)ln t= + ln t 1+ t, t N(0, 2 )Zivot & Yollin (Copyright 2012) time Series with State Space ModelsR/ Finance 201214 / 90406080100120406080mmLog-Normal Stochastic Volatility ModelDefine t= ( , ,ln t) . Then the state-space representation isME : ln|rt|=(1 0 1) t+vtTE : ln t = 1 000 100 1 ln t 1 + 00 t The system matrices areF=(1 0 1),V= 2/8G= 1 000 100 1 ,W= 0 000 000 0 2 Zivot & Yollin (Copyright 2012) time Series with State Space ModelsR/ Finance 201215 / 90406080100120406080mmLog-Normal Stochastic Volatility ModelBecause ln tfollows a stationary AR(1)E[ln t] = 1 ,var(ln t) = 2 1 2 The initial distribution is 0 N(m0,C0)m0= /(1 ) ,C0= 0 000 000 0 2 1 2 Note: in dlm, you can t have elements ofC0exactly zero.

5 Use very smallnumber 1e-7 & Yollin (Copyright 2012) time Series with State Space ModelsR/ Finance 201216 / 90406080100120406080mmSpecifying a State Space Model with thedlmpackageState space models in dlm are represented as lists with namedcomponents associated with the system matrices and initial valueparametersModel ParameterList NameTime Varying NameFFF JFFVv JVGGG JFFWW JWC0C0m0m0dataXyt=Ft t+vtvt NID(0,Vt) t=Gt t 1+wtwt NID(0,Wt)Zivot & Yollin (Copyright 2012) time Series with State Space ModelsR/ Finance 201217 / 90406080100120406080mmSpecifying a State Space Model with thedlmpackageFunctionModeldlmgeneric DLMdlmModARMAARMA processdlmModPolynth order polynomial DLMdlmModRegLinear regressiondlmModSeasPeriodic Seasonal factorsdlmModTrigPeriodic Trigonometric formTable: Functions to create dlm objectsZivot & Yollin (Copyright 2012) time Series with State Space ModelsR/ Finance 201218 / 90406080100120406080mmExample Models: ARMA(1,1)R Code: Create an ARMA model with DLM> ######################################## > # state space for ARMA(1,1)> ######################################## > # ARMA(1,1) with phi= , theta= , sig2=1> library(dlm)> library(methods)> = dlmModARMA(ar= , ma= , sigma2=1)> class( )[1] "dlm"> names( )[1] "m0" "C0" "FF" "V" "GG" "W" "JFF" "JV" "JGG" "JW"Zivot & Yollin (Copyright 2012) time Series with State Space ModelsR/ Finance 201219 / 90406080100120406080mmExample Models: ARMA(1,1)R Code.

6 Create an ARMA model with DLM> $FF[,1] [,2][1,] 1 0$V[,1][1,] 0$GG[,1] [,2][1,] 1[2,] 0$W[,1] [,2][1,] [2,] $m0[1] 0 0$C0[,1] [,2][1,] 1e+07 0e+00[2,] 0e+00 1e+07 Zivot & Yollin (Copyright 2012) time Series with State Space ModelsR/ Finance 201220 / 90406080100120406080mmExample Models: Regression with time -varying parametersR Code: TVP Regression Model> library(PerformanceAnalytics)> data(managers)> # extract HAM1 and SP500 excess returns> HAM1 = 100*(managers[,"HAM1", drop=FALSE] - managers[,"US 3m TR", drop=FALSE])> sp500 = 100*(managers[,"SP500 TR", drop=FALSE] - managers[,"US 3m TR",drop=FALSE])> colnames(sp500) = "SP500"> s2v = 1> s2a = > s2b = > = dlmModReg(X=sp500, addInt=TRUE,dV=s2v, dW=c(s2a, s2b))Zivot & Yollin (Copyright 2012) time Series with State Space ModelsR/ Finance 201221 / 90406080100120406080mmExample Models: Regression with time -varying parametersR Code: TVP Regression Model> [c("FF","V","GG","W","m0","C0")]$FF[,1] [,2][1,] 1 1$V[,1][1,] 1$GG[,1] [,2][1,] 1 0[2,] 0 1$W[,1] [,2][1,] [2,] $m0[1] 0 0$C0[,1] [,2][1,] 1e+07 0e+00[2,] 0e+00 1e+07 Zivot & Yollin (Copyright 2012) time Series with State Space ModelsR/ Finance 201222 / 90406080100120406080mmExample Models: Regression with time -varying parametersR Code: TVP Regression Model> [c("JFF","JV","JGG","JW")]$JFF[,1] [,2][1,] 0 1$JVNULL$JGGNULL$JWNULL> head( $X)SP5001996-01-30 & Yollin (Copyright 2012) time Series with State Space ModelsR/ Finance 201223 / 90406080100120406080mmExample Models: Log-Normal AR(1) SV ModelR Code: Log-Normal AR(1) SV Model> ######################################## > # EX 3.

7 State space for SV model> ######################################## > # ln|r(t)| = + lns(t) + v(t), v(t) ~ (0,pi^2/8)> # lns(t) = w + phi*lns(t-1) + w(t), w(t) ~ N(0, s2w)> # theta = ( , w, lns(t))'> # m0 = ( , w, w/(1-phi))'> # C0 = I*1e-7, C0[3,3] = sw2/(1-phi^2)> phi = > sig2n = 1> omega = > = matrix(c(1,0,1),1,3)> = pi^2/8> = matrix(c(1,0,0,0,1,0,0,1,phi),3,3)> = diag(0,3)> [3,3] = sig2n> = c( , omega, omega/(1-phi))> = diag(1,3)*1e-7> [3,3] = sig2n/(1-phi^2)> = dlm(FF= , V= , GG= ,W= , m0= , C0= )Zivot & Yollin (Copyright 2012) time Series with State Space ModelsR/ Finance 201224 / 90406080100120406080mmExample Models: Log-Normal AR(1) SV ModelR Code: Log-Normal AR(1) SV Model> $FF[,1] [,2] [,3][1,] 1 0 1$V[,1][1,] $GG[,1] [,2] [,3][1,] 1 0 [2,] 0 1 [3,] 0 0 $W[,1] [,2] [,3][1,] 0 0 0[2,] 0 0 0[3,] 0 0 1$m0[1] $C0[,1] [,2] [,3][1,] 1e-07 0e+00 [2,] 0e+00 1e-07 [3,] 0e+00 0e+00 & Yollin (Copyright 2012) time Series with State Space ModelsR/ Finance 201225 / 90406080100120406080mmSignal Extraction and PredictionIn a state space model, the unobserved state vector tis the signal and themeasurement errorwtis the noise.

8 Given observed datay1,..,yTthegoals of state space estimation are:1 Optimal signal extraction2 Optimalh step ahead prediction of states and dataZivot & Yollin (Copyright 2012) time Series with State Space ModelsR/ Finance 201226 / 90406080100120406080mmFiltering and SmoothingThere are two types of signal extraction:1 Filtering: Optimal estimates of tgiven information available at timet,It={y1,..,yt}:E[ t|It] = filtered estimate of 2 Smoothing: Optimal estimates of tgiven information available attimeT,IT={y1,..,yT}E[ t|IT] = smoothed estimate of Zivot & Yollin (Copyright 2012) time Series with State Space ModelsR/ Finance 201227 / 90406080100120406080mmThe Kalman FilterTheKalman filteris a set of recursion equations for determining theoptimal estimates of the state vector tgiven information available at timet, filter consists of two sets of equations:1 Prediction equations2 Updating equationsTo describe the filter, letmt=E[ t|It] = optimal estimator of tbased onItCt=E[( t mt)( t mt) |It] = MSE matrix ofmtZivot & Yollin (Copyright 2012) time Series with State Space ModelsR/ Finance 201228 / 90406080100120406080mmPrediction EquationsGivenmt 1andCt 1at timet 1,the optimal predictor of tand itsassociated MSE matrix aremt|t 1=E[ t|It 1] =Gtmt 1Ct|t 1=E[( t mt 1)( t mt 1) |It 1]=GtCt 1G t+WtThe corresponding optimal predictor ofytgive information att 1 isyt|t 1=E[yt|It 1] =Ftmt|t 1 Theprediction errorand its MSE matrix areet=yt yt|t 1=yt Ftmt|t 1=Ft( t mt|t 1) +vtE[ete t] =Qt=FtCt|t 1F t+VtZivot & Yollin (Copyright 2012) time Series with State Space ModelsR/ Finance 201229 / 90406080100120406080mmUpdating EquationsWhen new observationsytbecome available, the optimal predictormt|t 1and its MSE matrix are updated usingmt=mt|t 1+Ct|t 1F tQ 1t(yt Ftmt|t 1)

9 =mt|t 1+Ct|t 1F tQ 1tvtCt=Ct|t 1 Ct|t 1F tQ 1tFtCt|t 1 Note:Kt=Ct|t 1F tQ 1t= Kalman gain matrix. It gives the weight onnew informationet=yt Ftmt|t 1in the updating equation & Yollin (Copyright 2012) time Series with State Space ModelsR/ Finance 201230 / 90406080100120406080mmKalman SmootherOnce all dataITis observed, the optimal estimatesE[ t|IT] can becomputed using the backwards Kalman smoothing recursionsE[ t|IT] =mt|T=mt+C t(mt+1|T Gt+1mt)E[( t mt|T)( t mt|T) |IT] =Ct|T=Ct+C t(Ct+1|T Ct+1|t)C tC t=CtG t+1C 1t+1|tThe algorithm starts by settingmT|T=mTandCT|T=CTand thenproceeds backwards fort=T 1,.., & Yollin (Copyright 2012) time Series with State Space ModelsR/ Finance 201231 / 90406080100120406080mmMaximum likelihood estimationLet denote the parameters of the state space model, which areembedded in the system matricesFt,Gt, are typically unknown and must be estimated from thedatay={y1,..,yT}.In the linear-Gaussian state space model, the parameter vector canbe estimated be estimated by maximum likelihood using theprediction error decomposition of the log-likelihood MLE=argmax lnL( |y) =T t=1lnf(yt|It 1; )wheref(yt|It 1; ) is the conditional density ofytgivenIt 1 Zivot & Yollin (Copyright 2012) time Series with State Space ModelsR/ Finance 201232 / 90406080100120406080mmPrediction Error DecompositionFrom the Kalman filter equations with a fixed value of we have thatyt|t 1 N(Ft( )mt|t 1( ),Qt( ))and sof(yt|It 1.)

10 = (2 Qt( )) 1/2exp{ 12et( ) Q 1tet( )}Theprediction error decompositionof the Gaussian log-likelihoodfunction follows immediately:lnL( |y) = NT2ln(2 ) 12T t=1ln|Qt( )| 12T t=1e t( )Q 1t( )et( )Zivot & Yollin (Copyright 2012) time Series with State Space ModelsR/ Finance 201233 / 90406080100120406080mmForecastingThe Kalman filter prediction equations produces in-sample 1-stepahead forecasts and MSE matricesOut-of-sampleh step ahead predictions and MSE matrices can becomputed from the prediction equations by extending the data sety1,..,yTwith a set