Time Series: Autoregressive models AR, MA, ARMA, ARIMA

1 time series : Autoregressive modelsAR, MA, ARMA, ARIMAM ingda ZhangUniversity of 23, 20181 / 77 Overview1 Introduction of time SeriesCategories and TerminologiesWhite Noise and Random WalkTime series Analysis2 ARIMA ModelsAR ProcessMA ProcessARMA ModelsARIMA Models3 ARIMA Modeling: A Toy Problem2 / 77 time series A time series is a sequential set of data points, measuredtypically over successive times. time series analysis comprises methods for analyzing timeseries data in order to extract meaningful statistics and othercharacteristics of the / 77 Categories and Terminologies time -domain approach: how does what happened today affectwhat will happen tomorrow?These approaches view the investigation of lagged relationshipsas most important, autocorrelation analysis.

2 Frequency-domain approach: what is the economic cyclethrough periods of expansion and recession?These approaches view the investigation of cycles as mostimportant, spectral analysis and wavelet analysis. This lecture will focus on time -domain / 77 Categories and Terminologies (cont.) time series containing records of a single variable is termedas univariate, but if records of more than one variable areconsidered then it is termed as multivariate. time series model is said to be linear or non-lineardepending on whether the current value of the series is alinear or non-linear function of past observations. a continuous time series observations are measured at everyinstance of time , whereas a discrete time series containsobservations measured at discrete points in time .

3 This lecture will focus on univariate, linear, discrete / 77 Components of a time series In general, a time series is affected by four components, , seasonal,cyclical and irregular components. TrendThe general tendency of a time series to increase, decrease orstagnate over a long period of per pound20052010201560708090110 The price of chicken: monthly whole bird spot price, Georgia docks, UScents per pound, August 2001 to July 2016, with fitted linear trend / 77 Components of a time series (cont.) In general, a time series is affected by four components, , seasonal,cyclical and irregular components. Seasonal variationThis component explains fluctuations within a year during theseason, usually caused by climate and weather conditions,customs, traditional habits, Earnings per Share19601965197019751980051015 Johnson & Johnson quarterly earnings per share, 84 quarters, 1960-I to / 77 Components of a time series (cont.)

4 In general, a time series is affected by four components, , seasonal,cyclical and irregular components. Cyclical variationThis component describes the medium-term changes caused bycircumstances, which repeat in cycles. The duration of a cycleextends over longer period of Mortality1970197219741976197819807090110 130 Average weekly cardiovascular mortality in Los Angeles County. There are508 six-day smoothed averages obtained by filtering daily values over the10 year period / 77 Components of a time series (cont.) In general, a time series is affected by four components, , seasonal,cyclical and irregular components. Irregular variationIrregular or random variations in a time series are caused byunpredictable influences, which are not regular and also do notrepeat in a particular variations are caused by incidences such as war, strike,earthquake, flood, revolution, is no defined statistical technique for measuring randomfluctuations in a time / 77 Combination of Four Components Considering the effects of these four components, twodifferent types of models are generally used for a time series .

5 Additive ModelY(t) =T(t) +S(t) +C(t) +I(t)Assumption: These four components are independent of eachother. Multiplicative ModelY(t) =T(t) S(t) C(t) I(t)Assumption: These four components of a time series are notnecessarily independent and they can affect one / 77 time series Example: White Noise White Noise A simple time series could be a collection of uncorrelatedrandom variables,{wt}, with zero mean = 0 and finitevariance 2w, denoted aswt wn(0, 2w). Gaussian White Noise A particular useful white noise is Gaussian white noise, whereinthewtare independent normal random variables (with mean 0and variance 2w), denoted aswt iidN(0, 2w). White noise time series is of great interest because if thestochastic behavior of all time series could be explained interms of the white noise model , then classical statisticalmethods would / 77 time series Example: Random Walk A random walk is the process by which randomly-movingobjects wander away from where they started.

6 Consider a simple 1-D process: The value of the time series at timetis the value of the seriesat timet 1 plus a completely random movement determinedbywt. More generally, a constant drift factor is +Xt 1+wt= t+t i=1wirandom walk0100200300400500 2002040608012 / 77 time series Analysis The procedure of using known data values to fit a time serieswith suitable model and estimating the correspondingparameters. It comprises methods that attempt to understandthe nature of the time series and is often useful for futureforecasting and simulation. There are several ways to build time series forecasting models ,but this lecture will focus onstochastic process. We assume a time series can be defined as a collection ofrandom variables indexed according to the order they areobtained in time ,X1,X2,X3.

7 Twill typically be discrete andvary over the integerst= 0, 1, 2,.. Note that the collection of random variables{Xt}is referred toas a stochastic process, while the observed values are referredto as a realization of the stochastic / 77 Measures of Dependence A complete description of a time series , observed as acollection ofnrandom variables at arbitrary time pointst1,t2,..,tn, for any positive integern, is provided by thejoint distribution function, evaluated as the probability thatthe values of the series are jointly less than thenconstants,c1,c2,..,cn; ,Ft1,t2,..,tn(c1,c2,..,cn) =Pr(Xt1 c1,Xt2 c2,..,Xtn cn). Unfortunately, these multidimensional distribution functionscannot usually be written easily. Therefore some informative descriptive measures can beuseful, such as mean function and / 77 Measurement Functions Mean function The mean function is defined as t= Xt=E[Xt] = xft(x)dx,provided it exists, where E denotes the usual expected valueoperator.

8 Clearly for white noise series , wt=E[wt] = 0 for allt. For random walk with drift ( 6= 0), Xt=E[Xt] = t+t i=1E[wi] = t15 / 77 Autocovariance for time series Lack of independence between adjacent values in time seriesXsandXtcan be numerically assessed. Autocovariance Function Assuming the variance ofXtis finite, the autocovariancefunction is defined as the second moment product (s,t) = X(s,t) =cov(Xs,Xt) =E[(Xs s)(Xt t)],for allsandt. Note that (s,t) = (t,s) for all time pointssandt. The autocovariance measures the linear dependence betweentwo points on the same series observed at different times. Very smooth series exhibit autocovariance functions that staylarge even when thetandsare far apart, whereas choppyseries tend to have autocovariance functions that are nearlyzero for large / 77 Autocorrelation for time series Autocorrelation Function (ACF) The autocorrelation function is defined as (s,t) = (s,t) (s,s) (t,t) According to Cauchy-Schwarz inequality| (s,t)|2 (s,s) (t,t),it s easy to show that 1 (s,t) 1.

9 ACF measures the linear predictability ofXtusing onlyXs. If we can predictXtperfectly fromXsthrough a linearrelationship, then ACF will be either +1 or / 77 Stationarity of Stochastic Process forecasting is difficult as time series is non-deterministic innature, we cannot predict with certainty what will occurin the future. But the problem could be a little bit easier if the time series isstationary: you simply predict its statistical properties will bethe same in the future as they have been in the past! A stationary time series is one whose statistical properties suchas mean, variance, autocorrelation, etc. are all constant overtime. Most statistical forecasting methods are based on theassumption that the time series can be renderedapproximately stationary after mathematical / 77 Which of these are stationary?

10 19 / 77 Strict Stationarity There are two types of stationarity, strictly stationary andweakly stationary. Strict Stationarity The time series {Xt,t Z}is said to be strictly stationary ifthe joint distribution of (Xt1,Xt2,..,Xtk) is the same as thatof (Xt1+h,Xt2+h,..,Xtk+h). In other words, strict stationarity means that the jointdistribution only depends on the difference h, not the time (t1,t2,..,tk). However in most applications this stationary condition is / 77 Weak Stationarity Weak Stationarity The time series {Xt,t Z}is said to be weakly stationary if1E[X2t]< , t Z;2E[Xt] = , t Z;3 X(s,t) = X(s+h,t+h), s,t,h Z. In other words, a weakly stationary time series {Xt}must havethree features: finite variation, constant first moment, andthat the second moment X(s,t) only depends on|t s|andnot depends onsort.