TIME SERIES MODELLING, INFERENCE AND …

1 RAQUEL PRADO and MIKE WEST time SERIESMODELLING, INFERENCEAND forecasting AMS, University of California, Santa Cruz ISDS, Duke UniversityContentsRaquel Prado and Mike WestPart I Univariate time Series11 Notation, Definitions and Basic Inference3 Problem Areas, Graphical Displays and Objectives3 Stochastic Processes and Stationarity8 Exploratory Analysis: Auto-Correlation and Cross-Correlation9 Exploratory Analysis: Smoothing and Differencing12A Primer on Likelihood and Bayesian Inference15 Posterior Sampling25 Discussion and Further Topics29 Appendix30 Problems312 Traditional time SERIES Models33 Structure of Autoregressions33 Forecasting39 Estimation in AR Models41 Further Issues on Bayesian INFERENCE for AR Models52 Autoregressive, Moving Average (ARMA) Models60 Discussion and Further Topics76 Appendix77 Problems773 The Frequency Domain794 Dynamic Linear Models805 State-Space TVAR Models81iiiivCONTENTS6 Other Univariate Models and Methods82 Part II Multivariate time Series837 Analysing Multiple time Series858 Multivariate Models869 Multivariate Models in Finance8710 Other Multivariate Models and Methods88 References89 Part IUNIVARIATE time SERIES1 Notation, Definitions and BasicInferenceProblem Areas, Graphical Displays and expressiontime SERIES data, ortime SERIES , usually refers to a set ofobservations collected sequentially in time .

2 These observations could have beencollected at equally-spaced time points. In this case we usethe notationytwith(t=.., 1,0,1,2,..), , the set of observations is indexed byt, the time atwhich each observation was taken. If the observations were not taken at equally-spaced points then we use the notationyti, withi= 1,2,.., and so,(ti ti 1)isnot necessarily equal to SERIES processis a stochastic process or a collection of random variablesytindexed in time . Note thatytwill be used throughout the book to denote a randomvariable or an actual realisation of the time SERIES processat timet. We use thenotation{yt,t T},or simply{yt}, to refer to the time SERIES process. IfTis ofthe form{ti,i N}, then the process is a discrete- time random process and ifTis an interval in the real line, or a collection of intervals in the real line, then theprocess is a continuous- time random process.

3 In this framework, a time SERIES datasetyt,(t= 1,..,n), also denoted byy1:n, is just a collection ofnequally-spacedrealisations of some time SERIES many statistical models the assumption that the observations are realisationsof independent random variables is key. In contrast, time SERIES analysis is con-cerned with describing the dependence among the elements ofa sequence of each timet,ytcan be a scalar quantity, such as the total amount of rainfallcollected at a certain location in a given dayt, or it can be ak-dimensional vectorcollectingkscalar quantities that were recorded simultaneously. For instance, if thetotal amount of rainfall and the average temperature at a given location are measuredin dayt, we havek= 2scalar quantitiesy1,tandy2,tand so, at timetwe havea 2-dimensional vector of observationsyt= (y1,t,y2,t) . In general, forkscalarquantities recorded simultaneously at timetwe have a realisationytof a vectorprocess{yt,t T},withyt= (y1,t.)

4 ,yk,t) .34 NOTATION, DEFINITIONS AND BASIC INFERENCE time0100020003000-400-300-200-1000100200 Fig. SERIES (units in millivolts) displays a portion of a human electroencephalogram or EEG,recorded on a patient s scalp under certain electroconvulsive therapy (ECT) condi-tions. ECT is an effective treatment for patients under major clinical depression(Krystalet al., 1999). When ECT is applied to a patient, seizure activity appearsand can be recorded via electroencephalograms. The SERIES corresponds to one of19 EEG channels recorded simultaneously at different locations over the scalp. Themain objective in analysing this signal is the characterisation of the clinical effi-cacy of ECT in terms of particular features that can be inferred from the recordedEEG traces. The data are fluctuations in electrical potential taken at time inter-vals of roughly one fortieth of a second (more precisely 256 Hz).

5 For a moredetailed description of these data and a full statistical analysis see Westet al.(1999);Krystalet al.(1999) and Pradoet al.(2001). From the time SERIES analysis viewpoint,the objective here is modelling the data in order to provide useful insight into theunderlying processes driving the multiple SERIES during a seizure episode. Studyingthe differences and commonalities among the 19 EEG channelsis also key. Univari-ate time SERIES models for each individual EEG SERIES could be explored and used toinvestigate relationships across the 19 channels. Multivariate time SERIES analyses in which the observed SERIES ,yt, is a 19-dimensional vector whose elements are theobserved voltage levels measured simultaneously at the 19 scalp locations at eachtimet can also be AREAS, GRAPHICAL DISPLAYS AND OBJECTIVES5time0100200300400500t=1-500t= 701-1200t=1701-2200t=2501-3000t=3001-350 0 Fig.

6 Of the EEG trace displayed in Figure EEG SERIES display a quasi-periodic behaviour that changes dynamically intime, as shown in Figure , where different portions of theEEG trace shown inFigure are displayed. In particular, it is clear that therelatively high frequencycomponents that appear initially are slowly decreasing towards the end of the time SERIES model used to describe these data should takeinto account theirnon-stationary and quasi-periodic structure. We discuss various modelling alterna-tives for these data in the subsequent chapters, including the class of time -varyingautoregressions or TVAR models and other multi-channel shows the annual per capita GDP (gross domestic product) timeseries for Austria, Canada, France, Germany, Greece, Italy, Sweden, UK and USAduring 1950 and 1983. Interest lies in the study of periodic behaviour of suchseries in connection with understanding business cycles.

7 Other goals of the analysisinclude forecasting turning points and comparing characteristics of the SERIES acrossthe national of the main differences between any time SERIES analysisof the GDP seriesand any time SERIES analysis of the EEG SERIES , regardless ofthe type of models usedin such analyses, lies in the objectives. One of the goals in analysing the GDP data isforecasting future outcomes of the SERIES for the several countries given the observedvalues. In the EEG study there is no interest in forecasting future values of theseries given the observed traces, instead the objective is finding an appropriate model6 NOTATION, DEFINITIONS AND BASIC 5 6 7 8 Canada1950196019701980102030 France1950196019701980610 14 18 Germany195019601970198020 40 60 19501960197019805678 USAFig. annual GDP time seriesthat determines the structure of the SERIES and its latent components.

8 Univariate andmultivariate analyses of the GDP data can be objectives of time SERIES analysis include monitoring a time SERIES inorder to detect possible on-line changes. This is important for control purposes inengineering, industrial and medical applications. For instance, consider a time seriesgenerated from the process{yt}withyt=( 1+ (1)t, yt 1> (M1) 1+ (2)t, yt 1 (M2),( )where (1)t N(0,v1), (2)t N(0,v2)andv1=v2= 1. Figure (a) shows atime SERIES plot of 1,500 observations simulated accordingto ( ). Figure (b)displays the values of an indicator variable, t, such that t= 1ifytwas generatedfromM1and t= 2ifytwas generated fromM2. The model ( ) belongs to theclass of so called threshold autoregressive (TAR) models, initially developed by (Tong, 1983; Tong, 1990). In particular, ( ) is a TAR model with two regimes,and so, it can be written in the following, more general, formyt=( (1)yt 1+ (1)t, +yt d>0 (M1) (2)yt 1+ (2)t, +yt d 0 (M2),( )with (1)t N(0,v1)and (2)t N(0,v2).))

9 These are non-linear models and theinterest lies in making INFERENCE aboutd, and the parameters (1), (2), AREAS, GRAPHICAL DISPLAYS AND OBJECTIVES7050010001500 40246time(a) (b)Fig. (a): Simulated time seriesyt; (b) Indicator variable tsuch that t= 1ifytwas asampled fromM1and t= 2ifytwas sampled ( ) serves the purpose of illustrating, at least fora very simple case,a situation that arises in many engineering applications, particularly in the area ofcontrol theory. From a control theory viewpoint we can thinkof model ( ) as abimodal process in which two scenarios of operation are handled by two controlmodes (M1andM2). In each mode the evolution is governed by a stochastic of order one, or AR(1) models (a formal definition of this typeof processes is given later in this chapter), were chosen in this example, but moresophisticated structures can be considered.

10 The transitions between the modes occurwhen the SERIES crosses a specific threshold and so, we can talk about an internally-triggered mode switch. In an externally-triggered mode switch the moves are definedby external terms of the goals of a time SERIES analysis we can considertwo possiblescenarios. In many control settings where the transitions between modes occur inresponse to controller s actions, the current state is always known. In this settingwe can split the learning process in two: learning the stochastic models that controleach mode conditional on the fact that we know in which mode weare, , inferring (1), (2),v1andv2, and learning the transition rule, that is, making inferences aboutdand assuming we know the values 1:n. In other control settings, where themode transitions do not occur in response to the controller s actions, it is necessary tosimultaneously infer the parameters associated to the stochastic models that describeeach mode and the transition rule.