Time Series Analysis - Auckland

1 time Series AnalysisLecture Notes for IhakaStatistics DepartmentUniversity of AucklandApril 14, 2005iiContents1 time Series .. Stationarity and Non-Stationarity .. Some Examples .. Annual Auckland Rainfall .. Nile River Flow .. Yield on British Government Securities .. Ground Displacement in an Earthquake .. United States Housing Starts .. Iowa City Bus Ridership .. 32 Vector Space Vectors In Two Dimensions .. Scalar Multiplication and Addition .. Norms and Inner Products .. General Vector Spaces .. Vector Spaces and Inner Products .. Some Examples .. Hilbert Spaces .. Subspaces .. Projections .. Hilbert Spaces and Prediction .. Linear Prediction .. General Prediction .. 153 time Series time Series .. Hilbert Spaces and Stationary time Series .. The Lag and Differencing Operators .. Linear Processes .. Autoregressive Series .

2 The AR(1) Series .. The AR(2) Series .. Computations .. Moving Average Series .. The MA(1) Series .. Invertibility .. Computation .. Autoregressive Moving Average Series .. The ARMA(1,1) Series .. The ARMA(p,q) Model .. Computation .. Common Factors .. The Partial Autocorrelation Function .. Computing the PACF .. Computation .. 374 Identifying time Series ACF Estimation .. PACF Estimation .. System Identification .. Model Generalisation .. Non-Zero Means .. Deterministic Trends .. Models With Non-stationary AR Components .. The Effect of Differencing .. ARIMA Models .. 495 Fitting and Model Fitting .. Computations .. Assessing Quality of Fit .. Residual Correlations .. Forecasting .. Computation .. Seasonal Models .. Purely Seasonal Models .. Models with Short-Term and Seasonal Components .. A More Complex Example.

3 666 Frequency Domain Some Background .. Complex Exponentials, Sines and Cosines .. Properties of Cosinusoids .. Frequency and Angular Frequency .. Invariance and Complex Exponentials .. Filters and Filtering .. Filters .. Transfer Functions .. Filtering Sines and Cosines .. Filtering General Series .. Computing Transfer Functions .. Sequential Filtering .. Spectral Theory .. The Power Spectrum .. The Cram er Representation .. Using The Cram er Representation .. Power Spectrum Examples .. Statistical Inference .. Some Distribution Theory .. The Periodogram and its Distribution .. An Example Sunspot Numbers .. Estimating The Power Spectrum .. Tapering and Prewhitening .. Cross Spectral Analysis .. Computation .. A Simple Spectral Analysis Package for R .. Power Spectrum Estimation .. Cross-Spectral Analysis .. Examples.

4 100viContentsChapter time SeriesTime seriesarise as recordings of processes which vary over time . A recordingcan either be a continuous trace or a set of discrete observations. We willconcentrate on the case where observations are made at discrete equally spacedtimes. By appropriate choice of origin and scale we can take the observationtimes to be 1, 2, ..Tand we can denote the observations byY1,Y2, .. , are a number of things which are of interest in time Series most important of these are:Smoothing: The observedYtare assumed to be the result of noise values tadditively contaminating a smooth signal t+ tWe may wish to recover the values of the underlying : We may wish to develop a simple mathematical model whichexplains the observed pattern ofY1,Y2, .. ,YT. This model may dependon unknown parameters and these will need to be : On the basis of observationsY1,Y2, .. ,YT, we may wish topredict what the value ofYT+Lwill be (L 1), and possibly to give anindication of what the uncetainty is in the : We may wish to intervene with the process which is producing theYtvalues in such a way that the future values are altered to produce afavourable Stationarity and Non-StationarityA key idea in time Series is that ofstationarity.

5 Roughly speaking, a timeseries is stationary if its behaviour does not change over time . This means, forexample, that the values always tend to vary about the same level and thattheir variability is constant over time . Stationary Series have a rich theory and12 Chapter 1. Introductiontheir behaviour is well understood. This means that they play a fundamentalrole in the study of time , not all time Series that we encouter are stationary. Indeed, non-stationary Series tend to be the rule rather than the exception. However, manytime Series are related in simple ways to Series which are stationary. Two im-portant examples of this are:Trend models: The Series we observe is the sum of a determinstictrendseries and a stationarynoiseseries. A simple example is the linear trendmodel:Yt= 0+ 1t+ common trend model assumes that the Series is the sum of aperiodic seasonal effect and stationarynoise. There are many models: The time Series we observe satisfiesYt+1 Yt= t+1where tis a stationary Series .

6 A particularly important model of this kindis therandom walk. In that case, the tvalues are independent shocks which perturb the current stateYtby an amount t+1to produce a newstateYt+ Some Annual Auckland RainfallFigure shows the annual amount of rainfall in Auckland for the years from1949 to 2000. The general pattern of rainfall looks similar throughout the record,so this Series could be regarded as being stationary. (There is a hint that rainfallamounts are declining over time , but this type of effect can occur over shortishtime spans for stationary Series .) Nile River FlowFigure shows the flow volume of the Nile at Aswan from 1871 to 1970. Theseare yearly values. The general pattern of this data does not change over timeso it can be regarded as stationary (at least over this time period). Yield on British Government SecuritiesFigure shows the percentage yield on British Government securities, monthlyover a 21 year period. There is a steady long-term increase in the yields.

7 Overthe period of observation a trend-plus-stationary Series model looks like it mightbe appropriate. An integrated stationary Series is another Some Ground Displacement in an EarthquakeFigure shows one component of the horizontal ground motion resulting froman earthquake. The initial motion (a little after 4 seconds) corresponds to thearrival of thep-wave and the large spike just before six seconds corresponds tothe arrival of thes-wave. Later features correspond to the arrival of surfacewaves. This is an example of a transient signal and cannot have techniquesappropriate for stationary Series applied to United States Housing StartsFigure shows the monthly number of housing starts in the Unites States (inthousands). Housing starts are a leading economic indicator. This means thatan increase in the number of housing starts indicates that economic growth islikely to follow and a decline in housing starts indicates that a recession may beon the Iowa City Bus RidershipFigure shows the monthly average weekday bus ridership for Iowa City overthe period from September 1971 to December 1982.

8 There is clearly a strongseasonal effect suprimposed on top of a general upward 1. IntroductionYearAnnual Rainfall (cm)195019601970198019902000801001201401 60180200 Figure : Annual Auckland rainfall (in cm) from 1949 to 2000 (fromPaul Cowpertwait).YearFlow1880190019201940196 0600800100012001400 Figure : Flow volume of the Nile at Aswan from 1871 to 1970 (fromDurbin and Koopman). Some Examples5 MonthPercent0501001502002502468 Figure : Monthly percentage yield on British Government securitiesover a 21 year period (from Chatfield).SecondsDisplacement024681012 1000100200300400500 Figure : Horizontal ground displacement during a small Nevadaearthquake (from Bill Peppin).6 Chapter 1. IntroductionTimeHousing Starts (000s)1966196819701972197450100150200 Figure : Housing starts in the United States (000s) (from S-Plus).TimeAverage Weekly Ridership1972197419761978198019824000600 0800010000 Figure : Average weekday bus ridership, Iowa City (monthly ave)Sep 1971 - Dec 2 Vector Space Vectors In Two DimensionsThe theory which underlies time Series Analysis is quite technical in spite of this, a good deal of intuition can be developed by approaching thesubject geometrically.

9 The geometric approach is based on the ideas ofvectorsandvector Scalar Multiplication and AdditionA good deal can be learnt about the theory of vectors by considering the two-dimensional case. You can think of two dimensional vectors as being littlearrows which have a length and a direction. Individual vectors can be stretched(altering their length but not their direction) and pairs of vectors can be addedby placing them head to get more precise, we ll suppose that a vectorvextends for a distancexin the horizontal direction and a distanceyin the vertical direction. This givesus a representation of the vector as a pair of numbers and we can write it asv= (x,y).Doubling the length of the vector doubles thexandyvalues. In a similarway we can scale the length of the vector by any value, and this results in asimilar scaling of thexandyvalues. This gives us a natural way of definingmultiplication of a vector by a (cx,cy)A negative value forcproduces a reversal of direction as well as change of lengthof magnitude|c|.

10 Adding two vectors amounts to placing them head to tail and taking the sumto be the arrow which extends from the base of the first to the head of the corresponds to adding thexandyvalues corresponding to the vectors. Fortwo vectorsv1= (x1,y1) andv2= (x2,y2) the result is (x1+x2,y1+y2). Thiscorresponds to a natural definition of addition for +v2= (x1+x2,y1+y2)78 Chapter 2. Vector Space Norms and Inner ProductsWhile coordinates give a complete description of vectors, it is often more usefulto describe them in terms of the lengths of individual vectors and the anglesbetween pairs of vectors. If we denote the length of a vector by u , then byPythagoras theorem we know u = x2+ length has a number of simple properties:Positivity:For every vectoru, we must have u >0, with equality if andonly ifu= :For every scalarcand vectoruwe have cu =|c| u .The Triangle Inequality:Ifuandvare vectors, then u+v 6 u + v .The first and second of these properties are obvious, and the third is simplya statement that the shortest distance between two points is a straight technical mathematical name for the length of a vector is thenormof the length of an individual vector gives some information aboutit, it is also important to consider the angles between pairs of vectors.