Introduction to Time Series Analysis. Lecture 1.

1 Introduction to Time Series Analysis. Lecture 1. Peter Bartlett 1. Organizational issues. 2. Objectives of time Series analysis. Examples. 3. Overview of the course. 4. Time Series models. 5. Time Series modelling: Chasing stationarity. 1. Organizational Issues Peter Bartlett. Office hours: Thu 1:30-2:30 (Evans 399). Fri 3-4 (Soda 527). Brad Luen. Office hours: Tue/Wed 2-3pm (Room TBA). bartlett/courses/153-fall2005/. Check it for announcements, assignments, slides, .. Text: Time Series Analysis and its Applications, Shumway and Stoffer. 2. Organizational Issues Computer Labs: Wed 12 1 and Wed 2 3, in 342 Evans. You need to choose one of these times. Please email bradluen@stat with your preference. First computer lab sections are on September 7. Classroom Lab Section: Fri 12 1, in 330 Evans.

2 First classroom lab section is on September 2. Assessment: Lab/Homework Assignments (40%): posted on the website. These involve a mix of pen-and-paper and computer exercises. You may use any programming language you choose (R, Splus, Matlab). The last assignment will involve analysis of a data set that you choose. Midterm Exam (25%): scheduled for October 20, at the Lecture . Final Exam (35%): scheduled for Thursday, December 15. 3. A Time Series SP500: 1960 1990. 400. 350. 300. 250. 200. $. 150. 100. 50. 0. 1960 1965 1970 1975 1980 1985 1990. year 4. A Time Series SP500: Jan Jun 1987. 340. 320. 300. 280. $. 260. 240. 220. 1987 year 5. A Time Series SP500 Jan Jun 1987. Histogram 30. 25. 20. 15. 10. 5. 0. 240 250 260 270 280 290 300 310. $. 6. A Time Series SP500: Jan Jun 1987.

3 Permuted. 340. 320. 300. 280. $. 260. 240. 220. 0 20 40 60 80 100 120. 7. Objectives of Time Series Analysis 1. Compact description of data. 2. Interpretation. 3. Forecasting. 4. Control. 5. Hypothesis testing. 6. Simulation. 8. Classical decomposition: An example Monthly sales for a souvenir shop at a beach resort town in Queensland. (Makridakis, Wheelwright and Hyndman, 1998). 4. x 10. 12. 10. 8. 6. 4. 2. 0. 0 10 20 30 40 50 60 70 80 90. 9. Transformed data 12. 11. 10. 9. 8. 7. 0 10 20 30 40 50 60 70 80 90. 10. Trend 12. 11. 10. 9. 8. 7. 0 10 20 30 40 50 60 70 80 90. 11. Residuals 1. 0. 1. 0 10 20 30 40 50 60 70 80 90. 12. Trend and seasonal variation 12. 11. 10. 9. 8. 7. 0 10 20 30 40 50 60 70 80 90. 13. Objectives of Time Series Analysis 1. Compact description of data.

4 Example: Classical decomposition: X t = Tt + St + Yt . 2. Interpretation. Example: Seasonal adjustment. 3. Forecasting. Example: Predict sales. 4. Control. 5. Hypothesis testing. 6. Simulation. 14. Unemployment data Monthly number of unemployed people in Australia. (Hipel and McLeod, 1994). 5. x 10. 8. 7. 6. 5. 4. 1983 1984 1985 1986 1987 1988 1989 1990. 15. Trend 5. x 10. 8. 7. 6. 5. 4. 1983 1984 1985 1986 1987 1988 1989 1990. 16. Trend plus seasonal variation 5. x 10. 8. 7. 6. 5. 4. 1983 1984 1985 1986 1987 1988 1989 1990. 17. Residuals 4. x 10. 8. 6. 4. 2. 0. 2. 4. 6. 1983 1984 1985 1986 1987 1988 1989 1990. 18. Predictions based on a (simulated) variable 5. x 10. 8. 7. 6. 5. 4. 1983 1984 1985 1986 1987 1988 1989 1990. 19. Objectives of Time Series Analysis 1. Compact description of data: Xt = Tt + St + f (Yt ) + Wt.

5 2. Interpretation. Example: Seasonal adjustment. 3. Forecasting. Example: Predict unemployment. 4. Control. Example: Impact of monetary policy on unemployment. 5. Hypothesis testing. Example: Global warming. 6. Simulation. Example: Estimate probability of catastrophic events. 20. Overview of the Course 1. Time Series models (a) Stationarity. (b) Autocorrelation function. (c) Transforming to stationarity. 2. Time domain methods 3. Spectral analysis 4. State space models(?). 21. Overview of the Course 1. Time Series models 2. Time domain methods (a) AR/MA/ARMA models. (b) ACF and partial autocorrelation function. (c) Forecasting (d) Parameter estimation (e) ARIMA models/seasonal ARIMA models 3. Spectral analysis 4. State space models(?). 22. Overview of the Course 1. Time Series models 2.

6 Time domain methods 3. Spectral analysis (a) Spectral density (b) Periodogram (c) Spectral estimation 4. State space models(?). 23. Overview of the Course 1. Time Series models 2. Time domain methods 3. Spectral analysis 4. State space models(?). (a) ARMAX models. (b) Forecasting, Kalman filter. (c) Parameter estimation. 24. Time Series Models A time Series model specifies the joint distribution of the se- quence {Xt } of random variables. For example: P [X1 x1 , .. , Xt xt ] for all t and x1 , .. , xt . Notation: X1 , X2 , .. is a stochastic process. x1 , x2 , .. is a single realization. We'll mostly restrict our attention to second-order properties only: EXt , E(Xt1 Xt2 ). 25. Time Series Models Example: White noise: Xt W N (0, 2 ). , {Xt } uncorrelated, EXt = 0, VarXt = 2.

7 Example: noise: {Xt } independent and identically distributed. P [X1 x1 , .. , Xt xt ] = P [X1 x1 ] P [Xt xt ]. Not interesting for forecasting: P [Xt xt |X1 , .. , Xt 1 ] = P [Xt xt ]. 26. Gaussian white noise Z xt 1 x2 /2. P [Xt xt ] = (xt ) = e dx. 2 . 2. 1. 0. 1. 2. 0 5 10 15 20 25 30 35 40 45 50. 27. Gaussian white noise 2. 1. 0. 1. 2. 0 5 10 15 20 25 30 35 40 45 50. 28. Time Series Models Example: Binary P [Xt = 1] = P [Xt = 1] = 1/2. 1. 0. 1. 0 5 10 15 20 25 30 35 40 45 50. 29. Random walk Pt St = i=1 Xi . Differences: St = St St 1 = Xt . 8. 6. 4. 2. 0. 2. 4. 0 5 10 15 20 25 30 35 40 45 50. 30. Random walk ESt ? VarSt ? 10. 5. 0. 5. 10. 15. 0 5 10 15 20 25 30 35 40 45 50. 31. Random Walk Recall S&P500 data. (Notice that it's smooth). SP500: Jan Jun 1987. 340. 320. 300.

8 280. $. 260. 240. 220. 1987 year 32. Random Walk Differences: St = St St 1 = Xt . SP500, Jan Jun 1987. first differences 10. 8. 6. 4. 2. 0. $. 2. 4. 6. 8. 10. 1987 year 33. Trend and Seasonal Models P. Xt = T t + S t + E t = 0 + 1 t + i ( i cos( i t) + i sin( i t)) + Et 6. 5. 4. 3. 0 50 100 150 200 250. 34. Trend and Seasonal Models Xt = T t + E t = 0 + 1 t + E t 6. 5. 4. 3. 0 50 100 150 200 250. 35. Trend and Seasonal Models P. Xt = T t + S t + E t = 0 + 1 t + i ( i cos( i t) + i sin( i t)) + Et 6. 5. 4. 3. 0 50 100 150 200 250. 36. Trend and Seasonal Models: Residuals 0. 0 50 100 150 200 250. 37. Time Series Modelling 1. Plot the time Series . Look for trends, seasonal components, step changes, outliers. 2. Transform data so that residuals are stationary. (a) Estimate and subtract Tt , St.

9 (b) Differencing.. (c) Nonlinear transformations (log, ). 3. Fit model to residuals. 38. Nonlinear transformations Recall: Monthly sales. (Makridakis, Wheelwright and Hyndman, 1998). 4. x 10 12. 12. 10 11. 8. 10. 6. 9. 4. 8. 2. 0 7. 0 10 20 30 40 50 60 70 80 90 0 10 20 30 40 50 60 70 80 90. 39. Differencing Recall: S&P 500 data. SP500: Jan Jun 1987 SP500, Jan Jun 1987. first differences 340 10. 8. 320. 6. 4. 300. 2. 280 0. $. $. 2. 260. 4. 6. 240. 8. 220 10. 1987 1987 year year 40. Differencing and Trend Define the lag-1 difference operator, (think first derivative'). Xt = Xt Xt 1 = (1 B)Xt , where B is the backshift operator, BXt = Xt 1 . If Xt = 0 + 1 t + Yt , then Xt = 1 + Yt . Pk If Xt = i=0 i ti + Yt , then k Xt = k! k + k Yt , where k Xt = ( k 1 Xt ) and 1 Xt = Xt . 41.

10 Differencing and Seasonal Variation Define the lag-s difference operator, s Xt = Xt Xt s = (1 B s )Xt , where B s is the backshift operator applied s times, B s Xt = B(B s 1 Xt ). and B 1 Xt = BXt . If Xt = Tt + St + Yt , and St has period s (that is, St = St s for all t), then s Xt = Tt Tt s + s Yt . 42. Least Squares Regression Model: X t = 0 + 1 t + W t . 0. = 1 t + Wt , 1.. X 1 1 W1. 1 .. X2 1 2 W2 . = 0 + .. 1 .. | {z } . XT 1 T WT. | {z } | {z } | {z }. x Z w 43. Least Squares Regression x = Z + w. Least squares: choose to minimize kwk2 = kx Z k2 . Solution satisfies the normal equations: kwk2 = 2Z 0 (x Z ) = 0. If Z 0 Z is nonsingular, the solution is unique: = (Z 0 Z) 1 Z 0 x. 44. Least Squares Regression Properties of the least squares solution ( = (Z 0 Z) 1 Z 0 x): Linear.