Regression III: Advanced Methods Syllabus - Summer 2018

1 1 Regression III: Advanced MethodsSyllabus - Summer 2018 Instructor:Dave ArmstrongCanada Research Chair in Political MethodologyDepartment of Political ScienceDepartment of Statistics and Actuarial Sciences (by courtesy)University of Western Hours: 1:30PM-2:30PM M-F (or by appointment)Course website: AssistantsNick DavisChris SchwarzPolitical SciencePolitical ScienceUniversity of Wisconsin-Milwaukee Overview and Course ObjectivesThe Regression III course takes a considerably different form than the first two re-gression courses at the Summer Program. This course will hopefully prepare you for thethings you will encounter when you publish quantitative work with linear models. Initiallinear model classes focus on the assumptions and theoretical considerations of linearmodels and generally walk you through estimation and interpretation. Good courses alsodeal with diagnostics, though these often get less time than they should. Further, it is notalways obvious what violations of these assumptions will lead to in practical terms.

2 Thiscourse will provide you with a systematic approach to assessing, fixing and presentingyour linear model results. Though we focus almost exclusively on the linear model (wewill allude to nonlinear models occasionally), the logic we follow will be helpful in dealingwith nonlinear models as RequirementsThis course is a practical, data-analytic extension of what you learned in your de-partment s linear models class or the Regression II class at the ICPSR Summer such, I assume you are familiar with the types of things taught in these courses -Gauss-Markov assumptions, properties of OLS estimators, and statistical inference forlinear model coefficients. While I assume this knowledge exists, we will spend reviewthese ideas briefly in class where necessary. If you are not sure where you belong in the2series of linear models courses at the Summer Program, please see me or the SummerProgram director and we will make sure you end up the most appropriate recently experimented with allowing participants to use either Stata or R and found,from student evaluations, and from both R and Stata users that the support for bothpieces of software was both distracting and unnecessary.

3 Stata users found that it wasreasonably easy to pick up R for the purposes of the course. Further, Some of the specificsoftware used in the course does not exist (or exist in the same useful way) in Stata. Ifound that the implementation in Stata often required some programming (loops, macros, ) and that was a threshold many participants did not want to cross. Thus, R will bethe only officially supported software. That said, I have (now quite old) Stata code forsome of the examples in class and would be happy to share with interested you want to use this course as an opportunity to strengthen your R skills, but havelittle familiarity with that software, you should take the R workshop taught in the firstfew weeks of the first you re one of those glutton-for-punishment types, you may also find it usefulto learn LATEX. LATEXis a system for typesetting documents. People find it most use-ful for typesetting documents that are heavy on mathematical notation, but this is justthe tip of the iceberg.

4 LATEXhas its own bibliographic software (BibTEX) and will au-tomatically build (and re-build) tables of contents, lists of figures and lists of tables. Italso automatically numbers (and re-numbers when necessary) tables, figures and equa-tions, changing appropriately formed references to those objects when table, figure orequation numbers change. Best of all, common LATEX typesetting engines are free ( links to the software appropriate foryour OS). Everything I present in class is written in LATEX; specifically, the slides areall made with a package called Beamer with R integration through knitr . Thereare some nice literate programming tools (Sweave, knitr and StatWeave) that integrateLATEX, R and Stata as well. Further, there are those who see LATEXas a sort of secrethandshake for nerds. So, if you want to be one of the cool kids, then you should def-initely try it; everyone else is doing it. There is a LATEX workshop on the first Mondaynight of the Summer Course Text(s)No one text effectively presents all of the material that will be covered in this said, much of the material is covered (and covered well) in:Fox, John.

5 (2015) Applied Regression Analysis and Generalized Linear Oaks, CA: Sage Publications, , John and Sanford Weisberg. (2011) An R Companion to Applied Oaks, CA: Sage Publications, , Gareth, Daniela Witten, Trevor Hastie and Robert Tibshirani. (2013) An Introduction to Statistical Learning with Application in York: Springer. Available for free ~gareth/ISL/ISLR%20 Sixth% The R Companionis a great book for those cur-rently learning R. I would highly recommend getting the recently updated and ex-panded second edition. This is widely recognized as one of the best ways for SocialScientists to get into will also use a number of other books and articles to deal with more specializedissues. These are listed below (along with the appropriate chapters/pages) for the classesin which we use SoftwareOne of R s main virtues from the grad-student point of view is that the base packageand all of the add-ons (called packages in R) are free. You can download the base packageof R from the Comprehensive RArchive Network (CRAN) As of this writing, the most recent version is R is updated acouple of times per year so you ll have to look back here periodically for updates.

6 Wewill be using a number of user-contributed packages that we will discuss as they Related SoftwareA good text editor is invaluable when using R and LATEX. TEXW orks is a good, freeeditor for LATEX that works in most environments, including Windows and Mac( ). RStudio is a free IDE (Integrated Development Environment)for R that includes a nicely-featured text editor ( ). If you relooking for a general purpose text editor, I currently use Microsoft s VS Code ( ), though there are other great options, too like Atom Course ScheduleEach entry represents a single topic. Readings are designated either as suggested ( )or supplemental ( ). For most of you, this is not the only class you are taking and asthe weeks fly by, your time will undoubtedly be too limited to read everything indicatedin the Syllabus . However, this should serve as a nice reference to which you can return ifthe intricacies of a particular topic have faded from your Preliminary Material (Tuesday, June 26)Readings: James et al.

7 (2013) Chapter 2 Fox (2015), Chapters 1 & 2 Fox and Weisberg (2011), Chapters 1 & 22. OLS II: Effective Presentation (Wednesday-Thursday, June 27-28)(a) Factors and contrasts; quasi-variances and graphical displays(b) Interactions and effect displays(c) Standardization and relative importanceReadings: Armstrong II (2013) Berry, Golder and Milton (2012) Armstrong II (2013) Silber, Rosenbaum and Ross (1995)3. Lab I: (Friday, June 29)(a) Factors and contrasts(b) Interactions(c) Relative Importance54. Linearity: Diagnostics, Transformations and Polynomials (Monday, July 2)(a) Diagnosing linearity through residual plots(b) Fixing non-linearity with data transformations and polynomials(c) Linearity and ordinal variables(d) Alternating Least Squares Optimal Scaling (ALSOS)Readings: Fox (2015) Chapters 4 & 12 (Sections ) Fox and Weisberg (2011) Chapter 3 Jacoby (1999) Box and Tidwell (1962)5. Re-sampling Techniques and Regression (Tuesday, July 3)(a) Bootstrapping and Jackknifing(b) Cross-validationReadings: James et al.

8 (2013) Chapter 5 Fox (2015) Chapter 21 Davison and Hinkley (1997) Stone (1974) Efron and Tibshirani (1993)6. Model Selection (Thursday, July 5)(a) Theoretical issues in model searching and post-data model construction(b) Model selection criteria and multi-model inference.(c) Subset selection modelsReadings: Fox (2015) Chapter 22 James et al. (2013) Chapter 6 Burnham and Anderson (2004) Leamer and Leonard (1983) Leamer (1983) Box (1976), Box and Hunter (1962)67. Non-Linearity, Smoothing and Splines (Friday, July 6 and Monday, July 9)(a) Nonparametric Smoothing - Lowess(b) Inference for Regression smoothers(c) Regression Splines(d) Generalized Additive ModelsReadings: Fox (2015) Chapters 17 & 18 James et al. (2013) Chapter 7 Keele (2008) Chapters 2-68. Flexible Models: Tree-based Regression , Multivariate Adaptive Regression Splines(Tuesday, July 10) Fundamentals of flexible models Automatic variable selection Inference and effects in statistical learning models When (and when not) to use these kinds of models Montgomery and Olivella (forthcoming) James et al.

9 (2013) Chapter 7 Berk (2016) Section Lab II: (Wednesday, July 11)(a) Non-linearity transformations(b) Polynomials(c) Smoothers and splines(d) Trees and other flexible methods10. Regression Discontinuity Designs (Thursday, July 12)Readings: Cattaneo, Idrobo and Titiunik (2017) Calonico, Cattaneo and Titiunik (2015) Keele (2015) Sekhon and Titiunik (2016)711. Finite Mixture Models (Friday, July 13)Readings: Imai and Tingley (Forthcoming) Gr un and Leisch (2008) Gr un and Leisch (2007)12. Missing Data and Multiple Imputation (Monday, July 16)(a) Whats the problem with missing data?(b) When can we fix it?(c) How do we impute the data and use those imputations?Readings: Fox (2015) Chapter 20 van Buuren and Groothuis-Oudshoorn (2011) Honaker and King (2010) Cranmer and Gill (2013) Akande, Li and Reiter (forthcoming) Xia and Yang (2016) Resseguier, Giorgi and Paoletti (2011) Schafer (1997) Rubin (1987)13. Lab IV (Tuesday, July 17)(a) RDD(b) Mixture Models(c) Missing Data and Multiple Imputation14.

10 Outliers and Influential Data: Diagnostics (Wednesday, July 18)(a) Outliers, leverage and influential data(b) Hat values, standardized residuals, Cook s D(c) M-estimation (and extension) and iterative re-weighted least squares(d) Diagnostics for outliers revisitedReadings: Fox (2015) Chapter 11 Fox and Weisberg (2011) Chapter 6 (pp 101-201) Andersen (2008) Fox (2015) Chapter 19 Cantoni and Ronchetti (2001) Rousseeuw and Leroy (1987) Jasso (1985, 1996), Kahn and Udry (1986)815. Non-constant error variance and collinearity: Diagnostics and Fixes (Thursday, July19)(a) Residual plots(b) ML transformations ofY(c) Weighted least squares(d) Heteroskedastic linear Regression (e) Robust standard errorsReadings: Fox (2015) Chapters 12 & 13 Fox and Weisberg (2011) Chapters 3 & 6 Long and Ervin (2000) King and Roberts (2015) Harvey (1976) Cribari-Neto (2004), Cribari-Neto, Souza and Vasconcellos (2007), Cribari-Netoand da Silva (2011)16. Critiques of the Linear Regression Model (Friday, July 20)(a) How important are the assumptions behind OLS Regression ?