PROBABILITY AND STATISTICS FOR ECONOMISTS

1 PROBABILITYAND STATISTICSFOR ECONOMISTSBRUCE E. HANSENC ontentsPrefacexAcknowledgementsxiMathema tical PreparationxiiNotationxiii1 Basic PROBABILITY .. and Events .. Function.. of the PROBABILITY Function .. Outcomes .. Events .. PROBABILITY .. of Total PROBABILITY .. Rule.. and Combinations .. With and Without Replacement .. Hands .. Fields* .. Proofs* ..182 Random .. Variables .. Random Variables .. of Expectations .. Function .. Random Variables .. Functions .. of Continuous Random Variables.. Transformations .. of Continuous Random Variables .. of Expectations .. Notation .. and Variance .. s Inequality .. of Jensen s Inequality* .. Distributions .. Distributions .. Distributions .. Generating Function .. Function .. : Mathematical Details*..523 Parametric .. Distribution .. Distribution.. Distribution .. Distribution .. Distribution.. Binomial Distribution .. Distribution .. Distribution.

2 Exponential Distribution .. Exponential Distribution .. Distribution .. Distribution .. t Distribution .. Distribution.. Distribution .. Distribution .. Distribution .. Chi-Square .. Distribution.. Distribution .. Distribution.. Distribution .. Value Distribution.. of Normals .. Proofs* ..714 Multivariate .. Random Variables .. Distribution Functions .. Mass Function .. Density Function.. Distribution .. Expectation .. Distribution for DiscreteX.. Distribution for ContinuousX.. Conditional Densities .. and Correlation .. Expectation .. of Iterated Expectations .. Variance .. lder s and Minkowski s Inequalities*.. Notation .. Inequalities* .. Random Vectors .. of Multivariate Vectors .. Transformations .. Distributions .. and Uniqueness of the Conditional Expectation* .. 1095 Normal and Related .. Normal .. of the Normal Distribution .. Cumulants.

3 Quantiles .. and Censored Normal Distributions .. Normal .. of the Multivariate Normal .. , t, F, and Cauchy Distributions .. Polynomials* .. Proofs* .. Illustration .. , Parameters, Estimators .. Mean .. Value of Transformations .. of Parameters .. Distribution.. Bias.. Variance .. Squared Error .. Unbiased Estimator .. of Variance .. Error .. Means .. STATISTICS .. Moments of Sample Mean* .. Sampling Model .. Residuals .. Variance Estimation .. Ratio .. Normal Sampling .. 1467 Law of Large .. Limits .. in PROBABILITY .. s Inequality .. Law of Large Numbers .. Chebyshev s.. Moments .. Mapping Theorem .. Over Distributions* .. Sure Convergence and the Strong Law* .. Proofs* .. 1628 Central Limit .. in Distribution .. Mean .. Moment Investigation .. of the Moment Generating Function.

4 Limit Theorem .. the Central Limit Theorem .. Central Limit Theorem .. Method .. Distribution for Plug-In Estimator .. Matrix Estimation .. Order Symbols.. Proofs* .. 1769 Advanced Asymptotic Theory* .. Central Limit Theory .. Heterogeneous CLTs .. CLT .. Integrability .. Stochastic Bounds .. of Moments .. Expansion for the Sample Mean .. Expansion for Smooth Function Model.. Expansions .. Proofs .. 18810 Maximum Likelihood .. Model .. Analog Principle .. Property .. , Hessian, and Information.. r-Rao Lower Bound .. Examples .. Cram r-Rao Bound for Functions of Parameters .. Consistent Estimation.. Asymptotic Normality .. Asymptotic Cram r-Rao Efficiency .. Variance Estimation .. Kullback-Leibler Divergence .. Approximating Models .. Distribution of the MLE under Mis-Specification.

5 Variance Estimation under Mis-Specification .. Technical Proofs* .. Exercises .. 22111 Method of .. Means .. Functions .. Moments .. Unbiased Estimation .. Models .. of Parametric Models .. Equations .. Asymptotic Distribution for Moment Equations .. Example: Euler Equation .. Empirical Distribution Function .. Sample Quantiles .. Robust Variance Estimation .. Technical Proofs* .. Exercises .. 24612 Numerical .. Function Evaluation and Differentiation .. Finding .. in One Dimension .. of Minimization .. in Multiple Dimensions .. Optimization .. Minimization .. and Tricks .. Exercises .. 26913 Hypothesis .. and Rejection .. I and II Error .. Tests .. Tests .. Does AcceptH0 Mean AboutH0? .. Test with Normal Sampling .. t-test .. Likelihood Ratio Test for Simple Hypotheses.

6 Neyman-Pearson Lemma.. Likelihood Ratio Test Against Composite Alternatives .. Likelihood Ratio and t tests.. Statistical Significance .. P-Value .. Composite Null Hypothesis .. Asymptotic Uniformity .. Summary .. Exercises .. 29314 Confidence .. Confidence Intervals .. Intervals for the Sample Mean under Normal Sampling.. Intervals for the Sample Mean under non-Normal Sampling.. Intervals for Estimated Parameters .. Interval for the Variance .. Intervals by Test Inversion .. of Confidence Intervals .. Uniform Confidence Intervals .. Exercises .. 30315 Shrinkage .. Squared Error .. Shrinkage Estimator .. Calculation .. of the Stein Effect.. Part Estimator .. Proofs* .. Exercises .. 31416 Bayesian .. PROBABILITY Model .. Density.. Estimation .. Priors .. Distribution.

7 Prior .. Sampling .. Sampling .. Credible Sets .. Bayesian Hypothesis Testing .. Sampling Properties in the Normal Model .. Asymptotic Distribution .. Exercises .. 33417 Nonparametric Density .. Density Estimation .. Density Estimator .. of Density Estimator .. of Density Estimator .. Estimation and Standard Errors .. of Density Estimator .. Kernel .. Bandwidth .. Sheather-Jones Bandwidth* .. Recommendations for Bandwidth Selection.. Practical Issues in Density Estimation .. Computation .. Asymptotic Distribution .. Undersmoothing .. Technical Proofs* .. Exercises .. 35518 Empirical Process .. Theorem.. , Covering, and Bracketing Numbers .. Law of Large Numbers .. Central Limit Theory .. for Asymptotic Equicontinuity.. s Theorem .. Proofs* .. Exercises.

8 371 AMathematics .. Value Theorem .. Integral .. Function .. Algebra .. 379 References382 PrefaceThis textbook is the first in a two-part series covering the core material typically taught in a course in econometrics. The sequence and STATISTICS for ECONOMISTS (this volume) (the next volume)The textbooks are written as an integrated series, but either can be used as a stand-alone first volume covers intermediate-level mathematical STATISTICS . It is a gentle yet a rigorous treat-ment using calculus but not measure theory. The level of detail and rigor is similar to that of Casellaand Berger (2002) and Hogg and Craig (1995). The material is explained using examples at the level ofHogg and Tanis (1997), targeted to students of economics. The goal is to be accessible to students with avariety of backgrounds yet attain full mathematical who desire a gentler treatment may try Hogg and Tanis (1997). Readers who desire moredetail are recommended to read Casella and Berger (2002) or Shao (2003).

9 Readers wanting a measure-theoretic foundation in PROBABILITY are recommended to read Ash (1972) or Billingsley (1995). For ad-vanced statistical theory see van der Vaart (1998), Lehmann and Casella (1998), and Lehmann and Ro-mano (2005), each of which has a different emphasis. Mathematical STATISTICS textbooks with similargoals as this textbook include Ramanathan (1993), Amemiya (1994), Gallant (1997), and Linton (2017).Technical material which is not essential for the main concepts are presented in the starred (*) sec-tions. This material is intended for students interested in the mathematical details. Others may skipthese sections with no loss of 1-5 cover PROBABILITY theory. Chapters 6-18 cover statistical end-of-chapter exercises are important parts of the text and are central for textbook could be used for a one-semester course. It can also be used for a one-quarter course(as done at the University of Wisconsin) if a selection of topics are skipped.

10 For example, the material inChapter 3 should probably be viewed as reference rather than taught; Chapter 9 is for advanced students;Chapter 11 can be covered in brief; Chapter 12 can be left for reference; and Chapters 15-18 are optionaldepending on the book and its companionEconometricswould not have been possible if it were not for the amaz-ing flow of unsolicited advice, corrections, comments, and questions I have received from students, fac-ulty, and other readers over the twenty years I have worked on this project. I have received emails cor-rections and comments from so many individuals I have completely lost track of the list. So rather thanpublish an incomplete list, I simply give an honest and thoroughThank Youto every single thanks go to Xiaoxia Shi, who typed up my handwritten notes for Econ 709 a few years ago,creating a preliminary draft for this most heartfelt thanks goes to my family: Korinna, Zoe, and Nicholas. Without their love andsupport over these years this project would not have been of the author s royalties will be re-gifted to charitable PreparationStudents should be familiar with integral, differential, and multivariate calculus, as well as linear ma-trix algebra.