University of Toronto

1 probability and StatisticsThe Science of UncertaintySecond EditionMichael J. Evans and Je rey S. RosenthalUniversity of TorontoContentsPrefaceix1 probability probability : A Measure of Uncertainty .. Why Do We Need probability Theory?.. probability VennDiagramsandSubsets .. Properties of probability Models .. Uniform probability on Finite Spaces .. Conditional probability and Independence .. Conditional probability .. IndependenceofEvents .. Continuity FurtherProofs(Advanced) .. 312 Random Variables and CdfsofDiscreteDistributions .. CdfsofAbsolutelyContinuousDistributions .. Distributions Neither discrete Nor Continuous (Advanced).

2 One-DimensionalChangeofVariable .. TheDiscreteCase .. Joint probability Functions .. JointDensityFunctions .. ConditioningandIndependence .. Conditioning on discrete Random Variables .. Conditioning on Continuous Random Variables .. IndependenceofRandomVariables .. Multidimensional Change of TheDiscreteCase .. TheContinuousCase(Advanced) .. Simulating probability Distributions .. SimulatingDiscreteDistributions .. (Advanced) .. 1253 Variance,Covariance,andCorrelation .. CharacteristicFunctions(Advanced).. ConditionalExpectation .. Conditional Variance (Advanced) .. Inequalities .. Jensen sInequality(Advanced).

3 GeneralExpectations(Advanced) .. FurtherProofs(Advanced) .. 1944 Sampling Distributions and SamplingDistributions .. Convergence in probability .. Convergence with probability 1 .. TheStrongLawofLargeNumbers .. ConvergenceinDistribution .. TheCentralLimitTheorem .. The Central Limit Theorem and Assessing Error .. NormalDistributionTheory .. TheChi-SquaredDistribution .. FurtherProofs(Advanced) .. 2465 Statistical WhyDoWeNeedStatistics?.. Inference Using a probability Model .. FinitePopulations .. SimpleRandomSampling .. SomeBasicInferences .. PlottingData .. TypesofInference .. 2896 Likelihood TheLikelihoodFunction .. Maximum Likelihood Estimation.

4 The Multidimensional Case (Advanced).. StandardErrors,Bias,andConsistency .. ConfidenceIntervals .. TestingHypothesesandP-Values .. Sample-Size Calculations: Sample-SizeCalculations:Power .. distribution -FreeMethods .. MethodofMoments .. The Sign Statistic and Inferences about Quantiles .. LargeSampleBehavioroftheMLE(Advanced).. 3647 Bayesian Sampling from the Posterior Using Gibbs Sampling(Advanced).. ConjugatePriors .. EmpiricalBayes .. FurtherProofs(Advanced) .. Derivation of the Posterior distribution for the Location-ScaleNormalModel .. Derivation ofJ( ( 0, ))for the Location-Scale Normal .. 4318 Optimal Optimal Unbiased Estimation.

5 The Rao Blackwell Theorem and Rao Blackwellization .. Completeness and the Lehmann Scheff Theorem .. The Cramer Rao Inequality (Advanced) .. Optimal Hypothesis Testing .. ThePowerFunctionofaTest .. TheNeyman PearsonTheorem .. LikelihoodRatioTests(Advanced) .. Optimal Bayesian Inferences .. DecisionTheory(Advanced).. FurtherProofs(Advanced) .. 4739 Model CheckingtheSamplingModel .. Residual and probability PredictionandCross-Validation .. WhatDoWeDoWhenaModelFails?.. Checking for Prior Data The Problem with Multiple Checks .. 50910 Relationships Among .. The Cause EffectRelationshipsandExperiments .. RandomPredictor .. BayesianFormulation.

6 Quantitative Response and Predictors .. TheMethodofLeastSquares .. TheSimpleLinearRegressionModel .. BayesianSimpleLinearModel(Advanced).. The Multiple Linear Regression Model (Advanced) .. Quantitative Response and Categorical Predictors .. OneCategoricalPredictor(One-WayANOVA) .. RepeatedMeasures(PairedComparisons).. TwoCategoricalPredictors(Two-WayANOVA) .. RandomizedBlocks .. One Categorical and One Quantitative Categorical Response and Quantitative Predictors .. (Advanced) .. 60711 Advanced Topic Stochastic TheDistributionoftheFortune .. TheGambler sRuinProblem .. StationaryDistributions .. Markov Chain Limit Theorem .. TheMetropolis .. Definition of a Martingale.

7 ExpectedValues .. Brownian Motion as a Limit .. DiffusionsandStockPrices .. 668 Appendices675A Mathematical Derivatives .. Matrix Multiplication .. MultivariableIntegrals .. 679B 699C Common 706D Chi-Squared distribution Quantiles .. Quantiles .. Binomial distribution Probabilities .. 724E Answers to Odd-Numbered Exercises729 Index750 PrefaceThis book is an introductory text on probability and statistics, targeting students whohave studied one year of calculus at the University level and are seeking an introductionto probability and statistics with mathematical content. Where possible, we providemathematical details, and it is expected that students are seeking to gain some masteryover these, as well as to learn how to conduct data analyses.

8 All the usual method-ologies covered in a typical introductory course are introduced, as well as some of thetheory that serves as their text can be used with or without a statistical computer package. It is our opin-ion that students should see the importance of various computational techniques inapplications, and the book attempts to do this. Accordingly, we feel that computationalaspects of the subject, such as Monte Carlo, should be covered, even if a statisticalpackage is not used. Almost any statistical package is suitable. AComputationsappendix provides an introduction to the R language. This covers all aspects of thelanguage needed to do the computations in the text. Furthermore, we have providedthe R code for any of the more complicated computations.

9 Students can use theseexamples as templates for problems that involve such computations, for example, us-ing Gibbs sampling. Also, we have provided, in a separate section of this appendix,Minitab code for those computations that are slightly involved, , Gibbs programming experience is required of students to do the have organized the exercises in the book into groups, as an aid to suitable for all students and offer practice in applying the concepts discussedin a particular greater understanding, and a student can ex-pect to spend more thinking time on these. If a problem is marked (MV), then it willrequire some facility with multivariable calculus beyond thefirst calculus course, al-though these problems are not necessarily problems that moststudents willfind difficult;these are only for students who have no trouble with theExercisesand theProblems.

10 There are alsoComputer ExercisesandComputerProblems, where it is expected that students will make use of a statistical package inderiving have included a number ofDiscussion Topicsdesigned to promote criticalthinking in students. Throughout the book, we try to point students beyond the masteryof technicalities to think of the subject in a larger frame of reference. It is important thatstudents acquire a sound mathematical foundation in the basic techniques of probabilityand statistics, which we believe this book will help students accomplish. Ultimately,however, these subjects are applied in real-world contexts, so it is equally importantthat students understand how to go about their application and understand what issuesarise.