PROBABILITY AND MATHEMATICAL STATISTICS

1 PROBABILITY . AND. MATHEMATICAL STATISTICS . Prasanna Sahoo Department of Mathematics University of Louisville Louisville, KY 40292 USA. v THIS BOOK IS DEDICATED TO. AMIT. SADHNA. MY PARENTS, TEACHERS. AND. STUDENTS. vi vii Copyright c 2013. All rights reserved. This book, or parts thereof, may not be reproduced in any form or by any means, electronic or mechanical, including photocopying, recording or any information storage and retrieval system now known or to be invented, without written permission from the author. viii ix PREFACE. This book is both a tutorial and a textbook. This book presents an introduc- tion to PROBABILITY and MATHEMATICAL STATISTICS and it is intended for students already having some elementary MATHEMATICAL background. It is intended for a one-year junior or senior level undergraduate or beginning graduate level course in PROBABILITY theory and MATHEMATICAL STATISTICS .

2 The book contains more material than normally would be taught in a one-year course. This should give the teacher flexibility with respect to the selection of the content and level at which the book is to be used. This book is based on over 15. years of lectures in senior level calculus based courses in PROBABILITY theory and MATHEMATICAL STATISTICS at the University of Louisville. PROBABILITY theory and MATHEMATICAL STATISTICS are difficult subjects both for students to comprehend and teachers to explain. Despite the publication of a great many textbooks in this field, each one intended to provide an im- provement over the previous textbooks, this subject is still difficult to com- prehend. A good set of examples makes these subjects easy to understand. For this reason alone I have included more than 350 completely worked out examples and over 165 illustrations.

3 I give a rigorous treatment of the fun- damentals of PROBABILITY and STATISTICS using mostly calculus. I have given great attention to the clarity of the presentation of the materials. In the text, theoretical results are presented as theorems, propositions or lemmas, of which as a rule rigorous proofs are given. For the few exceptions to this rule references are given to indicate where details can be found. This book contains over 450 problems of varying degrees of difficulty to help students master their problem solving skill. In many existing textbooks, the examples following the explanation of a topic are too few in number or too simple to obtain a through grasp of the principles involved. Often, in many books, examples are presented in abbreviated form that leaves out much material between steps, and requires that students derive the omitted materials themselves.

4 As a result, students find examples difficult to understand. Moreover, in some textbooks, examples x are often worded in a confusing manner. They do not state the problem and then present the solution. Instead, they pass through a general discussion, never revealing what is to be solved for. In this book, I give many examples to illustrate each topic. Often we provide illustrations to promote a better understanding of the topic. All examples in this book are formulated as questions and clear and concise answers are provided in step-by-step detail. There are several good books on these subjects and perhaps there is no need to bring a new one to the market. So for several years, this was circulated as a series of typeset lecture notes among my students who were preparing for the examination 110 of the Actuarial Society of America.

5 Many of my students encouraged me to formally write it as a book. Actuarial students will benefit greatly from this book. The book is written in simple English; this might be an advantage to students whose native language is not English. I cannot claim that all the materials I have written in this book are mine. I have learned the subject from many excellent books, such as Introduction to MATHEMATICAL STATISTICS by Hogg and Craig, and An Introduction to Prob- ability Theory and Its Applications by Feller. In fact, these books have had a profound impact on me, and my explanations are influenced greatly by these textbooks. If there are some similarities, then it is due to the fact that I could not make improvements on the original explanations. I am very thankful to the authors of these great textbooks.

6 I am also thankful to the Actuarial Society of America for letting me use their test problems. I thank all my students in my PROBABILITY theory and MATHEMATICAL STATISTICS courses from 1988 to 2005 who helped me in many ways to make this book possible in the present form. Lastly, if it weren't for the infinite patience of my wife, Sadhna, this book would never get out of the hard drive of my computer. The author on a Macintosh computer using TEX, the typesetting system designed by Donald Knuth, typeset the entire book. The figures were gener- ated by the author using MATHEMATICA, a system for doing mathematics designed by Wolfram Research, and MAPLE, a system for doing mathemat- ics designed by Maplesoft. The author is very thankful to the University of Louisville for providing many internal financial grants while this book was under preparation.

7 Prasanna Sahoo, Louisville xi xii TABLE OF CONTENTS. 1. PROBABILITY of Events .. 1. Introduction Counting Techniques PROBABILITY Measure Some Properties of the PROBABILITY Measure Review Exercises 2. Conditional PROBABILITY and Bayes' Theorem .. 27. Conditional PROBABILITY Bayes' Theorem Review Exercises 3. Random Variables and Distribution Functions .. 45. Introduction Distribution Functions of Discrete Variables Distribution Functions of Continuous Variables Percentile for Continuous Random Variables Review Exercises 4. Moments of Random Variables and Chebychev Inequality . 73. Moments of Random Variables Expected Value of Random Variables Variance of Random Variables Chebychev Inequality Moment Generating Functions Review Exercises xiii 5. Some Special Discrete Distributions.

8 107. Bernoulli Distribution Binomial Distribution Geometric Distribution Negative Binomial Distribution Hypergeometric Distribution Poisson Distribution Riemann Zeta Distribution Review Exercises 6. Some Special Continuous Distributions .. 141. Uniform Distribution Gamma Distribution Beta Distribution Normal Distribution Lognormal Distribution Inverse Gaussian Distribution Logistic Distribution Review Exercises 7. Two Random Variables .. 185. Bivariate Discrete Random Variables Bivariate Continuous Random Variables Conditional Distributions Independence of Random Variables Review Exercises 8. Product Moments of Bivariate Random Variables .. 213. Covariance of Bivariate Random Variables Independence of Random Variables Variance of the Linear Combination of Random Variables Correlation and Independence Moment Generating Functions Review Exercises xiv 9.

9 Conditional Expectations of Bivariate Random Variables 237. Conditional Expected Values Conditional Variance Regression Curve and Scedastic Curves Review Exercises 10. Functions of Random Variables and Their Distribution . 257. Distribution Function Method Transformation Method for Univariate Case Transformation Method for Bivariate Case Convolution Method for Sums of Random Variables Moment Method for Sums of Random Variables Review Exercises 11. Some Special Discrete Bivariate Distributions .. 289. Bivariate Bernoulli Distribution Bivariate Binomial Distribution Bivariate Geometric Distribution Bivariate Negative Binomial Distribution Bivariate Hypergeometric Distribution Bivariate Poisson Distribution Review Exercises 12. Some Special Continuous Bivariate Distributions.

10 317. Bivariate Uniform Distribution Bivariate Cauchy Distribution Bivariate Gamma Distribution Bivariate Beta Distribution Bivariate Normal Distribution Bivariate Logistic Distribution Review Exercises xv 13. Sequences of Random Variables and Order STATISTICS .. 353. Distribution of Sample Mean and Variance Laws of Large Numbers The Central Limit Theorem Order STATISTICS Sample Percentiles Review Exercises 14. Sampling Distributions Associated with the Normal Population .. 395. Chi-square distribution Student's t-distribution Snedecor's F -distribution Review Exercises 15. Some Techniques for Finding Point Estimators of Parameters .. 413. Moment Method Maximum Likelihood Method Bayesian Method Review Exercises 16. Criteria for Evaluating the Goodness of Estimators.