A Short Introduction to Probability - University of Queensland

1 A Short Introduction toProbabilityProf. Dirk P. KroeseSchool of Mathematics and PhysicsThe University of Queenslandc 2018 Kroese. These notes can be used for educational purposes, pro-vided they are kept in their original form, including this title 2018 KroeseContents1 Random Experiments and Probability Random Experiments .. Sample Space .. Events .. Probability .. Counting .. Conditional Probability and independence .. Rule .. of Total Probability and Bayes Rule .. 272 Random Variables and Probability Random Variables .. Probability Distribution .. Distributions .. Distributions .. Expectation .. Transforms .. Some Important Discrete Distributions .. Distribution .. Distribution .. distribution .. Distribution .. Distribution .. Some Important Continuous Distributions .. Distribution .. Distribution.

2 , or Gaussian, Distribution .. and 2-distribution .. 523 Generating Random Variables on a Introduction .. Random Number Generation .. The Inverse-Transform Method .. Generating From Commonly Used Distributions .. 59 Copyrightc 2018 Kroese2 Contents4 Joint Joint Distribution and Independence .. Joint Distributions .. Joint Distributions .. Expectation .. Conditional Distribution .. 795 Functions of Random Variables and Limit Jointly Normal Random Variables .. Limit Theorems .. 89A Exercises and Problem Set 1 .. Answer Set 1 .. Problem Set 2 .. Answer Set 2 .. Problem Set 3 .. Answer Set 3 .. 105B Sample Exam 1 .. Exam 2 .. 110C Summary of Formulas113D Statistical Tables119 Copyrightc 2018 KroesePrefaceThese notes form a Short , but comprehensive, second-year Introduction to prob-ability. You might find these notes useful as a source of reference and an aid toself-learning for courses such as STAT2202, STAT2003, and STAT2004.

3 A morepolished version, with additional material, but without appendix A (Exercisesand Solutions) and appendix B (Sample Exams), forms Part I of the Kroese and Chan (2014).Statistical Modeling and Computation,Springer, New can be downloaded (free) completely master the material, it is important that you1. visit the lectures, where many extra examples will be provided;2. do the tutorial exercises and the exercises in the appendix, which arethere to help you with the technical side of things; you will learn hereto apply the concepts learned at the lectures,3. carry out random experiments on the computer. This will give you abetter intuition about how randomness of these will be essential if you wish to understand Probability beyond fillingin the formulas .Notation and ConventionsThroughout these notes I try to use a uniform notation in which, as a rule, thenumber of symbols is kept to a minimum. For example, I preferqijtoq(i,j),XttoX(t), andEXtoE[X].

4 The symbol := denotes is defined as . We will also use the forrandom variableand (or iid) forindependent and identically 2018 Kroese4 PrefaceI will use thesans seriffont to denote Probability distributions. For exampleBindenotes the binomial distribution, andExpthe exponential references to Examples, Theorems, etc. are of the same form. For example,Theorem refers to the second theorem of Chapter 1. References to formula sappear between brackets. For example, ( ) refers to formula 4 of Chapter Hsu, H. (1997). Probability , Random Variables & Random s Outline Series, McGraw-Hill, New York. Ross, S. M. (2005).A First Course in Probability , 7th ed., Prentice-Hall,Englewood Cliffs, NJ. Kroese, P, Chan, (2014).Statistical Modeling and Computa-tion, Springer, New York. Feller, W. (1970).An Introduction to Probability Theory and Its Applica-tions, Volume I., 2nd ed., Wiley & Sons, New 2018 KroeseChapter 1 Random Experiments andProbability Random ExperimentsThe basic notion in Probability is that of arandom experiment: an experi-ment whose outcome cannot be determined in advance, but is nevertheless stillsubject to of random experiments are:1.

5 Tossing a die,2. measuring the amount of rainfall in Brisbane in January,3. counting the number of calls arriving at a telephone exchange during afixed time period,4. selecting a random sample of fifty people and observing the number ofleft-handers,5. choosing at random ten people and measuring their (Coin Tossing)The mostfundamentalstochastic experimentis the experiment where a coin is tossed a number of times, sayntimes. Indeed,much of Probability theory can be based on this simple experiment, as we shallsee in subsequent chapters. To better understand how this experiment behaves,we can carry it out on a digital computer, for example in Matlab. The followingsimple Matlab program, simulates a sequence of 100 tosses with a fair coin(thatis, heads and tails are equally likely), and plots the results in a bar = (rand(1,100) < 1/2)bar(x)Copyrightc 2018 Kroese6 Random Experiments and Probability ModelsHerexis a vector with 1s and 0s, indicating Heads and Tails, say.

6 Typicaloutcomes for three such experiments are given in Figure : Three experiments where a fair coin is tossed 100 times. The darkbars indicate when Heads (=1) can also plot the average number of Heads against the number of the same Matlab program, this is done in two extra lines of code:y = cumsum(x)./[1:100]plot(y)The result of three such experiments is depicted in Figure Notice that theaverage number of Heads seems to converge to 1/2, but there is a lot of 2018 Random : The average number of heads inntosses, wheren= 1,.., (Control Chart)Control charts, see Figure , are frequentlyused in manufacturing as a method forquality control. Each hour the averageoutput of the process is measured for example, the average weight of 10bags of sugar to assess if the process is still in control , for example, if themachine still puts on average the correct amount of sugar in the bags. Whenthe process>Upper Control Limitor<Lower Control Limitand an alarm israised that the process is out of control, , the machine needs to be adjusted,because it either puts too much or not enough sugar in the bags.

7 The questionis how to set the control limits, since the random process naturally fluctuatesaround its centre or target c23569101112487 Center lineUCLLCL + c1 Figure : Control ChartCopyrightc 2018 Kroese8 Random Experiments and Probability ModelsExample (Machine Lifetime)Suppose 1000 identical components aremonitored for failure, up to 50,000 hours. The outcome of such a randomexperiment is typically summarised via the cumulative lifetime table and plot, asgiven in Table and Figure , respectively. Here F(t) denotes the proportionof components that have failed at timet. One question is how F(t) can bemodelled via a continuous functionF, representing the lifetime distribution ofa typical (h)failed F(t) (h)failed F(t) : The cumulative lifetime tabletF(t) (est.) : The cumulative lifetime tableCopyrightc 2018 Random Experiments9 Example 4-engine aeroplane is able to fly on just one engine on eachwing.

8 All engines are : A aeroplane with 4 unreliable enginesNumber the engines: 1,2 (left wing) and 3,4 (right wing). Observe which engineworks properly during a specified period of time. There are 24= 16 possibleoutcomes of the experiment. Which outcomes lead to system failure ? More-over, if the Probability of failure within some time period is known for each ofthe engines, what is the Probability of failure for the entire system? Again thiscan be viewed as a random are two more pictures of randomness. The first is a computer-generated plant , which looks remarkably like a real plant. The second is real datadepicting the number of bytes that are transmitted over some communicationslink. An interesting feature is that the data can be shown to exhibit fractal behaviour, that is, if the data is aggregated into smaller or larger time intervals,a similar picture will : Plant growth1251301351401451501550200040006000 8000100001200014000 Intervalnumber of bytesFigure : Telecommunications dataWe wish to describe these experiments via appropriate mathematical models consist of three building blocks: asample space, a set ofeventsand aprobability.

9 We will now describe each of these 2018 Kroese10 Random Experiments and Probability Sample SpaceAlthough we cannot predict the outcome of a random experiment with certaintywe usually can specify a set of possible outcomes. This gives the first ingredientin our model for a random space of a random experiment is the set of allpossible outcomes of the of random experiments with their sample spaces are:1. Cast two dice consecutively, ={(1,1),(1,2),..,(1,6),(2,1),..,(6,6)}.2 . The lifetime of a machine (in days), =R+={positive real numbers}.3. The number of arriving calls at an exchange during a specified time in-terval, ={0,1, }=Z+.4. The heights of 10 selected people. ={(x1,..,x10),xi 0,i= 1,..,10}=R10+.Here (x1,..,x10) represents the outcome that the length of the first se-lected person isx1, the length of the second person isx2, et that for modelling purposes it is often easier to take the sample spacelarger than necessary. For example the actual lifetime of a machine wouldcertainly not span the entire positive real axis.

10 And the heights of the 10selected people would not exceed 3 EventsOften we are not interested in a single outcome but in whether or not one of agroupof outcomes occurs. Such subsets of the sample space are will be denoted by capital lettersA,B,C,.. We say that eventAoccursif the outcome of the experiment is one of the elements 2018 Events11 Examples of events are:1. The event that the sum of two dice is 10 or more,A={(4,6),(5,5),(5,6),(6,4),(6,5),(6 ,6)}.2. The event that a machine lives less than 1000 days,A= [0,1000).3. The event that out of fifty selected people, five are left-handed,A={5}.Example (Coin Tossing)Suppose that a coin is tossed 3 times, and thatwe record every head and tail (not only the number of heads or tails). Thesample space can then be written as ={HHH,HHT,HTH,HTT,THH,THT,TTH,TTT},where , for example, HTH means that the first toss is heads, the second tails,and the third heads. An alternative sample space is the set{0,1}3of binaryvectors of length 3, , HTH corresponds to (1,0,1), and THH to (0,1,1).]