2.1.5 Gaussian distribution as a limit of the Poisson ...

1 Gaussian distribution as a limit of the Poisson distributionA limiting form of the Poisson distribution (and many others seetheCentral limit Theorembelow) is theGaussian distribution . In deriving the Poisson distribution we took the limit of thetotal number of eventsN ; we now take the limit that the mean value is very large. Let swritethe Poisson distribution asPn= ne n!.(45)Now letx=n= (1 + ) where #1 and $1. Since n = , this means that we will also beconcerned with large values ofn, in which case the discretePngoes over to a continuous pdf in thevariablex. Using Stirling s formula forn!:x! 2 x e xxxasx (46)we find7p(x)= (1+ )e 2 e (1+ )[ (1 + )] (1+ )+1/2=e (1 + ) (1+ ) 1/2 2 (see footnote) =e 2/2 2 (48)Substituting back forx, with =(x )/ , yieldsp(x)=e (x )2/(2 ) 2 (49)This is aGaussian, or Normal8, distribution with mean and variance of.

2 The Gaussian distributionis the most important distribution in probability, due to its role in the central limit theorem , whichloosely says that the sum of a large number of independent quantities tends to have a Gaussian form,independent of the pdf of the individual measurements. The above specific derivation is somewhatcumbersome, and it will actually be more elegant to use the central limit theorem to derive theGaussian approximation to the Poisson More on the GaussianThe Gaussian distribution is so important that we collect some properties here. It is normally writtenasp(x)=1(2 )1/2 e (x )2/2 2,(50)7 Maths Notes: The limit of a function like (1 + ) (1+ )+1/2with #1 and $1 can be found by taking thenatural log, then expanding in tosecondorder and using #1:ln[(1 + ) (1+ )+1/2] = [ (1 + )+1/2] ln(1 + ) = ( +1/2+ )( 2/2+O( 3))) + 2/2+O( 3) (47)8 The name Normal was given to this distribution by the statistician K.

3 Pearson, who almost immediately regrettedintroducing the name. It is also sometimes called the that is the mean and the standard deviation. The first statement is obvious: consider x ,which must vanish by symmetry since it involves integration of anodd function. To prove the secondstatement, write (x )2 =1(2 )1/2 3 y2e y2/2dy(51)and do the integral by that the distribution is correctly normalized is harder, but there is a clever trick, which isto extend to a two-dimensional Gaussian for two independent (zero-mean) variablesxandy:p(x, y)=12 2e (x2+y2)/2 2.(52)The integral over both variables can now be rewritten using polar coordinates: p(x, y)dx dy= p(x, y)2 r dr=12 2 2 r e r2/2 2dr(53)and the final expression clearly integrates toP(r > R) = exp( R2/2 2),(54)so the distribution is indeed correctly , the single Gaussian distribution has no analyticexpression for its integral, even thoughthis is often of great interest.

4 As seen in the next section, we often want to know the probability ofa Gaussian variable lying above some value, so we need to know the integral of the Gaussian ; thereare two common notations for this (assuming zero mean):P(x < X ) = (X);(55)P(x < X )=12[1 + erf(X/ 2)],(56)where theerror functionis defined aserf(y) 2 y0e t2dt.(57)A useful approximation for the integral in the limit of highxisP(x > X ))e X2/2(2 )1/2X(58)(which is derived by a Taylor series:e (x+$)2/2)e x2/2e x$and the linear exponential in%can beintegrated). Tails and measures of rarenessSo far, we have asked questions about the probability of obtaining a particular experimental outcome,but this is not always a sensible question. Where there are many possible outcomes, the chance of20 Figure 3:The Gaussian distribution , illustrating the area under various parts of the curve, dividedin units of.

5 Thus the chance of being within1 of the mean is 68%; 95% of results are within2 of the mean; results are within3 of the given one happening will be very small. This is most obviouswith a continuous pdf,p(x): theprobability of a result in the rangex=atox=a+ xisp(a) x, so the probability of gettingx=aexactly is precisely zero. Thus it only makes sense to ask about the probability ofxlying in a most common reason for calculating the probability of a given outcome (apart from betting), isso we can test hypotheses. The lectures will give an in-depth discussion of this issue later, as it canbe quite subtle. Nevertheless, it is immediately clear that we want to set possible outcomes to anexperiment in some order of rareness, and we will be suspicious of ahypothesis when an experimentgenerates an outcome that ought to be very rare if the hypothesis were , we need to define a typical value forxand some means of deciding ifxis far from thetypical value.

6 For a Gaussian , we would naturally the mean, as the typical value and (x )/ asthe distance. But how do we make this general? There are two other common measures of location:The ModeThe value ofxwherep(x) has a MedianThe value ofxsuch thatP(>x) = the Gaussian , both these measures are equal to the mean. In general, the median is the safestchoice, since it is easy to devise pathological examples where the mode or even the mean is not welldefined. Following on from the median, we can defineupper and lower quartiles, which togetherenclose half the probability the valuesx1andx2, whereP(<x1)= andP(>x2)= suggests a measure of rareness of events, which is to single outevents that lie in the tails ofthe distribution , where eitherP(<x)$1 orP(>x)$1.

7 This can be done in a 1-tailed or a 2-tailed manner, depending on whether we choose to count only high excursions, or to be impressedby deviations in either area under the tail of a pdf is called ap value, to emphasise that we have to be careful withmeaning. If we get, say,p= this means that there is a probability of to get a valueasextreme as this one, or worse, on a given hypothesis. So we need tohave a hypothesis in mind21to start with; this is called thenull hypothesis it would typically be something non-committal,such as there is no signal . If we get apvalue that is small, this is some evidence against the nullhypothesis, in which case we can claimed to have detected a signal. Small values ofpare describedas giving significant evidence against the null hypothesis: forp= we would say this result issignificant at the 5% level.

8 For completeness, we should mention the termconfidence level, which is complementary to thesignificance level. IfP(<x1)= andP(>x2)= , then we would say that, on the nullhypothesis,x1< x < x2at 90% confidence, orx<x2at 95% shown later in the course, the p value is not the same as the probability that thenull hypothesis is correct, although many people think thisis the case. Nevertheless,whenpis small, you are on good grounds in disbelieving the null of thepvalues corresponding to particular places in the Gaussian are listed in table 1. Theweakest evidence would be a 2 result, which happens by chance about 5% of the time. This ishardly definitive, although it may provoke further work. Buta3 result is much less probable.

9 Ifyou decide to reject the null hypothesis wheneverx >3 , you will be wrong only one time in3000. Nevertheless, discovery in some areas of science can insist on stronger evidence, perhaps atthe 5 level (1-sidedp= 10 7). This is partly because may not itself be known precisely, butalso because many different experiments can be performed: if wesearch for the Higgs Boson in 100different independent mass ranges, we are bound to find a result that is significant at about the 1% 1: Tails of the Gaussianx/ , this process is not perfect. If we make an observation and getx $ , this actuallyfavours a narrower distribution than the standard one, but broader distributions are easier to detect,because the probability of an extreme deviation falls The likelihoodAlthough we have argued that the probability of a continuous variable having exactly some givenvalue is zero, the relative probability of having any two values,p(x=a)/p(x=b) is well can be extended to the idea of relative probabilities for larger datasets, wherendrawings aremade from the pdf.

10 We can approach this using themultinomial distribution , where we extendthe binomial to something with more than two possible outcomes: we toss a six-sided dice seventimes, and ask what is the probability of getting three ones, twotwoes, a five and a six? Numberthe possible results of each trial, and say each occursn1times,n2times etc., with probabilitiesp1,p2etc., out ofNtrials. Imagine first the case where the trials give a string of 1 s,followed by 2 s probability of this happening ispn11pn22pn33 . If we don t care which trials give these particularnumbers, then we need to multiply by the number of ways in whichsuch an outcome could choosing then1first, which can happenCNn1ways; thenn2can be chosenCN all these factors, we get the simple resultp=N!