PROBABILITY AND STATISTICS - ERNET

1 PROBABILITY AND STATISTICSMANJUNATH KRISHNAPURCONTENTS1. What is STATISTICS and what is PROBABILITY ?52. Discrete PROBABILITY spaces73. Examples of discrete PROBABILITY spaces124. Countable and uncountable175. On infinite sums196. Basic rules of probability237. Inclusion-exclusion formula258. Bonferroni s inequalities289. Independence - a first look3010. Conditional PROBABILITY and independence3111. Independence of three or more events3412. Discrete PROBABILITY distributions3513. General PROBABILITY distributions3814. Uncountable PROBABILITY spaces - conceptual difficulties3915. Examples of continuous distributions4216. Simulation4717.

2 Joint distributions5118. Change of variable formula5419. Independence and conditioning of random variables5820. Mean and Variance6221. Makov s and Chebyshev s inequalities6722. Weak law of large numbers6823. Monte-Carlo integration6924. Central limit theorem7025. Poisson limit for rare events7326. Entropy, Gibbs distribution741. Introduction772. Estimation problems783. Properties of estimates824. Confidence intervals8515. Confidence interval for the mean896. Actual confidence by simulation907. Testing problems - first example928. Testing for the mean of a normal population949. Testing for the difference between means of two normal populations9510.

3 Testing for the mean in absence of normality9711. Chi-squared test for goodness of fit9812. Tests for independence10013. Regression and Linear regression102 Appendix A. Lecture by lecture plan110 Appendix B. Various pieces1112 Probability41. WHAT IS STATISTICS AND WHAT IS PROBABILITY ?Sometimes STATISTICS is described asthe art or science of decision making in the face of are some examples to illustrate what it the apocryphal story of two women who go to King Solomon with a child, eachclaiming that it is her own daughter. The solution according to the story uses human psychologyand is not relevant to recall here. But is this a reasonable question that the king can decide?

4 Daughters resemble mothers to varying degrees, and one cannot be absolutely sure of guessingcorrectly. On the other hand, by comparing various features of the child with those of the twowomen, there is certainly a decent chance to guess we could always get the right answer, or if we could never get it right, the question would nothave been interesting. However, here we have uncertainty, but there is a decent chance of gettingthe right answer. That makes it interesting - for example, we can have a debate betweeneyeistsandnosistsas to whether it is better to compare the eyes or the noses in arriving at a IISc cricket team meets the Basavanagudi cricket club for a match.

5 Unfortunately,the Basavanagudi team forgot to bring a coin to toss. The IISc captain helpfully offers his coin, butcan he be trusted? What if he spent the previous night doctoring the coin so that it falls on oneside with probability3/4(or some other number)?Instead of cricket, they could spend their time on the more interesting question of checking ifthe coin isfairorbiased. Here is one way. If the coin is fair, in a large number of tosses, commonsense suggests that we should get about equal number of heads and tails. So they toss the coin100 times. If the number of heads is exactly 50, perhaps they will agree that it is fair. If the numberof heads is 90, perhaps they will agree that it is biased.

6 What if the number of heads is 60? Or 35?Where and on what basis to draw the line between fair and biased? Again we are faced with thequestion of making decision in the face of psychic claims to have divine visions unavailable to most of us. You are assignedthe task of testing her claims. You take a standard deck of cards, shuffle it well and keep it facedown on the table. The psychic writes down the list of cards in some order - whatever her visiontells her about how the deck is ordered. Then you count the number of correct guesses. If thenumber is 1 or 2, perhaps you can dismiss her claims. If it is 45, perhaps you ought to be take herseriously. Again, where to draw the line?

7 The logic is this. Roughly one may say thatsurpriseis just the name for our reaction to an eventthat we a priorithought had low PROBABILITY . Thus, we approach the experiment with the belief thatthe psychic is just guessing at random, and if the results are such that under that random-guess-hypothesis they have very small PROBABILITY , then we are willing to discard our preconception andaccept that she is a low a PROBABILITY is surprising? In the context of psychics, let us say,1/10000. Once we fixthat, we must find a numberm 52such that by pure guessing, the PROBABILITY to get more than5mcorrect guesses is less that1/10000. Then we tell the psychic that if she gets more thanmcorrectguesses, we accept her claim, and otherwise, reject her claim.

8 This raises the simple (and you cando it yourself)Question a deck of52cards, find the numbermsuch thatP(by random guessing we get more thanmcorrect guesses)< :There are many situations in real life where one is required to make decisions underuncertainty. A general template for the answer could be to fix a small number that we allowas the PROBABILITY of error, and deduce thresholds based on it. This brings us to the question ofcomputing probabilities in various : PROBABILITY theory is a branch of pure mathematics, and forms the theoretical basisof STATISTICS . In itself, PROBABILITY theory has some basic objects and their relations (like real num-bers, addition etc for analysis) and it makes no pretense of saying anything about the real are given and theorems are then deduced about these objects, just as in any other part a very important aspect of PROBABILITY is that it isapplicable.

9 In other words, there are manysituations in which it is reasonable to take a model in probabilityIn the example above, to compute the PROBABILITY one must make the assumption that the deckof cards was completely shuffled. In other words, all possible 52! orders of the 52 cards areassumed to be equally likely. Whether this assumption is reasonable or not depends on how wellthe card was shuffled, whether the psychic was able to get a peek at the cards, whether someinsider is informing the psychic of the cards etc. All these are non-mathematical questions, andmust be decided on other : PROBABILITY and STATISTICS are very relevant in many situations that do not involve anyuncertainty on the face of it.

10 Here are some of data. Large files in a computer can be compressed to a .zip formatand uncompressed when necessary. How is it possible to compress data like this? To give a verysimple analogy, consider a long English word likeinvertebrate. If we take a novel and replace everyoccurrence of this word with zqz , then it is certainly possible to recover the original novel (since zqz does not occur anywhere else). But the reduction in size by replacing the 12-letter wordby the 3-letter word is not much, since the wordinvertebratedoes not occur often. Instead, if wereplace the 4-letter word then by zqz , then the total reduction obtained may be much higher,as the word then occurs quite suggests the following optimal way to represent words in English.