Example: bankruptcy

BELIEF IN THE LAW OF SMALL NUMBERS

Psychological Bulletin 1971, Vol. 76, No. 2, 105-110. BELIEF IN THE LAW OF SMALL NUMBERS . AMOS TVERSKY AND DANIEL KAHNEMAN 1. Hebrew University of Jerusalem People have erroneous intuitions about the laws of chance. In particular, they regard a sample randomly drawn from a population as highly representative, that is, similar to the population in all essential characteristics. The prevalence of the BELIEF and its unfortunate consequences for psychological research are illustrated by the responses of professional psychologists to a questionnaire con- cerning research decisions. "Suppose you have run an experiment on 20 Apparently, most psychologists have an ex- subjects, and have obtained a significant re- aggerated BELIEF in the likelihood of success- sult which confirms your theory (z = , p fully replicating an obtained finding.)

Psychological Bulletin 1971, Vol. 76, No. 2, 105-110 BELIEF IN THE LAW OF SMALL NUMBERS AMOS TVERSKY AND DANIEL KAHNEMAN 1 Hebrew University of Jerusalem

Tags:

  Small, Number, Belief, The law of small numbers

Information

Domain:

Source:

Link to this page:

Please notify us if you found a problem with this document:

Other abuse

Transcription of BELIEF IN THE LAW OF SMALL NUMBERS

1 Psychological Bulletin 1971, Vol. 76, No. 2, 105-110. BELIEF IN THE LAW OF SMALL NUMBERS . AMOS TVERSKY AND DANIEL KAHNEMAN 1. Hebrew University of Jerusalem People have erroneous intuitions about the laws of chance. In particular, they regard a sample randomly drawn from a population as highly representative, that is, similar to the population in all essential characteristics. The prevalence of the BELIEF and its unfortunate consequences for psychological research are illustrated by the responses of professional psychologists to a questionnaire con- cerning research decisions. "Suppose you have run an experiment on 20 Apparently, most psychologists have an ex- subjects, and have obtained a significant re- aggerated BELIEF in the likelihood of success- sult which confirms your theory (z = , p fully replicating an obtained finding.)

2 The < .05, two-tailed). You now have cause to sources of such beliefs, and their consequences run an additional group of 10 subjects. What for the conduct of scientific inquiry, are what do you think the probability is that the re- this paper is about. Our thesis is that people sults will be significant, by a one-tailed test, have strong intuitions about random sam- separately for this group?" pling; that these intuitions are wrong in fun- If you feel that the probability is some- damental respects; that these intuitions are where around .35, you may be pleased to shared by naive subjects and by trained sci- know that you belong to a majority group. entists; and that they are applied with un- Indeed, that was the median answer of two fortunate consequences in the course of sci- SMALL groups who were kind enough to re- entific inquiry.

3 Spond to a questionnaire distributed at meet- We submit that people view a sample ran- ings of the Mathematical Psychology Group domly drawn from a population as highly and of the American Psychological Associa- representative, that is, similar to the popula- tion. tion in all essential characteristics. Conse- On the other hand, if you feel that the quently, they expect any two samples drawn probability is around .48, you belong to a from a particular population to be more simi- minority. Only 9 of our 84 respondents gave lar to one another and to the population than answers between .40 and .60. However, .48 sampling theory predicts, at least for SMALL happens to be a much more reasonable esti- samples. mate than . The tendency to regard a sample as a rep- 1. The ordering of authors is random. We wish to resentation is manifest in a wide variety of thank the many friends and colleagues who com- situations.

4 When subjects are instructed to mented on an earlier version, and in particular we generate a random sequence of hypothetical arc indebted to Maya Bar-Hillel, Jack Block, Jacob Cohen, Louis L. Gultman, John W. Tukey, Ester tosses of a fair coin, for example, they pro- Samuel, and Gideon Shwarz. duce sequences where the proportion of Requests for reprints should be sent to Amos Tvcrsky, Center for Advanced Study in the Behav- with known variance, one would compute the power ioral Sciences, 202 Junipero Scrra Boulevard, Stan- of the test against the hypothesis that the population ford, California 94305. mean equals the mean of the first sample. Since the 2. The required estimate can be interpreted in sev- size of the second sample is half that of the first, the eral ways. One possible approach is to follow com- computed probability of obtaining z^> is only mon research practice, where a value obtained in one.

5 473. A theoretically more justifiable approach is to study is taken to define a plausible alternative to interpret the requested probability within a Baycsian the null hypothesis. The probability requested in the framework and compute it relative to some appropri- question can then be interpreted as the power of the ately selected prior distribution. Assuming a uni- second test ( , the probability of obtaining a sig- form prior, the desired posterior probability is .478. nificant result in the second sample) against the Clearly, if the prior distribution favors the null hy- alternative hypothesis defined by the result of the pothesis, as is often the case, the posterior proba- first sample. In the special case of a test of a mean bility will be even smaller. 105. 106 AMOS TVERSKY AND DANIEL KAHNEMAN. heads in any short segment stays far closer process.

6 The two beliefs lead to the same con- to .SO than the laws of chance would predict sequences. Both generate expectations about (Tune, 1964). Thus, each segment of the re- characteristics of samples, and the variability sponse sequence is highly representative of of these expectations is less than the true- the "fairness" of the coin. Similar effects are variability, at least for SMALL samples. observed when subjects successively predict The law of large NUMBERS guarantees that events in a randomly generated series, as in very large samples will indeed be highly rep- probability learning experiments (Estes, 1964) resentative of the population from which they or in other sequential games of chance. Sub- are drawn. If, in addition, a self-corrective jects act as if every segment of the random tendency is at work, then SMALL samples should sequence must reflect the true proportion: if also be highly representative and similar to the sequence has strayed from the population one another.

7 People's intuitions about random proportion, a corrective bias in the other direc- sampling appear to satisfy the law of SMALL tion is expected. This has been called the NUMBERS , which asserts that the law of large gambler's fallacy. NUMBERS applies to SMALL NUMBERS as well. The heart of the gambler's fallacy is a mis- Consider a hypothetical scientist who lives conception of the fairness of the laws of by the law of SMALL NUMBERS . How would his chance. The gambler feels that the fairness of BELIEF affect his scientific work? Assume out- the coin entitles him to expect that any devi- scientist studies phenomena whose magnitude ation in one direction will soon be cancelled is SMALL relative to uncontrolled variability, by a corresponding deviation in the other. that is, the signal-to-noise ratio in the mes- Even the fairest of coins, however, given the sages he receives from nature is low.

8 Our sci- limitations of its memory and moral sense, entist could be a meteorologist, a pharma- cannot be as fair as the gambler expects it to cologist, or perhaps a psychologist. be. This fallacy is not unique to gamblers. If he believes in the law of SMALL NUMBERS , Consider the following example: the scientist will have exaggerated confidence in the validity of conclusions based on SMALL The moan IQ of the population of eighth graders in a cily is known to be 100. You have selected a samples. To illustrate, suppose he is engaged random sample of SO children for a study of educa- in studying which of two toys infants will tional achievements. The first child tested has an prefer to play with. Of the first five infants IQ of ISO. What do you expect the mean IQ to be studied, four have shown a preference for the for the whole sample?

9 Same toy. Many a psychologist will feel some The correct answer is 101. A surprisingly large confidence at this point, that the null hypothe- number of people believe that the expected JQ sis of no preference is false. Fortunately, such for the sample is still 100. This expectation a conviction is not a sufficient condition for can be justified only by the BELIEF that a journal publication, although it may do for a random process is self-correcting. Idioms such book. By a quick computation, our psycholo- as "errors cancel each other out" reflect the gist will discover that the probability of a image of an active self-correcting process. result as extreme as the one obtained is as Some familiar processes in nature obey such high as 3/8 under the null hypothesis. laws: a deviation from a stable equilibrium To be sure, the application of statistical produces a force that restores the equilibrium.

10 Hypothesis testing to scientific inference is The laws of chance, in contrast, do not work beset with serious difficulties. Nevertheless, the that way: deviations are not canceled as computation of significance levels (or likeli- sampling proceeds, they are merely diluted. hood ratios, as a Bayesian might prefer). Thus far, we have attempted to describe forces the scientist to evaluate the obtained two related intuitions about chance. We pro- effect in terms of a valid estimate of sampling posed a representation hypothesis according to variance rather than in terms of his subjective which people believe samples to be very simi- biased estimate. Statistical tests, therefore, lar to one another and to the population from protect the scientific community against overly which they are drawn. We also suggested that hasty rejections of the null hypothesis ( , people believe sampling to be a self-correcting Type 1 error) by policing its many members BELIEF IN SMALL NUMBERS 107.


Related search queries