Significane Testing and the Correlation - …

1 Statistical Significance of a Correlation 1 Running Head: SIGNIFICANCE Testing AND correlations Testing the Statistical Significance of a Correlation R. Michael Furr Wake Forest University Address correspondence to: Mike Furr Department of Psychology Wake Forest University Winston-Salem, NC 2706 336-758-5024 Statistical Significance of a Correlation 2 Testing the Statistical Significance of a Correlation Researchers from Psychology, Education, and other social and behavioral sciences are very concerned with statistical significance. If a researcher conducts a study and finds that the results are statistically significant, then the he or she has greater confidence in the effects revealed by the study. When results are statistically significant, researchers are more likely to believe that the effects are real and not likely to have occurred by chance.

2 The goal of science is to understand our physical or social world, and this occurs in part by being able to judge which research findings are real and which are flukes and red herrings. This paper describes the procedures through which researchers determine whether the results of a study are statistically significant. It presents the logic, technical steps, and interpretation of a test of statistical significance, specifically for researchers examining a Correlation between two variables. Many textbooks provide in-depth introductions to statistical significance, but there appear to be no sources that provide such an introduction in the context of correlations . Most introductory statistics textbooks in Psychology provide concepts and procedures for significance Testing in the context of means, but the extension of significance Testing to correlations is usually very slight.

3 Typically, the coverage of significance Testing for correlations , if it is discussed at all, focuses on computational procedures, bypassing the conceptual foundations and interpretations. In fact, the organization of some introductory statistics textbooks implies that correlations and significance Testing are completely separate issues. For example, a chapter on Correlation might be in a section labeled Descriptive Statistics and the chapters related to significance Testing might be included in a section labeled Inferential Statistics. Although general statistics textbooks omit deep coverage of the conceptual and practical foundations of significance Testing for correlations , one might suspect that such coverage could be found in sources that focus on correlational procedures specifically ( , Archdeacon, 1994; Bobko, 2001; Chen & Popovich, 2002; Cohen & Cohen, 1983; Edwards; 1984; Ezekiel, 1941; Miles & Shevlin, 2001; Pedhazur, 1997).

4 Unfortunately, these sources also omit in-depth discussions of basic concepts in significance Testing . The more advanced sources naturally assume that readers already have a solid grasp Statistical Significance of a Correlation 3 of basic concepts in significance Testing . Unfortunately, even the more introductory sources provide little background in basic concepts in statistically significance as related to correlations . A number of potential problems arise from the fact that no sources provide in-depth discussions of significance Testing as related to correlations . First, some budding researchers might be left with the mistaken and potentially confusing belief that correlations and significance tests are unrelated issues. Although the computation and interpretation of a Correlation can proceed without reference to a significance test, correlations are rarely reported without an accompanying significance test.

5 Second, even if researchers are aware that correlations can be tested for statistical significance, they might have difficulty connecting fundamental concepts in significance Testing ( , parameters, confidence intervals, distributions of inferential statistics) to correlations . The existing sources make little effort t to generalize concepts articulated in the context of means or frequencies to correlational analyses. Third, the existing sources create difficulty for course instructors who cover correlational analyses before other kinds of analyses. For example, some Psychology Departments divide their Research Methods and Statistics courses into a correlational semester and an experimental semester. If the correlational course is taken before the experimental course, then instructors who teach the correlational course face a dilemma.

6 They can ignore significance Testing of correlations , they can provide a cursory coverage of significance Testing of correlations , or they can assign readings that present significance Testing in the context of means or frequencies. A solid understanding of significance Testing as related to correlations may be particularly important as the field evolves in two ways. First, researchers are increasingly aware of the importance of effect sizes, such as correlations (American Psychological Association, 2001; Capraro & Capraro, 2003; Furr, 2004; Heldref Foundation, 1997; Kendall, 1997; Murphy, 1997; Rosenthal, Rosnow, & Rubin, 2000; Thompson, 1994, 1999; Wilkinson & APA Task Force on Statistical Inference, 1999). Second, many in the field recognize that regression, based on a correlational foundation, is a general approach to data analysis that can incorporate much that is typically conceptualized as Analysis of Variance.

7 As the awareness and use of effect sizes and correlational analytic procedures continue to grow, and as advanced Statistical Significance of a Correlation 4 correlational procedures continue to emerge, researchers should have a solid understanding of the connections between correlations and significance Testing . The current paper is intended to partially fill this hole in the basic statistical literature. It describes what statistical significance is about, presents fundamental concepts in evaluating statistical significance, and details the procedures for Testing the statistical significance of a Correlation . Samples and Populations: Inferential Statistics Imagine that Dr. Cartman wants to know whether the Scholastic Aptitude Test (SAT) is a valid predictor of college freshman performance at the local university.

8 To address this issue, he recruits a sample of 200 freshmen from the university. The students give their consent for Dr. Cartman to have access to their academic records, from which he records their SAT scores and their first-year college Grade Point Average. Based on these data, Dr. Cartman finds that the Correlation between SAT scores and GPA is .40, which is a positive Correlation of moderate size. This Correlation tells him that, within the sample, the students with relatively high SAT scores tend to have relatively high GPAs (and that students with relatively low SAT scores tend to have relatively low GPAs). Based on this finding in his sample, Dr. Cartman is tempted to conclude that the SAT is indeed a useful predictor of freshman GPA at the University.

9 But how much confidence should Dr. Cartman have in this conclusion? He might be hesitant to use the results found in a sample of 200 students to make an inference about whether SAT scores are correlated with GPAs in the entire freshman student body. The question of statistical significance arises from the fact that scientists would like to make conclusions about psychological phenomena, effects, differences, or relationships between variables as they exist in a large population (or populations) of people (or rats, monkeys, etc, depending on the scientist s area of expertise). For example, Dr. Cartman would like to make conclusions about whether SAT scores are correlated with GPAs in the entire freshman student body. Similarly, a clinical psychologist might be interested in whether a new drug generally helps to alleviate depression, within the population of all people who might take the drug.

10 Or a social psychologist hypothesizes that romantic Statistical Significance of a Correlation 5 couples in which the partners have similar profiles of personality traits tend to be happier than couples in which the partners have dissimilar personalities. This researcher would be interested in concluding whether similarity and romantic happiness are generally correlated with each other within the population of all couples in romantic relationships. Despite their desire to make conclusions about large populations, researchers generally study only samples of people recruited from the larger population of interest. In our example, Dr. Cartman would like to make conclusions about the entire freshman class at the university, but he is able to recruit a sample of only 200 students from the student body.