Testing Group Difierences using T-tests, ANOVA, …

1 Testing Group Differences using T-tests, ANOVA, and NonparametricMeasuresJamie DeCosterDepartment of PsychologyUniversity of Alabama348 Gordon Palmer HallBox 870348 Tuscaloosa, AL 35487-0348 Phone: (205) 348-4431 Fax: (205) 348-8648 January 11, 2006I would like to thank Anne-Marie Leistico and Angie Maitner for comments made on an earlier version ofthese notes. If you wish to cite the contents of this document, the APA reference for them would beDeCoster, J. (2006). Testing Group Differences using T-tests, ANOVA, and Nonparametric (month, day, and year you downloaded the notes, without the parentheses) from future versions of these notes or help with data analysis RIGHTS TO THIS DOCUMENT ARE RESERVEDC ontents1 Introduction12 Testing One or two Means53 Testing a Single Between-Subjects Factor154 Testing Multiple Between-Subjects Factors305 Testing a Single Within-Subjects Factor476 Testing Multiple Within-Subjects Factors54iChapter Data and Data Sets The information that you collect from an experiment, survey, or archival source is referred to as yourdata.

2 Most generally, data can be defined as a list of numbers possessing meaningful relations. For analysts to do anything with a Group of data they must first translate it into adata set. A dataset is a representation of data, defining a set of variables that are measured on a set of cases. Avariableis simply a feature of an object that can be categorized or measured by a variable takes on differentvaluesto reflect the particular nature of the object being values that a variable takes will vary when measurements are made on different objects atdifferent times. A data set will typically contain measurements on several different variables.

3 Each time that we record information about an object we create acasein the data set. Likevariables, a data set will typically contain multiple cases. The cases should all be derived fromobservations of the same type of object, with each case representing a different example of thattype. Cases are also sometimes referred to asobservations. The object type that defines yourcases is called yourunit of analysis. Sometimes the unit of analysis in a data set will be verysmall and specific, such as the individual responses on a questionnaire. Sometimes it will be verylarge, such as companies or nations. The most common unit of analysis in social science researchis the participant or subject.

4 If the value of a variable has actual numerical meaning (so that it measures the amount of somethingthat a case has), it is called acontinuousvariable. An example of a continuous variable measured ona planet would be the amount of time it took to orbit the sun. If the value of a variable is used toindicate a Group the case is in, it is called acategoricalvariable. An example of a categorical variablemeasured on a country would be the continent in which it is terms of the traditional categorizations given to scales, a continuous variable would have either aninterval, or ratio scale, while a categorical variable would have a nominal scale.

5 Ordinal scales sortof fall in between. Bentler and Chou (1987) argue that ordinal scales can be reasonably treated ascontinuous as long as they have four or more categories. However, if an ordinal variable only has oneor two categories you are probably better off either treating it as categorical, or else using proceduresspecifically designed to handle ordinal data. When describing a data set, you should name your variables and state whether they are continuousor categorical. For each continuous variable you should state its units of measurement, while for eachcategorical variable you should state how the different values correspond to different groups.

6 Youshould also report the unit of analysis of the data set. You typically would not list the specific cases,although you might describe where they came from. When conducting a statistical analysis, you often want to use values of one variable to predict orexplain the values of another variable. In this case the variable you look at to make your prediction is1called theindependent variable(IV) while the variable you are trying to predict is called thedependentvariable(DV). General information about statistical inference Most of the time that we collect data from a Group of subjects, we are interested in making inferencesabout some larger Group of which our subjects were a part.

7 We typically refer to the Group we want togeneralize our results to as thetheoretical population. The Group of all the people that could potentiallybe recruited to be in the study is theaccessible population, which ideally is a representative subset ofthe theoretical population. The Group of people that actually participate in our study is oursample,which ideally is a representative subset of the accessible population. Very often we will calculate the value of anestimatein our sample and use that to draw conclusionsabout the value of a correspondingparameterin the underlying population. Populations are usually toolarge for us to measure their parameters directly, so we often calculate an estimate from a sample drawnfrom the population to give us information about the likely value of the parameter.

8 The procedure ofgeneralizing from data collected in a sample to the characteristics of a population is example, let us say that you wanted to know the average height of 5th grade boys. What youmight do is take a sample of 30 5th grade boys from a local school and measure their heights. Youcould use the average height of those 30 boys (a sample estimate) to make a guess about the averageheight of all 5th grade boys (a population parameter). We typically use Greek letters when referring to population parameters and normal letters when refer-ring to sample statistics. For example, the symbol is commonly used to represent mean value of apopulation, while the symbolXis commonly used to represent the mean value of a sample.

9 One type of statistical inference you can make is called ahypothesis test. A hypothesis test uses thedata from a sample to decide between anull hypothesisand analternative hypothesisconcerning thevalue of a parameter in the population. The null hypothesis usually makes a specific claim about theparameters (like saying that the average height of 5th grade boys is 60 inches), while the alternativehypothesis says that the null hypothesis is false. Sometimes the alternative hypothesis simply says thatthe null hypothesis is wrong (like saying that the average height of 5th grade boys isnot60 inches).Other times the alternative hypothesis says the null hypothesis is wrong in a particular way (like sayingthat the average height of 5th grade boys isless than60 inches).

10 For example, you might have a null hypothesis stating that the average height of 5th grade boys isequal to the average height of 5th grade girls, and an alternative hypothesis stating that the two heightsare not equal. You could represent these hypotheses using the following : {boys}= {girls}Ha: {boys}6= {girls}where H0refers to the null hypothesis and Harefers to the alternative hypothesis. To actually perform a hypothesis test, you must collect data from a sample drawn from the populationof interest that will allow you to discriminate your hypotheses. For example, if your hypotheses involvethe value of a population parameter, then your data should provide a sample estimate corresponding tothat parameter.