VALIDITY Page 1 I. Internal and External Validity

1 VALIDITY Page 1 March 14, and External VALIDITY goal of research is to establish clear relationships among the variables of interest. Thisis true whether you choose an experimental or a correlational design. Regardless of yourresearch design, the conclusions you draw from your results depend on how well youconstruct your study within the bounds of that design. I would like to discuss twoimportant properties of good research design: Internal VALIDITY and External VALIDITY designs are invented to enable the researcher to answer research questions asvalidly, objectively, accurately, and economically as possible. research plan is deliberately and specifically conceived and executed to bringempirical evidence to bear on the research problems.

2 Is a wide range of possible designs; theoretically, at least, as many designs ofresearch exist as there are problems. are carefully worked out to yield dependable and valid answers to the researchquestions epitomized by the hypotheses. , as researchers, should recognize that almost no research design can satisfy all of thesefactors, yet we should strive to satisfy as many as Page 2 March 14, 2006 are two general factors that must be borne in mine when designing a study: Internal VALIDITY and External VALIDITY . VALIDITY asks the question: Did X, the experimental manipulation really make asignificant difference? , anything affecting the control of a design becomes a problem of Internal VALIDITY . are more or less internally valid depending on how well extraneousvariables have been controlled by the extraneous variables are not controlled in the experiment, we cannot know whetherobserved changes in the experimental group are due to the experimental treatment or toan extraneous and Stanley have identified eight classes of extraneous -- experimental treatments extend over a period of time, providingopportunity for other events to occur beside the experimental treatment.

3 Forexample, suicide rates for the year of 1963 an increase maybe found because of theevent of November -- while the experimental treatment is in progress, biological orpsychological processes within the subject are likely to Page 3 March 14, -- In many psychological experiments a pretest is administered, followed bythe experimental treatment and then a posttest. If the two tests are similar, subjectsmay show an improvement simply as an effect of their experience with the pretest,that is, they have become -- a change in behavior may be observed from pretest to posttestbecause the nature of the measuring instrument has Regression. Whenever a test-retest procedure is used to assess change asan effect of the experimental treatment, the possibility exist that statistical regressioncan account for observed change (regression towards the mean).

4 Selection. In experimental designs in which a control group is used, theeffects of the treatment can sometime be confounded because of differential selectionof subjects for the experimental and control Mortality -- selective attrition of - maturation interaction. This extraneous variable is similar to differentialselection except that maturation is the specific confounding have seen now how eight different kinds of extraneous variables can threaten theinternal VALIDITY of an experiment. It is necessary for the researcher to select anappropriate experimental design that controls such Page 4 March 14, key word here is control and I would like to talk about the notion of the notion of the Control Group review of the research literature indicate that control group was not used beforethis century in psychological research.

5 Furthermore, the importance of control seems notto have been clearly and generally understood or applied in psychological, sociological,and educational research much before 1920. , to judge studies published recently, the importance of experimental control doesnot seem to be completely understood yet in psychology. control group is generally believed to be the group which does not receive anytreatment. this is not necessarily so. The control group may be another treatment this is another experimental group and in the traditional sense, there wouldbe no control group (for example anxiety and performance). there is more than one experimental group and any two groups are givendifferent treatments, control is present in the sense of comparison.

6 Long as there is an attempt to make two groups systematically different on a dependentvariable, a comparison is Page 5 March 14, 2006 , the traditional notion that an experimental group should receive the treatment notgiven to a control group is a special case of the more general rule that comparison groupsare necessary for the Internal VALIDITY of any scientific VALIDITY second factor to keep in mind when designing a study is External VALIDITY . Externalvalidity means representativeness or generalizability. an experiment has been completed and a relation found, to what population canit be generalized? are two general factors that affect the generalizability or External VALIDITY offindings from VALIDITY concerns the extent to which the results of an experiment can begeneralized from the specific sample that was studied to a larger group of )experimentally accessible population (college students)2)target population (all of the USA) VALIDITY -- concerns the extent to which the results of an experiment canbe generalized from the set of environmental conditions created by the researcher toVALIDITY Page 6 March 14, 2006other environmental conditions.

7 There are ten factors that affect the ecologicalvalidity of an )Explicit description of the experimental treatment (replication)2)Multiple-treatment interference each subject receive more than one treatmentA B C sequential effect. If the treatment is always presented in the same orderchanges could be due to practice or fatigue )Hawthorne effect. Researchers often give special attention to their subjects; thisfactor, not the experimental treatment itself, may cause a change in )Novelty and disruption effects -- a novel experimental treatment may beeffective simply because it is different from what the subjects normally special problem for educational and clinical )Experimenter effect (Rosenthal effect).6)Pretest sensitization.

8 In some instances the pretest may act as part of theexperimental treatment and thus affect one's research results. If the experimentis repeated without the pretest, different research results will probably beobtained. For example, an investigator wishes to demonstrate that her courseon cultural sensitive improve students attitudes towards other ethnic Page 7 March 14, 2006 They administer an attitude test to students in her class and a group of studentsin physic class. The students in her class maybe sensitized to the )Posttest sensitization. The possibility exists that the results of an experiment aredependent upon the administration of a posttest. The posttest may cause certainideas presented during the treatment phase to fall into place.

9 The same wouldbe true for students in the above example when given at )Interaction of history and treatment effects. This point in time effect. Forexample, interest in high school students attitude on alcohol consumption andtwo high school die in an alcohol related auto accident. Or doing a study onAmerican opinion about capital punishment and JFK's is assassinated or CharlesMasion murders )Measurement of the dependent variable. Does the dependent variablerepresent the construct that you are measuring (is visual hallucinations ameasure of schizophrenia).10)Interaction of time of measurement and treatment effects. The usual practiceis to administer the posttest immediately after subjects have completed theexperimental treatment.

10 For example, changing stereotypes of ethnic groups byviewing a film on ethnic diversity an immediate posttest may show a change buta week later find no Page 8 March 14, 2006 designing an experiment, the researcher should carefully consider the real-life settingto which they wish to generalize the results of the experiment. the researcher should review the design of the experiment on each of the 10 factorsdescribed to determine the extent of discrepancy, if any, between the experimentalconditions and the real life setting. a discrepancy cannot be minimized it should be noted in the research report as a limiton the generalizability of the research of extraneous variables way to make results more generalizable is to control extraneous variables.