Educational Research - ODU - Old Dominion …

1 EducationalResearchFundamentals forthe ConsumerSECOND EDITIONJAMES H. MCMILLANV irginia Commonwealth University* HarperCollins CollegePublishersEx~urirr Editor: CbristopherJrnnisonProject Coordination and Text Design: Rottle, Shaw & M rtberill, Desigmr: Kay PrtronioElectronic Production ~Manager: Christine PearsonElectronic Page Makeup: RuttIe, Sbaw & Wrtherill, Manager: Helenr C. LandersPrinter and Binder: R. R. Donnellc~ & Sons CompanyCover Printer: The Lrhigh Press, perrmssmn IO use copynghted marcrial, gratetirl acknowledgment is madeto thr copyright holders on pp. 358-359,which are hereby made pars of rhiscopyright 0 I996 by James H. McMillan411 rights reseru-ed. Printed in the United Statcr of Amrrica. No part ofthis book may be usrd or reproduced in any mannrr rvhatsoevrr withoutwritwn permission, except in the case of brief quotations embodied incritical articles and reviews.

2 For infonnarion addrrss HarperCollins CollegePublishers, 10 East 53rd Street, NewYork, r\Y 10022. ForinJarmatiun aboutany HarperCollins title, product, or raou~m, plmsr visit our World Wide Web site athttp:,, , James Research : fundarrrenrals for rhc consumer / James - 2nd bibiiographical references and O-675-99864-91. 1996370 .78--dc2095-16506 CIP95969798 987654321 ContentsTo the InstructorxvTo the Studentxix1. Introduction to Research inEducation 1 SOURCES OF KNOWLEDGE 2 Personal Experience 2 Tradition 3 Authority 3 The Scientific Approach 4 THE NATURE OF SCIENTIFIC INQUIRY4 The Purpose of Scientific Inquiry4 Characteristics of Scientific Inquiry5 The Purpose of Theories6 APPLYING SCIENTIFIC INQUIRY TOEDUCATION 7 TYPES OF Educational RESEARCH9 Two Traditions of Research : Quantitative andQualitative 9 BasicResearch 10 Applied Research10 action Research12 Evaluation Research 12 Nonexoerimental13I~ Research 12 Experimental ResearchFORMAT TO REPORT EDUCATIONALRESEARCH 14 Title and Author(s) 16 Abstract 16 Introduction 16 Review of LiteratureSpecific Research Question or HypothesisMethod and Design 17 Results 17 Discussion 17 Conclusions 18 References 181617viCONTENTSANATOMY OF A Research ARTICLE16 OUTLINE SUMMARY 18 STUDY QUESTIONS 27 SAMPLE TEST QUESTIONS 282.

3 Variables, Research Problems,and Hypotheses 31 VARIABLES IN Educational RESEARCH32 Constitutive and Operational Definitions32 Types of Variables33 Research PROBLEMS 36 Sources for Research Problems39 CONSUMER TIPS: CRITERIA FOR EVALUATINGRESEARCH PROBLEMS 42 HYPOTHESES 46 Why Researchers Use Hypotheses46 Types of Hypotheses 47 CONSUMER TIPS: CRITERIA FOR EVALUATINGRESEARCH HYPOTHESES 49 OUTLINE SUMMARY 51 STUDY QUESTIONS 52 SAMPLE TEST QUESTIONS533. Locating and ReviewingRelated Literature 55 THE PURPOSE OF REVIEWING RELATEDLITERATURE 56 Refining the Research Problem56 Developing Significance for the Research56 Identifying Methodological Techniques 57 Identifying Contradictory Findings 57 Developing Research Hypotheses 57 Learning About New Information57 CONTENTSviiSTEPS TO REVIEW RELATED LITERATURE58 Step One: Locate Existing Reviews and OtherInformation in Secondary Sources58 Step Two: Identify Key Terms65 Step Three: Identify the Appropriate JournalIndexes and Abstracts 67 Step Four: Search Indexes for PrimarySources 68 Step Five: Summarize and Analyze PrimarySource Information 74 Step Six: Organize the Review76 CONSUMER TIPS: CRITERIA FOR EVALUATINGTHE REVIEW OF LITERATURE77 OUTLINE SUMMARY 80 STUDY QUESTIONS 81 SAMPLE TEST QUESTIONS 824.

4 Subjects and Sampling 84 INTRODUCTION TO SAMPLING 85 What Is a Subject?85 What Is aPopulation? 85 What Is a Sample?86 TYPES OF SAMPLING PROCEDURES86 Probability Sampling 86 NonprobabilitySampling 91 HOW SUBJECTS AND SAMPLING AFFECTRESEARCH 94 Knowledge of Sampling Procedures94 Volunteer Samples 94 Sample Size 96 Subject Motivation 97 Sampling Bias 98 CONSUMER TIPS: CRITERIA FOR EVALUATINGSUBJECTS SECTIONS OF REPORTS ANDSAMPLING PROCEDURES 98 OUTLINE SUMMARY 100 STUDY QUESTIONS 101 SAMPLE TEST QUESTIONS 101viiiCONTENTS5. Foundations of EducationalMeasurement 104 INTRODUCTION TO MEASUREMENT 105 Definition of Measurement 105 The Purpose of Measurement for Research106 Scales of Measurement 106 NNDAMENTAL PRINCIPLES OF DESCRIPTIVESTATISTICS FOR UNDERSTANDINGMEASUREMENT 108 Frequency Distributions 109 Measures ofCentral Tendency 112 Measures ofVariability 113 Correlation 115 VALIDITY OF Educational MEASURESll6 Definition of Validity118 Types ofEvidence for Judging Validity119 Effect ofValidity on Research 122 RELIABILITY OF EDUCATIONALMEASURES of Reliability 124 Effect ofReliability on Research 127 OUTLINE SUMMARY QUESTIONS 130 SAMPLE TEST QUESTIONS1316.

5 Types of EducationalMeasures 134 CLASSIFYING Educational MEASURES and Criterion-Referenced Tests136 Standardized Tests 137 Interpreting TestScores 141 PERSONALITY ASSESSMENT 143 CONTENTSixATTITUDE, VALUE, AND INTERESTINVENTORIES 144 Types of Inventories 145 Problems inMeasuring Noncognitive Traits 148 OBSERVATIONS 150inference 150 Laboratory Observation 151 Structured Field Observations 152 Observer Effects 153 INTERVIEWS 154 Types of interview Questions155 Interviewer Effects 155 LOCATING AND EVALUATING EDUCATIONALMEASURES 157 CONSUMER TIPS: CRITERIA FOR EVALUATINGINSTRUMENTATION 158 OUTLINE SUMMARY 182 STUDY QUESTIONS 184 SAMPLE TEST QUESTIONS t&e, Cowela tional, andCausal-Comparative Research167 THEPURPOSEOFNONEXPERIMENTALRESEARCH 188 DESCRIPTIVE STUDIES 188 Characteristics of Descriptive Studies168 CONSUMER TIPS: CRITERIA FOR 5 VALUATlNGDESCRIPTIVE STUDIES 170 RELATIONSHIP STUDIES 171 Relationship Determined by Differences171 Simple Correlational Studies 172 Prediction Studies 176 CONSUMER TIPS: CRITERIA FOR EVALUATINGCORRELATIONAL STUDIES 178 ICONTENTSUSING SURVEYS IN DESCRIPTIVE ANDRELATIONSHIP STUDIES 182 Cross-Sectional Surveys 182 Longitudinal Surveys 183 CAUSAL-COMPARATIVE STUDIES Post Facto Research184 CorrelationalCausal-Comparative Research 186 CONSUMER TIPS: CRITERIA FOR EVALUATINGCAUSAL-COMPARATIVE Research 186 OUTLINE SUMMARY 187 STUDY QUESTIONS 168 SAMPLE TEST QUESTIONS 1698.

6 Experimental andSingle-Subject Research 192 CHARACTERISTICS OF EXPERIMENTALRESEARCH 193 EXPERIMENTAL VALIDITY 194 History 195 Selection 196 Maturation 196 Pretesting 197 Instrumentation 197 TreatmentReplications 198 Subject Attrition 198 Statistical Regression 198 Diffusion of Treatment 199 ExperimenterEffects 199 Subject Effects 200 TYPES OF EXPERIMENTAL DESIGNS201 Single-Group Posttest-Only Design 201 SingleGroup Pretest-Posttest Design 202 Nonequivalent-Groups Posttest-Only Design 203 Nonequivalent-Groups Pretest-PosttestDesign 204 Randomized-Groups Posttest-Only Design 206 Randomized-Groups Pretest-PosttestDesign 207 Factorial Experimental Designs 209 CONSUMER TIPS: CRITERIA FOR EVALUATINGEXPERIMENTALRESEARCH 210 CONTENTSxiSINGLE-SUBJECT Research 212 Characteristics of SingleSubject Research212 Types of Single-Subject Designs213 CONSUMER TIPS: CRITERIA FOR EVALUATINGSINGLE-SUBJECT Research 215 OUTLINE SUMMARY 217 STUDY QUESTIONS 218 SAMPLE TEST QUESTIONS 2199.

7 Analyzing StatisticalInferences 221 THEPURPOSEANDNATUREOFINFERENTIALSTATISTI CS 221 Degree of Certainty 222 Estimating Errorsin Sampling and Measurement222 TheNull Hypothesis 223 INTERPRETING RESULTS OF INFERENTIALTESTS 225 The t-Test 226 Simple Analysis ofVariance 227 Factorial Analysis ofVariance 228 Analysis of Covariance229 Multivariate Statistics 230 Chi-Square 230 CONSUMER TIPS: CRITERIA FOR EVALUATINGINFERENTIAL STATISTICS 232 OUTLINE SUMMARY 234 STUDY QUESTIONS 234 SAMPLE TEST QUESTIONS 23510. Qualitative and HistoricalResearch 238 QUALITATIVE Research 239 Characteristics of Qualitative Research239 Qualitative Research Problems 241xiiCONTENTSE ntering the Research Site243 SelectingParticipants 243 Obtaining QualitativeInformation 244 Analyzing QualitativeData 248 Credibility of QualitativeResearch 250 CONSUMER TIPS: CRITERIA FOR EVALUATINGQUALITATIVE Research 253 HISTORICAL Research 254 The Historical Method 255 CONSUMER TIPS.

8 CRITERIA FOR EVALUATINGHISTORICAL Research 258 OUTLINE SUMMARY 259 STUDY QUESTIONS 261 SAMPLE TEST QUESTIONS Discussion andConclusions 265 PURPOSEANDNATUREOFTHEDISCUSSION 265 INTERPRETATION OF THE RESULTS266 Interpretation Related to the Problemand/or Hypothesis 266 InterpretationRelated to Methodology 266 Interpretation Based on StatisticalProcedures 269 Interpretation Related toPrevious Research 270 CONCLUSIONS 272 Limitations 273 Recommendations andImplications 276 CONSUMER TIPS: CRITERIA FOR EVALUATINGDISCUSSION SECTIONS 277 OUTLINE SUMMARY 279 STUDY QUESTIONS 279 SAMPLE TEST QUESTIONS280 CONTENTSXIII12. The Intelligent Consumer:Putting It All Together282 QUESTIONS FOR QUANTITATIVE STUDIES263 QUESTIONS FOR QUALITATIVE STUDIES286 QUESTIONS FOR HISTORICAL STUDIES287 EXAMPLES OF Research ARTICLES288 Article 1: A Study of Academic Time-on-Task inthe Elementary School 288 Evaluation ofArticle 1 298 Article 2: Reducing TeacherStress 304 Evaluation of Article 2316 Article 3: Kindergarten Readiness andRetention: A Qualitative Study of Teachers Beliefs and Practices 322 Evaluation of Article 3349 Appendix: Answers to SampleTest QuestionshTef~t?

9 NcesAcknowledgmentsI&? %352353358360 SubjectsandSam@ingNonprobabilityAffects ResearchLimitationsof ProceduresThethird major part of Research reports is the methodology or metk-odssection. As noted in Chapter 1, the first subsection of themethodology section usually describes the subjects from whom data arecollected. The manner in which subjects are selected has important im-plications for identiwng factors that affect subject performance and forSUBJECTS AND SAMPLING85generalizing the results. Hence it is necessary to understand who thesubjects are and how they were TO SAMPLINGWhat Is a Subject?A subject is an individual who participates in a Research study or is some-one from whom data are collected. In experiments, for example, eachsubject: person from ,++,om dataare who is given a treatment and whose behavior is measured is con-sidered to be a subject. The term subject may also identify individualswhose behavior, past or present, is used as data, without their involve-ment in some type of treatment or intewention.

10 For instance, a re-searcher might use last year s fourth-grade test scores as data, and eachfourth-grader included is considered to be a subject. In qualitative re-search individuals are identified as patiicipants rather than Is a Population?A popuMion is a group of elements or cases, whether individuals, , or events, that conform to specific criteria and to which we intend~opu~ion: PWSOW to whomIwItS can be generalize the results of the Research . This group is also referred to asthe target p+~lation or ulziuene. The specification of the population be-gins with the Research problem and review of literature, through whicha population is described conceptually or in broad terms, for example,seventh-grade students, beginning teachers, principals, special educa-tion teachers, and so forth. A more specific definition is then needed,based on demographic characteristics. These characteristics are some-times referred to as delimitingvariables.