DETAILED PRENTICE HALL SCIENCE Science …

1 PRENTICE HALLREACHINGTODAY SSTUDENTSPRENTICE hall SCIENCE2000 2001 Study Comparing Performanceof Students Using PRENTICE hall SCIENCE ExplorerwithStudents Using Other SCIENCE ProgramsABSTRACTThis study investigated the effects of SCIENCE textbook programs at the eighthgrade level. Eighth- grade SCIENCE students were assigned to either a study group( PRENTICE hall text) or a control group (current text). Students were tested at thestart of the academic year with a nationally normed standardized test (theTerraNovaTMCTBS Complete Battery Plus). At the end of the full-year treatmentperiod, students were re-tested with the same standardized test. The eighth-gradestudystudents utilizing the PRENTICE hall SCIENCE Explorerprogramshowed significant learning improvement, while the controlstudents using other SCIENCE textbook programs did not.

2 OBJECTIVEThe object of this project was to determinewhether students who were enrolled in classes using PRENTICE hall SCIENCE Explorerincreased their SCIENCE knowledge and skills after using the program. An additional objective was to measure whether the use of the SCIENCE Explorerprogram increased student knowledge and skills at significantly higher levels than other textbook programs. The measures that were used were the NCE (Normal Curve Equivalent) and the OPI (Objectives Performance Index) of the TerraNovaTMCTBS Complete Battery study was designed so that there was a study groupto whom the PrenticeHall program was administered and a control groupto whom the program wasnot administered. The control group used whatever SCIENCE program had beenadopted for use in the groups were tested in September 2000, prior to the program s introduction,and then again at the end of the school year, in May 2001.

3 Therefore, for boththe study group and the control group, there was a pre-test score and a post-test score. Only students who completed both the pre- and post-tests wereincluded in this analysis. A total of 12 eighth- grade SCIENCE classes comprising223 students participated in the GroupPrentice hall SCIENCE ExplorerTextbook ProgramControl GroupOtherTextbook Programs6 classes115 studentsPRE-TEST(administered during1st week of school)6 classes108 studentsPOST-TEST (administered at end of school year)6 classes108 students6 classes115 studentsDETAILEDREPORTW here possible, one teacher taught both study and control classes. In only onesituation, where the same teacher could not teach both study and control classes,another teacher of similar teaching style, background, and tenure was selectedto teach the control class.

4 Both study and control classes were in the same schoolbuilding. Study and control classes were selected to be similar in student abilitylevels. Of the six schools participating in this research project, three schools were in rural settings, two were in suburban settings, and one was in an urban setting. These schools had a range of sizes. Study participants were from fourstates: Colorado, New Jersey, Washington, and tests were used to examine two issues with regard to program effec-tiveness: (1) that SCIENCE knowledge and skills increased among students usingPrentice hall SCIENCE Explorerfrom the pre-test to the post-test, and (2) the gainsfor the study group ( SCIENCE explorer ) were above that of the control group(other texts).STEP 1 This step consisted of an analysis of whether or not there was astatistically significant difference between the study group and the controlgroup on the pre-test this, a t-teston the difference between thestudy and control mean pre-test scores was used.

5 The measure used in this analysis was the NCE (Normal Curve Equivalent) of the TerraNova CTBS Complete Battery hypothesis of the t-testsshown below is that the pre-test means of the studyand control groups are equal; the alternative hypothesis is that they are notequal, for which a two-tail test is appropriate. If the significance of the t-value isless than or equal to .05, then the hypothesis is rejected at the 95% (5% signifi-cance) level and the alternate hypothesis is the significance of the t-value is greater than .05 (that is, , as shownbelow), we accept the hypothesis; that is, that the pre-test means of the studyand control groups are equal at the 95% level of confidence, showing nosignificant differences at the starting point of the 2 This step consisted of an analysis of whether or not there was a statistically significant difference between the pre-test and post-test scoreswithin the study group and within the control group.

6 For each group, a t-teston the difference between pre-test and post-test scores was used. The NCE was the measure used in this hypothesis of the t-tests shown at right is that the pre-test and post-testmeans are equal; the alternative hypothesis is that they are not equal. If the significance of the t-value is less than or equal to .05, then the hypothesis isrejected at the 95% (5% significance) level and the alternate hypothesis the study group, the significance of the t-value is less than .05 (that is, ,as shown at right). Therefore we accept the alternative hypothesis: the pre-testand post-test means of the study group are not equal at the 95% level of confidence. Thus, students using the PRENTICE hall SCIENCE Explorerprogram showed significant improvement in test scores from the pre-test to the pre-testNCE (base)Control pre-testNCE (base) Subject, GradeScience, (108) (115) (t-value) pre-testNCE (base)Study post-testNCE (base) Subject, GradeScience, (108) (108) (t-value) pre-testNCE (base)Control post-testNCE (base) Subject, GradeScience, (115) (115) (t-value) contrast, for the control group, the significance of the t-value is greater (that is, , as shown below).

7 Therefore we accept the original hypothesis:the pre-test and post-test means of the control group are equal at the 95% levelof confidence. Thus, students using other textbook programs did notshow significantimprovement in test scores from the pre-test to the Point Gain (pre to post)Control Mean Point Gain (pre to post) TerraNova CTBS Complete Battery Plus 8thGrade ScienceInstructional Objectives: SCIENCE InquiryPhysical ScienceLife ScienceEarth ScienceScience and TechnologyPersonal and Social Perspectives in ScienceTotal Gain+ + + + + + + + + + + + + + study group (using the PRENTICE hall SCIENCE Explorerprogram) showed significant learning improvement over the course of a full school year;the control group (using other textbook programs) did not show significantlearning improvement over this same period.

8 In addition, students using SCIENCE Explorershowed directionally higher gains on key SCIENCE instructionalobjectives compared to students using other textbook programs. The analysis of results also showed that teachers interactions with the SCIENCE Explorerprogram played a significant role in the outcome of student study of this phenomenon is 3In addition to NCE scores, performance on SCIENCE instructionalobjectives for the study and control groups was also analyzed. The measure used in this analysis was the OPI (Objectives Performance Index) of theTerraNovaTMCTBS Complete Battery Plus. Both students using the PRENTICE hall SCIENCE Explorerprogram (the study group)and students using other textbook programs (the control group) showed signifi-cant gains in key SCIENCE instructional areas over the course of a full school , students using the SCIENCE Explorerprogram appear to show gainsthat are directionally higher than students using other textbook POSTSCRIPTFor the purpose of completeness, we include here a brief discussion of the statistical tests and measures that were used.

9 All of the statistical tests are in the classical statistical domain, and are broadly used across all disciplines including psychometrics. The programs that were used in the computationalphase of this project were SPSS versions 6 was used states: The mean of one population is equal to themean of another population when the variance is unknown. The hypothesis is that the means are equal; the alternative hypothesis is that they are unequal,for which we use a two-sided test. The means and the variance are calculatedand the t-statistic is computed. The significance of the t-statistic is then computed. If the significance of the t-statistic is less than .05, the hypothesis isrejected at the 95% level of confidence; otherwise, the alternative hypothesis Differences:are cited as well as significant differences if they appearto lend understanding to the study results.

10 Directional differences should beapproached with more caution than significant differences. Normal Curve Equivalents: Comparison of Scores across Tests. The normal curve equivalent (NCE) scale, ranging from 1 to 99, coincides with the nationalpercentile scale at 1, 50, and 99. NCEs have many of the same characteristics aspercentile ranks, but have the additional advantage of being based on anequal-interval scale. The difference between two successive scores on the scalehas the same meaning throughout the scale. This property allows you to makemeaningful comparisons among different achievement test batteries and amongdifferent tests within the same battery. You can compare NCEs obtained by different groups of students on the same test or test battery by averaging thetest scores for the groups1.