Program Components and Their Relationship to Student ...

1 Reprinted from the Journal of Developmental Education, Volume 20, Issue 3, Spring, Components and Their Relationship to StudentPerformanceBy Hunter R. Boylan, Leonard B. Bliss, and Barbara S. BonhamABSTRACT: Much has been written in the past 20 years regarding the impact of various programcomponents on the success of developmental students . Among these Components , centralized programorganization, mandatory assessment, mandatory placement, tutoring, advising, and Program evaluationhave been mentioned most frequently.

2 As part of the National Study of Developmental Education severalof these Components were explored to determine Their Relationship to such measures as first-term GPA,cumulative GPA, retention, and performance in developmental courses. All of these Components werefound to have some Relationship to the success measures being studied. Centralized programorganization, tutoring with tutor training and systematic Program evaluation were found to be related tothe highest number of success measures. In November of 1988, the Exxon Education Foundation awarded a major research grant to theNational Center for Developmental Education.

3 The purpose of this grant was to conduct a national studydesigned to assess the efficacy of developmental education. The project studied developmental programsand students to determine the impact of developmental education, to identify successful developmentaleducation techniques and Components , and to determine what is known and not known aboutdevelopmental education. The Exxon Education Foundation provided over $750,000 for this project from 1988 to 1994 and iteventually became known as the National Study of Developmental Education.

4 The project is ongoingand, since 1994, has been supported by additional funds from the Exxon Education Foundation, theAlfred P. Sloan Foundation, the National Association for Developmental Education, and the NationalCenter for Developmental Education. The National Study of Developmental Education has explored a wide variety of issues discussed inthe literature of the field. One of the most important of these is the study of those service componentsmost commonly found in developmental programs . This article will examine which of these componentsare most closely associated with the success of students participating in developmental The population from which the sample of institutions was drawn included over 3,000 colleges anduniversities in the United States identified by Peterson's College Data Base (1989).

5 Proprietary andmilitary institutions were deleted from this population and a representative sample was selected from theremaining institutions. Using a circular, systematic sampling procedure, 160 institutions were selectedthrough a series of replicated subsamples. The subsamples were stratified by institutional type andgeographical region to insure representativeness. At each institution selected for inclusion in the study, a random sample of developmental students wasidentified. students were identified as developmental according to local criteria.

6 Whoever was placed, asa result of institutional assessment and/or advising in developmental courses or programs , wasconsidered to be a developmental Student for the purposes of the study. A random sample of just underProgram of 104/24/12 8:27 AM6,000 students was eventually selected for inclusion in the study. A review of the literature was then conducted to identify Program Components considered to beassociated with Student success. Based on this review, the following Components were identified:presence of centralized Program organizational structure, presence of mandatory assessment of students ,presence of mandatory placement of students ,availability of tutor training,availability of advising/counseling services, andpresence of Program evaluation.

7 A 44-page questionnaire was used to collect information about the developmental education programsat participating institutions and to identify whether or not a given component was present or absent. Thisinformation was then entered into the Program database. The data about Student performance , includingretention, first-semester GPA, cumulative GPA, and success in developmental courses, was obtainedfrom individual Student transcripts and entered in the Student database. The analysis for this study involved combining data from the Student and the Program databases.

8 Thestatistical analysis generating findings about Program Components involved several different tests whichwere conducted using the Statistical Package for the Social Sciences (SPSS) (1991) software observed significance level for all findings reported as statistically significant was less than .05(p< ). The causal-comparative method was used to explore the relationships among variables used in thestudy. This method focuses on revealing possible relationships between variables. This method issometimes referred to as "ex post facto research, since causes are studied after they have presumablyexerted Their effect on another variable" (Borg & Gall, 1983, p.)

9 533). The major advantage of thismethod is that it enables the study of relationships between many variables. The statistical tests usedmost frequently in causal comparative research are the t test, analysis of variance (ANOVA), or one ofthe nonparametric tests such as chi square. The independent variables, that is, each of the Program Components , were measured according towhether or not they were present. The dependent variables, first-semester GPA, cumulative GPA,retention, and success in developmental courses, differed in Their levels of measurement because theyrepresented different types of data.

10 Because the way in which data is analyzed depends upon the way itis measured, various statistical tests were used depending on the type of variable. Retention and success in developmental courses are nominal variables, also referred to as categoricalor dichotomous variables. The categories used for retention were: (a) students who have graduated or arestill retained at the institution and, (b) students who have withdrawn from the institution. Graduation andretention was measured after 5 1/2 years at 4-year institutions and 3 1/2 years at 2-year institutions.