Longitudinal Predictive Validity of a Learning …

1 Longitudinal Predictive Validity of a Learning Potential TestMari de BeerUniversity of South AfricaAddress correspondence to Mari de Beer, Department of Industrial and Organizational Psychology, Unisa, Box 392, Unisa,Pretoria 3000, South Africa. E-mail: Predictive Validityde BeerThe aim of this article is to provide empirical psychometric evidence of the ( Longitudinal ) Predictive Validity of a learningpotential measure the Learning Potential Computerised Adaptive Test (LPCAT) in comparison with standard statictests with school aggregate results as the criterion measure. Participants were 79 boys (mean age , SD = ) and72 girls (mean age , SD = ) attending two private schools. Correlation and regression analyses were used toevaluate the Predictive Validity of the Learning potential and standard test scores for school aggregate academic resultsas criterion measure.

2 Results indicate that Learning potential scores were statistically significant predictors ofaggregate academic results and provided results that were comparable to those of the standard test results providingempirical support for the use of Learning potential tests in mainstream educational : dynamic assessment (DA); Longitudinal Predictive Validity ; item response theory (IRT); computerised adaptive test-ing (CAT); Learning Potential Computerised Adaptive Test (LPCAT)The measurement of Learning potential has received consid-erable attention over the years (Budoff, 1987; Caffrey, Fuchs, &Fuchs, 2008; Embretson, 2004; Feuerstein, 1979; Grigorenko,2009; Grigorenko & Sternberg, 1998; Haywood, 2008; Lidz,1987a, 1987b, 1991, 2009; Murphy & Maree, 2006). This ap-proach to cognitive assessment which typically involves alearning opportunity during assessment in a test-train-retestprocess - is also referred to as dynamic assessment (DA).

3 Based on Vygotsky s concept of the zone of proximal develop-ment (ZPD) (Vygotsky, 1978), it allows for measurement of cur-rent as well as projected future levels of performance (Lidz,1991). It is generally considered fairer for the assessment ofdisadvantaged individuals because it allows for Learning duringassessment. However, despite sustained interest in DA overthe years, it has not yet been incorporated into mainstream as-sessment in education or in industry alongside standard as-sessment methods (Grigorenko & Sternberg, 1998; Lidz, 2009;Sternberg & Grigorenko, 2002). Reasons given are that thetest-train-retest strategy often results in longer testing times,measurement problems with comparison of resulting pre-testand post-test scores and limited available psychometric infor-mation on DA measures (Beckmann, 2006; Lidz, 2009; Stern-berg & Grigorenko, 2002).

4 Continuing research to investigate the psychometric proper-ties of DA tools is important if they are to be considered andused more widely in educational and industry settings (Caffreyet al., 2008; Lidz, 2009). Provision of information that allows forcomparison of the Predictive Validity of DA versus standardmeasures such as the results presented in this article couldbe useful in this Learning Potential ComputerisedAdaptive Test (LPCAT)The Learning Potential Computerised Adaptive Test(LPCAT: De Beer, 2000a, 2000b, 2005) uses IRT and CAT toaddress issues about DA noted by other authors (Embretson,1996; Embretson & Reise, 2000; Sijitsma, 1993a, 1993b).Three different types of nonverbal figural item formats are usedin the LPCAT namely figure series, figure analogies and patterncompletion.

5 Since they measure fluid general reasoning ability,they do not rely on language proficiency or scholastic back-ground and are deemed to be more equitable than items withverbal or scholastic content (Claassen, 1997; Foxcroft, 2004;Hugo & Claassen, 1991; Owen, 1998). In the typicaltest-train-retest approach of most DA tests, the LPCAT usestwo separate but linked adaptive tests with a focus on measure-ment of fluid ability by means of nonverbal figural content (DeBeer, 2005). A unidimensional structure was indicated by factoranalysis of items in the item banks and coefficient alpha valuesranging between and for different groups havebeen reported (De Beer, 2005).The Legal-professional ContextThe South African Employment Equity Act (EEA, 1998) re-quires psychological tests used to show evidence of the accept-ability of their psychometric properties and fairness towards dif-ferent groups (Foxcroft, 2004; Paterson & Uys, 2005).

6 Research results on the LPCAT have shown evidence on its re-liability and Validity for different groups (De Beer, 2000b, 2005,2006; Van der Merwe & De Beer, 2006; Van Eeden, De Beer &Coetzee, 2001). The fact that differential item functioning (DIF)analysis was used during test construction to eliminate biaseditems (De Beer, 2004), that the LPCAT incorporates DA andCAT in a power test with non-verbal figural content, uses onlythe space bar and Enter keys during administration and that thetest instructions are available in all 11 South African indigenouslanguages goes some way with regard to the fairness require-ment in the South African use of IRT and CAT provide several advantages to DA,namely accurate measurement of difference scores, shortenedtest times, equivalent measurement accuracy at all levels ofability and use of items that match the estimated ability level ofindividuals being tested throughout both the pre- and post-tests(Kim-Kang & Weiss, 2008.)

7 Van der Linden, 2008a, 2008b;Journal of Psychology in Africa 2010, 20(2), 225 232 Printed in USA - All Rights ReservedCopyright 2010 Journal ofPsychology in AfricaISSN 1433-0237 Weiss, 1983). In addition to the pre-test, post-test and differ-ence scores, the LPCAT also provides a composite score whichrepresents a global potential score. The composite score repre-sents a reasoned combination of the pre- and post-test scoreswhich allows for the fact that it is more difficult to show improve-ment when the initial pre-test performance level is high com-pared to a lower initial level of performance (De Beer, 2005).Research results for the LPCAT have generally shown that thepost-test score (which allows for maximum credit for the learn-ing that has been achieved) and the composite score (which al-lows for partial credit for the Learning that has been achieved)improve Predictive Validity compared to the pre-test score(which indicates current level of performance) (De Beer, 2005,;2008).

8 The South African National Qualifications Framework(NQF) provides level descriptors for all formal qualifications. Interms of LPCAT performance levels, the performance levels ofgroups at specific educational levels have been benchmarkedagainst the NQF framework (see table 1).These levels of performance can be used as a guideline ininterpreting LPCAT results to determine how close the individ-ual is to the indicated LPCAT level associated with a particularqualification level or in other words how much effort will be re-quired of the individual to reach the target level. Construct andpredictive Validity results for the LPCAT for samples at variouseducational levels have been reported (De Beer, 2000b, 2005,2006; Van der Merwe & De Beer, 2006; Van Eeden et al., 2001).

9 Goals of the StudyThe present study provides Longitudinal Predictive validityresults for the LPCAT and standard proficiency tests using ag-gregate academic performance as criterion measure. Resultsare presented for two separate studies which both involvedgathering of criterion data two years after the initial light of the stated need for Validity evidence of DA results(Caffrey et al., 2008; Lidz, 2009; Sternberg & Grigorenko, 2002)these results were deemed appropriate to present as evidencefor the Longitudinal Predictive Validity of a Learning potential testin comparison with standard (static) tests. The specific researchquestions were:1. Are LPCAT Learning potential results statistically significantpredictors of aggregate academic performance at a second-ary school level over a two-year period of time?

10 2. Are LPCAT Learning potential results comparable to stan-dard test results in terms of predicting aggregate academicperformance at a secondary school level over a two-yearperiod of time?MethodParticipants and SettingTwo independent sample groups from a boys only school(N=79; mean age , SD= ) and a girls only school(N=72 mean age , SD= ) respectively were both as-sessed at a senior primary level. The male sample group wereat grade 7 level at the time of the assessment and was evalu-ated prior to entry to the junior secondary (Grade 8) level. Thefemale sample group were at grade 6 level at the time of the as-sessment and was also evaluated prior to entry to the specificschool. Assessment in both cases was aimed at obtaining infor-mation on the performance levels of individuals and of thegroups on the selected measures to inform the schools of theprofile of prospective sample groups included pupils from different languageand culture groups but the specific demographic information ofindividual pupils in terms of language and culture groups wasnot made available to the and MeasuresParticipants took the English proficiency test (ProficiencyTest English second language intermediate level) (Chamberlain& Van der Schyff, 1991), two subtests of a standardised apti-tude test (for English language and for calculations respec-tively) (Claassen, Van Heerden, Vosloo & Wheeler, 2000), andthe LPCAT (De Beer, 2005).