STAR Reading™ Technical Manual - resources.renlearnrp.com

1 STAR reading Technical ManualRenaissance LearningPO Box 8036 Wisconsin Rapids, WI 54495-8036 Telephone: (800) 338-4204(715) 424-3636 Outside the US: : (715) 424-4242 Email (general questions): ( Technical questions): (international support): NoticeCopyright 2015 by Renaissance Learning, Inc. All Rights publication is protected by US and international copyright laws. It is unlawful to duplicate or reproduce any copyrighted material without authorization from the copyright holder. This document may be reproduced only by staff members in schools that have a license for STAR reading software. For more information, contact Renaissance Learning, Inc., at the address logos, designs, and brand names for Renaissance Learning s products and services, including but not limited to Accelerated Math, Accelerated Reader, AccelScan, AccelTest, AR, AR 360, ATOS, Core Progress, English in a Flash, Learnalytics, Progress Pulse, Renaissance Home Connect, Renaissance Learning, Renaissance Place, Renaissance-U, STAR, STAR 360, STAR Custom, STAR Early Literacy, STAR Math, STAR reading , STAR reading Spanish, Successful Reader, Subtext, and UClass, are trademarks of Renaissance Learning, Inc.

2 , and its subsidiaries, registered, common law, or pending registration in the United States and other countries. All other product and company names should be considered the property of their respective companies and and Macintosh are trademarks of Apple Inc., registered in the and other , LEXILE , and LEXILE FRAMEWORK are trademarks of MetaMetrics, Inc., and are registered in the United States and abroad. Copyright 2015 MetaMetrics, Inc. All rights SRRPiSTAR reading Technical ManualContentsIntroduction .. 1 STAR reading : Screening and Progress-Monitoring Assessment.. 1 Tier 1: Formative Assessment Process .. 1 Tier 2: Interim Periodic Assessments.. 2 Tier 3: Summative Assessments.

3 2 STAR reading Purpose .. 2 STAR reading Enterprise.. 3 Design of STAR reading .. 3 Three Generations of STAR reading Assessments .. 3 Overarching Design Considerations .. 4 Improvements to the STAR reading Test in Version 2 .. 6 Improvements Specific to STAR reading Versions 3 RP and Higher .. 7 Improvements Specific to STAR reading Version RP .. 8 Improvements Specific to STAR reading Version RP .. 8 Improvements Specific to STAR reading Enterprise .. 9 Test Interface .. 9 Practice Session .. 9 Adaptive Branching/Test Length ..10 Test Length: STAR reading .. 10 Test Length: STAR reading Enterprise .. 11 Test Repetition ..12 Item Time Limits ..13 Test Security..14 Split-Application Model.

4 14 Individualized Tests.. 14 Data Encryption .. 15 Access Levels and Capabilities .. 15 Test Monitoring/Password Entry .. 16 Final Caveat.. 16 STAR reading Enterprise and the Common Core State Standards ..16 Core Progress Learning Progression for reading and the Common Core State Standards .. 17 Test Administration Procedures ..18 ContentsiiSTAR reading Technical ManualContent and Item Development .. 19 Content Specification: STAR reading ..19 The Educational Development Laboratory s Core Vocabulary List: ATOS Graded Vocabulary List.. 19 Content Specification: STAR reading Enterprise ..19 Item Development Specifications: STAR reading ..23 Vocabulary-in-Context Item Specifications.

5 23 Authentic Text Passage Item Specifications .. 24 Item Development Specifications: STAR reading Enterprise ..27 Adherence to Skills .. 27 Level of Difficulty: Readability .. 27 Level of Difficulty: Cognitive Load, Content Differentiation, and Presentation .. 28 Efficiency in Use of Student Time .. 29 Balanced Items: Bias and Fairness.. 29 Accuracy of Content .. 30 Language Conventions .. 30 Item Components .. 30 Item and Scale Calibration .. 32 Background ..32 Calibration of STAR reading Items for Use in Version 2 ..33 Sample Description ..33 Item Presentation ..35 Item Difficulty..37 Item Discrimination ..37 Item Response Function ..38 Rules for Item Retention.

6 40 Calibration of Supplemental Items for Use in Version RP ..41 Computer-Adaptive Test Design ..43 Scoring in the STAR reading Tests ..44 Scale Calibration ..45 The Linking Study .. 45 Calibration of STAR reading Skills Items for Use in STAR reading Enterprise ..48 Reliability and Measurement Precision .. 4925-Item STAR reading Tests ..50 Generic Reliability .. 50 Split-Half Reliability .. 51 ContentsiiiSTAR reading Technical ManualAlternate Form Reliability .. 52 Standard Error of Measurement .. 5334-Item STAR reading Enterprise Tests ..55 Reliability Coefficients .. 55 Standard Error of Measurement .. 56 The National Center on Response to Intervention (NCRTI) and Screening.

7 58 The National Center on Intensive Intervention (NCII) and Progress Monitoring ..60 Validity.. 61 Relationship of STAR reading Scores to Scores on Other Tests of reading Achievement ..62 Relationship of STAR reading Scores to Scores on State Tests ofAccountability in reading ..83 Relationship of STAR reading Enterprise Scores to Scores on Previous Versions ..87 Meta-Analysis of the STAR reading Validity Data ..89 Post-Publication Study Data ..90 Predictive Validity: Correlations with SAT9 and the California Standards Tests ..90A Longitudinal Study: Correlations with SAT9.. 91 Concurrent Validity: An International Study of Correlations with reading Tests in England .. 92 Construct Validity: Correlations with a Measure ofReading Comprehension.

8 93 Investigating Oral reading Fluency and Developing the Estimated Oral reading Fluency Scale .. 95 Cross-Validation Study Results ..98 Summary of STAR reading Validity Data..99 Linking STAR and State Assessments: Comparing Student- and School-Level Data..99 Methodology Comparison ..100 Student-Level Data ..100 School-Level Data ..101 Accuracy Comparisons ..102 The National Center on Response to Intervention (NCRTI) and Screening .. 105 Aggregated Classification Accuracy Data ..106 Receiver Operating Characteristic (ROC) Curves as defined by NCRTI: ..106 Brief Description of the Current Sample and Procedure ..107 Aggregated Validity Data ..109 Disaggregated Validity and Classification Data.

9 109 The National Center on Intensive Intervention (NCII) and Progress Monitoring .. 111 ContentsivSTAR reading Technical ManualNorming .. 112 Test Score Norms .. 112 Sample Characteristics .. 113 Test Administration .. 116 Data Analysis .. 117 Growth Norms .. 118 Score Definitions .. 120 Types of Test Scores .. 120 Scaled Score (SS) ..121 Grade Equivalent (GE) ..121 Estimated Oral reading Fluency (Est. ORF) ..122 Comparing the STAR reading Test with Classical Tests ..123 Instructional reading Level (IRL) ..123 Special IRL Scores ..124 Understanding IRL and GE Scores ..124 Percentile Rank (PR) ..125 Normal Curve Equivalent (NCE) ..127 Student Growth Percentile (SGP).

10 127 Lexile Measures ..128 Lexile ZPD Ranges ..128 Lexile Measures of Students and Books: Measures of Student reading Achievement and Text Readability ..128 Special STAR reading Scores .. 130 Zone of Proximal Development (ZPD)..131 Diagnostic Codes ..131 Grade Placement.. 132 Indicating the Appropriate Grade Placement ..132 Compensating for Incorrect Grade Placements ..133 STAR reading in the Classroom .. 134 Goal Setting and Instructional Planning .. 134 Formative Assessment .. 136 Measuring Growth.. 137 Absolute versus Relative Growth ..137 Methods of Measuring Growth ..138 Pretest/Posttest Designs ..138 Longitudinal Designs ..141 Student Growth Percentile (SGP).