Florida Standards Assessments Achievement Level ...

1 Florida Standards Assessments Achievement Level Descriptions 2015 Updated January 25, 2016 Table of Contents Introduction .. 3 ALD Development Framework .. 3 Policy ALDs .. 3 Range ALDs .. 3 Target ALDs ( Just Barely ALDS) .. 4 Reporting ALDs .. 4 Range ALD Pre-Workshop Development .. 5 FSA Range ALD Workshop .. 5 Panel .. 6 Day 1 .. 8 Day 2 .. 10 Day 3 .. 11 Day 4 .. 11 References .. 12 Appendix A: Range ALDs for English Language Arts and Mathematics .. 13 Appendix B: Executive Summary ALDs for English Language Arts and Mathematics .. 154 3 Introduction Achievement Level descriptions (ALDs) describe a student s Level of Achievement ( , Below Satisfactory, Satisfactory, Above Satisfactory) on a large-scale assessment. The Florida department of education (hereafter referred to as the department ) developed a set of ALDs to guide (a) participants during the standard-setting process for its Florida Standards Assessments (FSA) in August 2015, (b) score interpretation on student reports, and (c) teacher understanding of expectations for the progressions of student performance at each Achievement Level .

2 With support from Christina Schneider from the National Center for the Improvement of Educational Assessment (Center for Assessment) and American Institutes for Research (AIR), the state of Florida developed the ALDs sequentially so that they could be interrelated and consistent with the ALD development process created by Egan, Schneider, and Ferrara (2012). The purpose of the ALD development framework is to enable valid inferences about student content area knowledge and skill in relation to a state s content Standards measured on a large-scale assessment. ALD Development Framework Egan, Schneider, and Ferrara (2012) proposed four stages of ALD development to correspond with the closely linked uses of ALDs in the field for item development, standard setting, and score reporting. The types of ALDs are Policy, Range, Target, and Reporting.

3 Their development is consistent with a construct-centered (Messick, 1994) and iterative (Plake, Huff & Reshetar, 2010) approach to assessment design. The types and purposes of ALDs are discussed below, beginning with Policy ALDs. Their development is where policymakers begin establishing the rigor of performance Standards . Policy ALDs In the first stage of the ALD development framework, the state develops Policy ALDs. Policy ALDs are important communication devices for a state s vision of what it means, for example, to be college and career ready. As such, a state optimally develops a policy-based claim. This claim clearly explicates the state s intended take-away message regarding a student s Achievement within each performance Level . The Policy ALDs should be consistent across grades with the exception of the policy description at the high school Level .

4 Nationally, the high school model is moving toward policy-based claims in regard to student readiness for college and careers. In Florida , Policy ALDs are known as the Achievement Level Policy Definitions (see Table 4). Range ALDs For each standard and performance Level on an assessment, Range ALDs should explicate observable evidence of Achievement , demonstrating how the skill changes and becomes more sophisticated across performance levels. Schneider, Huff, Egan, Gaines, and Ferrara (2013) wrote that for ALDs to be the foundation of test score interpretation, they should 4 reflect more complex knowledge, skills, and abilities (KSAs) as the performance levels increase ( , more complex KSAs should be expected for Advanced than for Proficient). This notion is consistent with what is termed a learning progression or learning trajectory in the research literature.

5 Learning trajectories are described increasingly in the literature as theoretical underpinnings for curriculum development, instruction, and assessment of learning. The purpose of a learning trajectory is to inform researchers and educators about general developmental pathways of learning so they can set reasonable, achievable learning goals and provide appropriate guidance for instruction and assessment in a given content area. Assessments that are being designed to measure student growth should be derived from the combination of the learning goals and the developmental progression engendered in a learning trajectory because, as Clements and Sarama (2004) wrote, "Developmental progressions .. [are] descriptions of children's thinking and learning .. and a related, conjectured route through a set of instructional tasks" (p.)

6 83). The outcome of instructional tasks or assessment tasks should be the same: observable evidence of what students know and can do. In terms of learning progressions, the Florida Standards may be thought of as the learning goals for students at each grade Level , and the Range ALDs may be considered developmental trajectories evidentiary statements regarding children s observable thinking and skills as they pass along the way to the learning goals. In the development of Range ALDs the state defines the expected learning trajectory, which is useful to teachers, but it also aligns the trajectory with its vision for student performance in terms of mastery of the content by embedding the item alignment framework ( , Depth of Knowledge) into the Range ALD descriptors. In Florida , these ALDs are known as the Range Achievement Level Descriptions.

7 Target ALDs ( Just Barely ALDS) Target ALDs define the state s policy and content-based expectations ( , what it means to be Proficient). Target ALDs are the lower-bound descriptions of a performance Level , and they are used to guide the cut score recommendation workshop. These descriptors target the skills all Proficient students, for example, should have in common. They are distilled from the Range ALDs and describe the minimum amount of information, for example, that the Just Proficient student should know and be able to do beyond, for example, the Basic Student. Because standard-setting participants are targeting the skills that separate the highest performing Basic student from the lowest performing Proficient student, these descriptions are shorter and describe just the skills that appear at the threshold between the two performance levels.

8 Reporting ALDs Reporting ALDs are optimally created after final cut scores are adopted. Reporting ALDs represent the reconciliation of the Target ALDs with the final cut scores. The target ALDs reflect a state s initial expectation for minimal student performance within an Achievement 5 Level , whereas the Reporting ALDs reflect actual student performance based upon the final approved cut scores. The reporting ALDs define the appropriate inferences stakeholders may make based upon the student s test score in relation to the final approved cut scores. A state should clearly explicate in the reporting ALDs whether the target student (the student located at the cut score) or typical student is being described. Some states are now using Range ALDs as the expanded version of a Reporting ALD. Because the trajectory can be either confirmed or contradicted when test data is reviewed, it is recommended that the Range ALDs be edited after the initial test administration.

9 Range ALD Pre-Workshop Development The department began the FSA Range ALD development process after reviewing versions of Range ALDs that are publicly available from other testing programs. The department s Test Development Center (TDC) staff worked with Christina Schneider from the Center for Assessment to express their vision regarding the interpretation of the Florida Standards and the Level of Achievement expected of students. Florida has determined that Level 3 on its Achievement Level Scale, which ranges from Level 1 to Level 5, indicates satisfactory performance. Levels 4 and 5 describe growth beyond the Level 3 expectations, and indicate proficiency in the Florida Standards . From March to April 2015, department staff fleshed out the evidence Level guideposts that describe the skills they expected to see as students worked toward deep understanding in the content areas.

10 This work was done so that Florida educators could convene to engage in the following tasks: 1. Evaluate the draft Range ALDs and provide recommendations for improving the draft language within each grade s ALDs, 2. Edit across grades for coherence of expectations, and 3. Identify places where language could be consolidated. FSA Range ALD Workshop The FSA ALD Workshop occurred April 28 May 1, 2015, in Tallahassee, Florida , for English language arts/literacy (ELA/literacy) in Grades 3 10 and mathematics in Grades 3 8, Algebra 1, Algebra 2, and Geometry. The purpose of the workshop was to refine the Range ALDs and obtain stakeholder input as well as develop bulleted executive summary descriptions for each Achievement Level based on the Range ALDs. The purpose of the bulleted summary descriptions was to distill the Range ALDs for standard-setting participants.