Two Rubrics for Critical Thinking Assessment: A Mini ...

1 1 Two Rubrics for Critical Thinking Assessment: A Mini-Training Session The 2005 Assessment Institute at IUPUI October 24, 2005 Facilitators Jerry K. Stonewater Susan K. Wolcott Univ. Director of Liberal Educ. & Assessment Independent Scholar & Consultant Miami University WolcottLynch Associates Oxford, OH Bellevue, WA Email: Email: Phone: 513-529-7135 Phone: 425-830-3962 Web site: Participant Outcomes: Identify Student Learning Outcomes for Critical Thinking Use a Rubric to Assess Student Papers Evaluate/Critique Assessment Rubric Design Recommend Improvements to Assessment Task Design 2 Identify Learning Outcomes for Critical Thinking ACTIVITY List Words/Phrases That Describe the Critical Thinking Outcomes You Would Like Students to Achieve.

2 3 Using a Rubric to Assess Critical Thinking RUBRIC: Set of scoring guidelines for assessing student performance Ideally, an Assessment Method Should: Link Assessment Results to Student Learning (Help Close the Loop ) Provide Students With Useful Feedback by Pointing to Ways They Can Improve 4 Miami University Experience Using WSU Rubric Available from Washington State University at The Critical Thinking Rubric 1) Identifies and summarizes the problem /question at issue (and/or the source's position). Scant Substantially Developed Does not identify and summarize the problem , is confused or identifies a different and inappropriate problem . Does not identify or is confused by the issue, or represents the issue inaccurately.

3 Identifies the main problem and subsidiary, embedded, or implicit aspects of the problem , and identifies them clearly, addressing their relationships to each other. Identifies not only the basics of the issue, but recognizes nuances of the issue. 2) Identifies and presents the STUDENT'S OWN perspective and position as it is important to the analysis of the issue. Scant Substantially Developed Addresses a single source or view of the argument and fails to clarify the established or presented position relative to one's own. Fails to establish other Critical distinctions. Identifies, appropriately, one's own position on the issue, drawing support from experience, and information not available from assigned sources.

4 3) Identifies and considers OTHER salient perspectives and positions that are important to the analysis of the issue. Scant Substantially Developed Deals only with a single perspective and fails to discuss other possible perspectives, especially those salient to the issue. Addresses perspectives noted previously, and additional diverse perspectives drawn from outside information. (continued) 5 4) Identifies and assesses the key assumptions. Scant Substantially Developed Does not surface the assumptions and ethical issues that underlie the issue, or does so superficially. Identifies and questions the validity of the assumptions and addresses the ethical dimensions that underlie the issue.

5 5) Identifies and assesses the quality of supporting data/evidence and provides additional data/evidence related to the issue. Scant Substantially Developed Merely repeats information provided, taking it as truth, or denies evidence without adequate justification. Confuses associations and correlations with cause and effect. Does not distinguish between fact, opinion, and value judgments. Examines the evidence and source of evidence; questions its accuracy, precision, relevance, completeness. Observes cause and effect and addresses existing or potential consequences. Clearly distinguishes between fact, opinion, & acknowledges value judgments. 6) Identifies and considers the influence of the context * on the issue.

6 Scant Substantially Developed Discusses the problem only in egocentric or sociocentric terms. Does not present the problem as having connections to other contexts-cultural, political, etc. Analyzes the issue with a clear sense of scope and context, including an assessment of the audience of the analysis. Considers other pertinent contexts. 7) Identifies and assesses conclusions, implications and consequences. Scant Substantially Developed Fails to identify conclusions, implications, and consequences of the issue or the key relationships between the other elements of the problem , such as context, implications, assumptions, or data and evidence. Identifies and discusses conclusions, implications, and consequences considering context, assumptions, data, and evidence.

7 Objectively reflects upon the their own assertions. (continued) 6 Contexts for Consideration 1. Cultural/Social Group, national, ethnic behavior/attitude 2. Scientific Conceptual, basic science, scientific method 3. Educational Schooling, formal training 4. Economic Trade, business concerns costs 5. Technological Applied science, engineering 6. Ethical Values 7. Political Organizational or governmental 8. Personal Experience Personal observation, informal character 7 A Cognitive Development Approach to Critical Thinking Rubric Design Steps for Better Thinking performance patterns 0, 1, 2, 3, & 4 correspond to Reflective Judgment Stages 3, 4, 5, 6, & 7.

8 Benefits of a Rubric Based on Levels of Cognitive Development: Improve insights about students Critical Thinking strengths and weaknesses. Identify the next steps in building student Critical Thinking skills . Provide students with more appropriate feedback for student learning. Improve interrater reliability. References: King, P. M., & Kitchener, K. S. (1994). Developing reflective judgment: Understanding and promoting intellectual growth and Critical Thinking in adolescents and adults. San Francisco: Jossey-Bass. Kitchener, K. S., & King, P. M. (1985/1996). Reflective judgment scoring manual. Reflective Judgment Associates. Fischer, K. W., & Pruyne, E. (2002).

9 Reflective Thinking in adulthood: Development, variation, and consolidation. In J. Demick & C. Andreoletti (Eds.), Handbook of adult development (pp. 169-197). New York: Lynch, C. L., & Wolcott, S. K. (2001). Helping your students develop Critical Thinking skills (IDEA Paper #37). Manhattan, KS: The IDEA Center. Available at Wolcott, S. K. and C. L. Lynch. (2002). Developing Critical Thinking skills : The Key to Professional Competencies. A tool kit. Sarasota, FL: American Accounting Association. skills in the Scoring Manual for the Reflective Judgment Interview Rubrics Based on a Model of Open-Ended problem solving skills : Steps for Better Thinking Rubric Steps for Better Thinking Competency Rubric 8 Steps for Better Thinking Rubric Less Complex Performance Patterns More Complex Performance Patterns Steps for Better Thinking skills "Confused Fact Finder" Performance Pattern 0 How performance might appear when Step 1, 2, 3, and 4 skills are weak "Biased Jumper" Performance Pattern 1 -How performance might appear when Step 1 skills are adequate, but Step 2, 3, and 4 skills are weak "Perpetual Analyzer" Performance Pattern 2 -How performance might appear when Step 1 and 2 skills are adequate.

10 But Step 3 and 4 skills are weak "Pragmatic Performer" Performance Pattern 3 -How performance might appear when Step 1, 2, and 3 skills are adequate, but Step 4 skills are weak "Strategic Re-Visioner" Performance Pattern 4 -How performance might appear when one has strong Step 1, 2, 3, and 4 skills Step 1: IDENTIFY A Identify and use relevant information B Articulate uncertainties A0 Uses very limited information; primarily "facts," definitions, or expert opinions B0 Either denies uncertainty OR attributes uncertainty to temporary lack of information or to own lack of knowledge A1 Uses limited information, primarily evidence and information supporting own conclusion* B1 Identifies at least one reason for significant and enduring uncertainty* A2 Uses a range of carefully evaluated, relevant information B2 Articulates complexities related to uncertainties and the relationships among different sources of uncertainty A3 Uses a range of carefully evaluated, relevant information.