Is This a Trick Question? - Kansas State University

1 Is this aIs this aIs this aIs this aIs this aTrickTrickTrickTrickTrickQuestion?Quest ion? Question? Question? Question? A Short Guideto Writing EffectiveTest QuestionsIs this aIs this aIs this aIs this aIs this aTrickTrickTrickTrickTrickQuestion?Quest ion? Question? Question? Question? A Short Guideto Writing EffectiveTest QuestionsDesigned & Developed by:Ben ClayKansas Curriculum CenterFormatting & Text Processing by:Esperanza RootThis publication was developed by theKansas Curriculum Center with fundsprovided by the Kansas StateDepartment of printing:October, 2001 Preface .. 1-2 Generally .. 3-5 General Tips About Testing.

2 3-4 When to Use Essay or Objective Tests .. 4-5 Matching Learning Objectives with Test Items .. 5 Planning the Test .. 6-12 Cognitive Complexity .. 6-7 Content Quality .. 8 Language Appropriateness .. 9 Transfer and Generalizability .. 9 Fairness .. 10 How to Defeat Student Guessing .. 11 General Test Taking Tips .. 12 Multiple Choice Test Items .. 13-19 Section Summary .. 13 Test Your Knowledge .. 14 Suggestions for Writing Multiple Choice Test Items .. 15-16 Multiple Choice Test Taking Tips .. 17-18 Aim for Higher Levels of Learning .. 19 True-False Test Items .. 20-26 Section Summary.

3 20 Test Your Knowledge .. 21 Suggestions for Writing True-False Test Items .. 22-23 Extreme Modifiers and Qualifiers .. 23 True-False Test Taking Tips .. 24 Variations in Writing True-False Test Items .. 24-25 Aim for Higher Levels of Learning .. 26 Matching Test Items .. 27-33 Section Summary .. 27 Test Your Knowledge .. 28-29 Suggestions for Writing Matching Test Items .. 30-31 Matching Test Taking Tips .. 32 Variations for Creating Matching Tests .. 33 Table of ContentsTable of ContentsTable of ContentsTable of ContentsTable of ContentsCompletion or Fill-in-the-Blank Test Items .. 34-37 Section Summary.

4 34 Test Your Knowledge .. 35 Suggestions for Writing Completion Test Items .. 36-37 Completion Test Taking Tips .. 37 Essay Test Items .. 38-44 Section Summary .. 38"I'd Like to Use Essay Tests, " .. 39 Read'Em and Weep Essay Test Items .. 39 Test Your Knowledge .. 40 Suggestions for Writing Essay Test Items .. 41-42 Four-Step Process in Grading Essay Tests .. 43 Essay Test Taking Tips .. 44 Additional Types of Test Items .. 45-51 Problem Solving .. 45 Using Authentic Assessments .. 46-47 Grading Authentic Assessments .. 48 Rubric Development .. 52-60 Purpose of Testing .. 52 Tips on Test Construction.

5 52 Test Layout Tips .. 52 Returning Tests and Giving Feedback .. 53 Alternative Testing Modes .. 54 Creating Fair Tests and Testing Fairly .. 55"I'd Like to Use Essay Tests, " .. 56-57 Test Administration Assignment .. 58 Cognitive Domain Guide .. 59 Affective Domain Guide .. 60 Bibliography .. notable concern of many teachers is that they frequently havethe task of constructing tests but have relatively little training orinformation to rely on in this task. Is this a Trick question ? is aninformation sourcebook for writing effective test questions. Thecentral focus of the sourcebook s content is derived from standardsdeveloped by the National Center for Research on Evaluation, Stan-dards, and Student Testing (CRESST).

6 CRESST s criteria for establishing the technical quality of a testencompasses seven areas: cognitive complexity, content quality,meaningfulness, language appropriateness, transfer andgeneralizability, fairness, and reliability. Each aspect is discussed inthe sourcebook in a straight-forward, jargon-free One contains information concerning general test constructionand introduces the six levels of intellectual understanding: knowl-edge, comprehension, application, analysis, synthesis, and evalua-tion. These levels of understanding assist in categorizing testquestions, with knowledge as the lowest level.

7 Since teachers tendto construct questions in the knowledge category 80% to 90% of thetime, throughout the sourcebook are examples of or suggestions fordeveloping higher order thinking skills. this supports Kansas current Quality Performance Accreditation initiative which hasestablished content and performance standards that cannot bemeasured by low-level Two of the information sourcebook is devoted to actual testquestion construction. Because of the diversity of assessmentoptions, the sourcebook focuses primarily on paper-and-penciltests, the most common type of teacher-prepared assessment.

8 Fivetest item types are discussed: multiple choice, true-false, matching,completion, and essay. Information covers the appropriate use ofeach item type, advantages and disadvantages of each item type,and characteristics of well written items. Suggestions for addressinghigher order thinking skills for each item type are also sourcebook was developed to accomplish three outcomes:!Teachers will know and follow appropriate principles for devel-oping and using assessment methods in their teaching, avoidingcommon pitfalls in student tend to use tests thatthey have prepared themselvesmuch more often than any othertype of test.

9 (How Teaching Mat-ters, NCATE, Oct. 2000)While assessment options are di-verse, most classroom educatorsrely on text and curriculum-em-bedded questions and tests thatare overwhelmingly classified aspaper-and-pencil (National Com-mission on Teaching andAmerica s Future, 1996).Formal training in paper-and-pen-cil test construction may occur atthe preservice level (52% of thetime) or as inservice preparation(21%). A significant number ofprofessional educators (48%) re-port no formal training in devel-oping, administering, scoring, andinterpreting tests (EducationWeek, National Survey of PublicSchool Teachers, 2000 ).

10 Students report a higher level oftest anxiety over teacher-madetests (64%) than over standard-ized tests (30%). The top threereasons why: poor test construc-tion, irrelevant or obscure mate-rial coverage, and unclear direc-tions. (NCATE, Summary Dataon Teacher Effectiveness, TeacherQuality, and Teacher Qualifica-tions , 2001.)(Continued on next )ii!Teachers will be able to identify and accommodate the limitationsof different informal and formal assessment methods.!Teachers will gain an awareness that certain assessment ap-proaches can be incompatible with certain instructional three outcomes directly support the standards developed bya joint commission established by the National Education Associa-tion, the American Federation of Teachers, and the National Coun-cil on Measurement in Education.