Is This a Trick Question? - Kansas State University

1 Is This aIs This aIs This aIs This aIs This aTrickTrickTrickTrickTrickQuestion?Quest ion? Question? Question? Question? A Short Guideto Writing EffectiveTest QuestionsIs This aIs This aIs This aIs This aIs This aTrickTrickTrickTrickTrickQuestion?Quest ion? Question? Question? Question? A Short Guideto Writing EffectiveTest QuestionsDesigned & Developed by:Ben ClayKansas Curriculum CenterFormatting & Text Processing by:Esperanza RootThis publication was developed by theKansas Curriculum Center with fundsprovided by the Kansas StateDepartment of printing:October, 2001 Preface .. 1-2 Generally .. 3-5 General Tips About Testing .. 3-4 When to Use Essay or Objective Tests .. 4-5 Matching Learning objectives with Test Items .. 5 Planning the Test .. 6-12 Cognitive Complexity .. 6-7 Content Quality .. 8 Language Appropriateness .. 9 Transfer and Generalizability .. 9 Fairness.

2 10 How to Defeat Student Guessing .. 11 General Test Taking Tips .. 12 Multiple Choice Test Items .. 13-19 Section Summary .. 13 Test Your Knowledge .. 14 Suggestions for Writing Multiple Choice Test Items .. 15-16 Multiple Choice Test Taking Tips .. 17-18 Aim for Higher Levels of Learning .. 19 True-False Test Items .. 20-26 Section Summary .. 20 Test Your Knowledge .. 21 Suggestions for Writing True-False Test Items .. 22-23 Extreme Modifiers and Qualifiers .. 23 True-False Test Taking Tips .. 24 Variations in Writing True-False Test Items .. 24-25 Aim for Higher Levels of Learning .. 26 Matching Test Items .. 27-33 Section Summary .. 27 Test Your Knowledge .. 28-29 Suggestions for Writing Matching Test Items .. 30-31 Matching Test Taking Tips .. 32 Variations for Creating Matching Tests .. 33 Table of ContentsTable of ContentsTable of ContentsTable of ContentsTable of ContentsCompletion or Fill-in-the-Blank Test Items.

3 34-37 Section Summary .. 34 Test Your Knowledge .. 35 Suggestions for Writing Completion Test Items .. 36-37 Completion Test Taking Tips .. 37 Essay Test Items .. 38-44 Section Summary .. 38"I'd Like to Use Essay Tests, " .. 39 Read'Em and Weep Essay Test Items .. 39 Test Your Knowledge .. 40 Suggestions for Writing Essay Test Items .. 41-42 Four-Step Process in Grading Essay Tests .. 43 Essay Test Taking Tips .. 44 Additional types of Test Items .. 45-51 Problem Solving .. 45 Using Authentic Assessments .. 46-47 Grading Authentic Assessments .. 48 Rubric Development .. 52-60 Purpose of Testing .. 52 Tips on Test Construction .. 52 Test Layout Tips .. 52 Returning Tests and Giving Feedback .. 53 Alternative Testing Modes .. 54 Creating Fair Tests and Testing Fairly .. 55"I'd Like to Use Essay Tests, " .. 56-57 Test Administration Assignment .. 58 Cognitive Domain Guide.

4 59 Affective Domain Guide .. 60 Bibliography .. notable concern of many teachers is that they frequently havethe task of constructing tests but have relatively little training orinformation to rely on in this task. Is This a Trick Question? is aninformation sourcebook for writing effective test questions. Thecentral focus of the sourcebook s content is derived from standardsdeveloped by the National Center for Research on Evaluation, Stan-dards, and Student Testing (CRESST).CRESST s criteria for establishing the technical quality of a testencompasses seven areas: cognitive complexity, content quality,meaningfulness, language appropriateness, transfer andgeneralizability, fairness, and reliability. Each aspect is discussed inthe sourcebook in a straight-forward, jargon-free One contains information concerning general test constructionand introduces the six levels of intellectual understanding: knowl-edge, comprehension, application, analysis, synthesis, and evalua-tion.

5 These levels of understanding assist in categorizing testquestions, with knowledge as the lowest level. Since teachers tendto construct questions in the knowledge category 80% to 90% of thetime, throughout the sourcebook are examples of or suggestions fordeveloping higher order thinking skills. This supports Kansas current Quality Performance Accreditation initiative which hasestablished content and performance standards that cannot bemeasured by low-level Two of the information sourcebook is devoted to actual testquestion construction. Because of the diversity of assessmentoptions, the sourcebook focuses primarily on paper-and-penciltests, the most common type of teacher-prepared assessment. Fivetest item types are discussed: multiple choice, true-false, matching,completion, and essay. Information covers the appropriate use ofeach item type, advantages and disadvantages of each item type,and characteristics of well written items.

6 Suggestions for addressinghigher order thinking skills for each item type are also sourcebook was developed to accomplish three outcomes:!Teachers will know and follow appropriate principles for devel-oping and using assessment methods in their teaching, avoidingcommon pitfalls in student tend to use tests thatthey have prepared themselvesmuch more often than any othertype of test. (How Teaching Mat-ters, NCATE, Oct. 2000)While assessment options are di-verse, most classroom educatorsrely on text and curriculum-em-bedded questions and tests thatare overwhelmingly classified aspaper-and-pencil (National Com-mission on Teaching andAmerica s Future, 1996).Formal training in paper-and-pen-cil test construction may occur atthe preservice level (52% of thetime) or as inservice preparation(21%). A significant number ofprofessional educators (48%) re-port no formal training in devel-oping, administering, scoring, andinterpreting tests (EducationWeek, National Survey of PublicSchool Teachers, 2000 ).

7 Students report a higher level oftest anxiety over teacher-madetests (64%) than over standard-ized tests (30%). The top threereasons why: poor test construc-tion, irrelevant or obscure mate-rial coverage, and unclear direc-tions. (NCATE, Summary Dataon Teacher Effectiveness, TeacherQuality, and Teacher Qualifica-tions , 2001.)(Continued on next )ii!Teachers will be able to identify and accommodate the limitationsof different informal and formal assessment methods.!Teachers will gain an awareness that certain assessment ap-proaches can be incompatible with certain instructional three outcomes directly support the standards developed bya joint commission established by the National Education Associa-tion, the American Federation of Teachers, and the National Coun-cil on Measurement in Education. The initial standards wereidentified in 1990 and revised in 1999.

8 In May 2001, a new listingwas issued under the title Standards for Teacher Competence inEducational Assessment of Students . The first two standardsdirectly reflect the outcomes of this sourcebook:!Teachers should be skilled in choosing assessment methodsappropriate for instructional discussion!Teachers should be skilled in developing assessment methodsappropriate for instructional no one document can thoroughly address the needs andconcerns expressed in all of this information, this sourcebook canbe a valuable resource for any teacher who is interested in measur-ing outcomes of significance, tapping into higher-level thinking andproblem solving skills, and constructing tests that effectively andfairly capture what a student Clay, CoordinatorKansas Curriculum CenterIn Kansas Commission onTeaching and America s Future(KCTAF), chaired by Dr.

9 AndyTompkins, Kansas Commissionerof Education, proposes to de-velop higher-quality alternativepathways to teaching as well asto reinvent teacher preparationand professional development. As secondary and postsecondaryinstitutions are exploring (out ofnecessity mostly) alternatives totraditional teacher recruitment,the need for training in assess-ment procedures and paper-and-pencil test construction in par-ticular, become more and Essay exams are easier to constructthan are objective F ?2. Essay exams require more thoroughstudent preparation and study timethan objective F ?3. Essay exams require writing skillswhere objective exams do F ?4. Essay exams teach a person howto F ?5. Essay exams are more subjectivein nature than are objective F ?6. Objective exams encourage guess-ing more so than essay F ?

10 7. Essay exams limit the extent ofcontent F ?8. Essay and objective exams canbe used to measure the samecontent or F ?9. Essay and objective exams areboth good ways to evaluate astudent s level of F ?Test Item QuizCircle the correct answerAnswers on next F=False ?=Unsure1. Objective items whichrequire students to select thecorrect response from severalalternatives or to supply aword or short phrase to answera question or complete astatement2. Subjective or essay itemswhich permit the student toorganize and present anoriginal items include:!multiple choice!true-false!matching!completionSub jective items include:!short-answer essay!extended-response essay!problem solving!performance test itemsTest yourknowledge ofthese two itemtypes byanswering thefollowingquestionsTwo generalcategories oftest items2 Quiz AnswersQuiz AnswersQuiz AnswersQuiz AnswersQuiz Answers1.