EXAM QUESTIONS CLASSIFICATION BASED ON BLOOM’S …

1 Journal of Theoretical and Applied Information Technology31stAugust2015. 2005-2015 JATIT & LLS. All rights :1817-3195447 EXAM QUESTIONS CLASSIFICATION BASED ONBLOOM S TAXONOMYCOGNITIVE LEVELUSINGCLASSIFIERSCOMBINATION1 DHUHA ABDULHADIABDULJABBAR,2 NAZLIA OMAR1, 2 Knowledge Technology Group, Center for AI Technology, Faculty of Information Science andTechnology, Universiti Kebangsaan Malaysia, 43600 Bangi, Selangor, through written examination is a traditional method but it is a universal test method practiced inmost of the educational institutions today. Therefore, the question must be provided in accordance with thesubject content learned by students tofulfillearning objectives. However, the process of QUESTIONS writingis very challenging step for the lecturer. The situation is getting more challenging when lecturers try toproduce good quality and fair QUESTIONS to assess different level of cognitive.

2 Thus, the Bloom s Taxonomyhas become a common reference for the teaching and learning process used as a guide for the production ofexam QUESTIONS . Exam QUESTIONS CLASSIFICATION presents a particular challenge is the CLASSIFICATION of shorttext QUESTIONS due to short text involves text with less than 200 characters. In addition, the features of shorttext are very sparse and far. This study proposed a new method to classify exam QUESTIONS automaticallyaccording to the cognitive levels of Bloom s taxonomy by implementing a combination strategy BASED onvoting algorithm that combines three machine learning classifiers. In this work, several classifiers are takeninto consideration. The classifiers are, Support Vector Machine (SVM), Na ve Bayes (NB), andk-NearestNeighbour (k-NN) that are used to classify the question with or without feature selection methods, namelyChi-Square, Mutual Information and Odd Ratio.

3 Then a combination algorithm is used to integrate theoverall strength of the three classifiers (SVM, NB, andk-NN). The CLASSIFICATION model achieves highestresult through the combination strategy by applying Mutual Information, which proved to be promising andcomparable to other similar experiments aimed toefficientlyintegrate different featureselection methods and CLASSIFICATION algorithms to synthesize a CLASSIFICATION procedure more :Bloom s taxonomy , Exam QUESTIONS , Voting Algorithm, Machine learning, Feature objective of teaching and learning canbe achieved by implementing assessments tomeasure students' cognitive levels. Therefore thewritten exam is a medium for educationists toconfirm student knowledge and understanding, aswell as, to assess the extent to which students areable to adapt a learning theory in a given situation[1].

4 taxonomy produced by a groupofeducational psychologists [2] is a system BASED oneducational objectives that classify the level oflearning and understanding according to the levelsof the taxonomy . There are three domains inBloom s taxonomy (BT): Cognitive, Affective andPsychomotor. At this moment, cognitive domainhad received the attention of this study as it isclosely related to the students knowledge andunderstanding in the classroom. This domain issynonymous with the assessments used in anyeducational institution because it explains howcognitive response and attitude are interrelated. Itconsists of six levels to induce students to climb toa higher level, or step, of thought starting with thesimplest of knowledge, then comprehension,application, analysis, synthesis and, finally,evaluation.

5 Table 1showseach cognitionlevel aswell asillustrating verbs and simple examples thathighlightthethoughtactivitytypeof that level withexamplesof QUESTIONS thatagreewithBloom'sCognitive of Theoretical and Applied Information Technology31stAugust2015. 2005-2015 JATIT & LLS. All rights :1817-3195448 Table1:SixCategories Of The Cognitive Domain BASED OnBloom s TaxonomyWith Illustrative ExamplesCategorySample KeywordsSampleBehavioursSample QuestionsKnowledge:Arrange, recognize, relate, label,list, memorize, recall, are able to define the 6levels of Bloom s , explain, indicate, locate,recognize, describe, discuss, can explain the purposeof Bloom s taxonomyExplain the structure ofa method in the , sketch, illustrate,operate, practice, ,explain, interpret, employ,solve, use, write an instructionalobjective for each Bloom staxonomy therelationship of all thepackages, classes andmethods of the , appraise, make adistinction calculate, categorize,differentiate, discriminate,distinguish, examine, compare and contrastcognitive and affective the advantages anddisadvantages of using acontainer class such asArrayList in place of , assemble, create, design,collect, compose, develop, set up,propose, write, organize.

6 Design a classificationmethod for educational objectivesthat combine cognitive, affective,andpsychomotor a JAVA programto show the , judge, predict, assess,attach, compare, defend choose,estimate, rate, compare, judge the effectivenessof Bloom s the concept ofinheritance and give thesample of code toillustrateyour QUESTIONS CLASSIFICATION can bedefined as the selection of a category for a givenquestion from the predetermined questionscategories BASED on their contents. The questionclassification is a unique form of text significant difference is that questionsclassification presents the issue of handling shortsentences, unlike text documents, and hence, lessinformation is availableon each question thatoccurs [3] [4].

7 It would be inappropriate to use onlystatistical methods (such as TF-IDF, C-value, andN-gram) to classify the exam QUESTIONS intoBloom s taxonomy category [5]. This is due to thefact that statistical techniques require large data ineachdocument to obtain high accuracy [6]. In fact,the accuracy will decrease when small data is usedfor a training document due to the lack of terms intext. In addition, the presence of terms thatcontribute to the density of text features is sparseand far. On the other hand, the rule- BASED approachuses rules determined manually by knowledgeableengineers, with the help of domain , developing such rules is tedious and timeconsuming [7]. Rule- BASED systems lack portabilityand , the high costofrulesmaintenancegoes up,even whendata isonly addition, the use of rulesis not effective when large data are employed sincea large set of rules must be developed.

8 Over recentyears, in an effort to overcome some of thesedifficulties, supervised machine learningapproaches have been formulated for theclassification of QUESTIONS [8] [7] [9]. Machinelearning approaches have a tendency to attain aquite high degree of accuracy for the taskofquestions CLASSIFICATION in less possible overcome the problem ofexamquestionclassification with a more effectivesolution, this study proposes a combination modelwhich combines three machine learning approachesusing a combination voting algorithm adopted toclassify question items toagreewith Bloom scognitive REVIEWM anystudies havesoughttoautomaticallyclassify exam questionsbased onBloom ,although limited,researchhasaddressedtheuse ofNaturalLanguageProcessing (NLP) techniques toresolve , some previous studies thatproposed approaches similar to the modelintroduced in this work should be reviewed.

9 VanHoeij et al. (2004) developed a CLASSIFICATION -basedtool that uses Bloom s anatomy to evaluate thecognitive level of short essay of Theoretical and Applied Information Technology31stAugust2015. 2005-2015 JATIT & LLS. All rights :1817-3195449 Experimental results revealed only moderateagreement on the general CLASSIFICATION of the testitems when a simplified CLASSIFICATION tool BASED onBloom s taxonomywas used. However, given moreinstructions and an enhanced classificationalgorithm, this approach may be useful inexamination quality control. Therefore, this modelrequires more instructions and an improvedalgorithm for CLASSIFICATION to be consideredforapplication in a real-world scenario for the qualitycontrol of school and Chung (2009) also appliedBloom s taxonomy toevaluate andclassify Englishquestion item scognition level.

10 Their researchincluded 14 general keywords for Bloom staxonomy and considered 288 test items. Theydetermined that the knowledge level of cognitionhas a 75% correct match. This result is relativelynot high enough to be considered for and Omar (2012) employed a rule- BASED approach forquestion CLASSIFICATION usingBloom s taxonomy in NLP. A rule- BASED approachevaluates and classifies written examinationquestions for computer subjects. Experimentalresults confirm that the technique can support theautomaticlabellingand categorization of questionson the basis of Bloom s taxonomy . This techniqueis not as dynamic as machine learning techniques,but nonetheless exhibits satisfactory techniques are not only time consumingand expensive to implement but are alsonot asrobust as other techniques, such as machinelearning.