Objective Structured Clinical Examination (OSCE) Revisited

1 INDIAN PEDIATRICS911 VOLUME 47__NOVEMBER 17, 2010 That learning is driven by assessment isa well known fact. This is also referred toas the steering effect of examinations .To foster actual learning, assessmentshould be educative and formative. Medicaleducation aims at the production of competentdoctors with sound Clinical skills. Competencyencompasses six inter-related domains as developedby Accreditation Council for Graduate MedicalEducation (ACGME): knowledge, patient care,professionalism, communication and interpersonalskills, practice based learning and improvement, andsystems based practice(1). Epstein and Hundert havedefined competence of a physician as the habitualand judicious use of communication, knowledge,technical skills, Clinical reasoning, emotions, valuesand reflection in daily practice for the benefit of theindividuals and the community being served (2).

2 The community needs to be protected fromincompetent physicians; and thus there is a need forsummative component in the assessment of BEYOND THE TRADITIONAL TOOLSThe traditional tools for assessment of medicalstudents have mainly consisted of written exams(essay type, multiple choice, and short-answer typequestions), bedside viva and Clinical casepresentation. These have focussed on the knows and knows how aspects, , the focus has been onthe base of the Miller s pyramid of competence ( ). These methods of assessment however havedrawn a lot of criticism over the years because oftheir inability to evaluate the top levels of thepyramid of competency in a valid and reliablemanner. The following flaws were realised: Objective Structured Clinical Examination (OSCE) RevisitedPIYUSH GUPTA, POOJA DEWAN AND TEJINDER SINGH*From the Department of Pediatrics, University College of Medical Sciences, Delhi, and *Department of Pediatrics,Christian Medical College, Ludhiana, to: Dr Piyush Gupta, Block R 6 A, Dilshad Garden, Delhi 110 095.

3 Structured Clinical Examination (OSCE) was introduced in 1975 as a standardized tool for objectively assessingclinical competencies - including history-taking, physical Examination , communication skills, data interpretation etc. Itconsists of a circuit of stations connected in series, with each station devoted to assessment of a particular competencyusing pre-determined guidelines or checklists. OSCE has been used as a tool for both formative and summativeevaluation of medical graduate and postgraduate students across the globe. The use of OSCE for formative assessmenthas great potential as the learners can gain insights into the elements making up Clinical competencies as well asfeedback on personal strengths and weaknesses. However, the success of OSCE is dependent on adequacy ofresources, including the number of stations, construction of stations, method of scoring (checklists and/or global scoring),the number of students assessed, and adequate time and money.

4 Lately, OSCE has drawn some criticism for its lack ofvalidity, feasibility, practicality, and objectivity. There is evidence to show that many OSCEs may be too short to achievereliable results. There are also currently no clear cut standards set for passing an OSCE. It is perceived that OSCEs testthe student s knowledge and skills in a compartmentalized fashion, rather than looking at the patient as a whole. Thisarticle focuses on the issues of validity, objectivity, reliability, and standard setting of OSCE. Presently, the Indianexperiences with OSCE are limited and there is a need to sensitise the Indian faculty and students. A cautious approach isdesired before it is considered as a supplementary tool to other methods of assessment for the summative examinationsin Indian words: Assessment, Clinical , Competency, India, OSCE, Reliability, E D I C A L EM E D I C A L EM E D I C A L EM E D I C A L EM E D I C A L EDDDDDUUUUUCCCCCAAAAATTTTTIIIIIOOOOONNNN NINDIAN PEDIATRICS912 VOLUME 47__NOVEMBER 17, 2010 GUPTA, et Revisited They test only the factual knowledge and prob-lem-solving skills of students, which may be ap-propriate only in the early stages of medical cur-riculum.

5 These methods do not evaluate the clini-cal competence of students. Important aspects likeperforming a particular physical Examination (shows how), Clinical maneuver, and communica-tion-skills are not tested. Only the end result istested and not the process of arriving at a result. The students are tested on different patients(patient variability). Each student is adjudged byonly one or two examiners, thereby a scope formarked variation in the marking by differentexaminers (examiner variability). These factorsincrease the subjectivity of marking (lack ofreliability). There is often a lack of clarity on what is actuallybeing tested (lack of validity). Assessment isusually global and not competency based. Students are not examined systematically on coreprocedures.

6 There is no systematic feedback from thestudents and obviate the drawbacks of conventionalclinical evaluation, Objective Structured clinicalexamination (OSCE) was first introduced by Hardenin 1975, as a more Objective , valid, and reliable toolof assessment(3). In an ideal OSCE, all domains ofcompetencies are tested, specially the process part;the Examination is organized to examine all studentson identical content by the same examiners usingpredetermined guidelines; and a systematic feedbackis obtained from both students and the is meant to test the shows how level of theMiller s pyramid(4).CONTENT AND PROCESS OF OSCEOSCE consists of a circuit of stations which areusually connected in series (Fig. 2). Each station isdevoted to evaluation of one particular student is asked to perform a particular task ateach station.

7 These stations assess practical,communication, technical, and data interpretationskills and there is a predetermined decision on thecompetencies to be tested. Students rotate around thecomplete circuit of stations, and keep on performingthe tasks at each of the stations. All students moveWHAT IS AN OSCE? Objective Structured Clinical Examination1. Ensures evaluation of set of predetermined Clinical Each Clinical competency is broken down into smaller components; , taking history, performing Examination ,interpreting investigations, communicating, Each component is assessed in turn and marks are allotted according to predetermined Miller s 121110987 consists of a circuit of stations which areusually connected in howShow howDoesCognition-knowledge-Behavior - skills/attitudesProfessional authenticityINDIAN PEDIATRICS913 VOLUME 47__NOVEMBER 17, 2010 GUPTA, et Revisited from one station to another in the same performance of a student is evaluatedindependently on each station, using a standardizedchecklist.

8 Thus, all students are presented with thesame test; and are assessed by the same or equivalentexaminers. Students are marked objectively on thechecklist(5) by the of OSCE stations: The stations arecategorized as procedure station or questionstation . Procedure stations are observed by theexaminer while question stations are unobserved(only a written answer is desired). Studentperformance on a Procedure station is observed andmarked there and then only while the Questionstations can be evaluated later. The details of thesestations along with specific examples have beendescribed previouly(5). Procedure station and aquestion station can also be used together. In theoriginal description of OSCE by Harden, everyProcedure station was followed by a Questionstation.

9 Students are given a task to perform inStation 1 (which is observed and assesses the processof performing the task) and the questions arepresented later (in Station 2). Questions in station 2are related to station 1 only. This has two advantages:(a) different domains of learning can be assessed bythem; and (b) the effect of cueing is minimized. It isalso advisable to incorporate a rest station for every30-40 minutes into the exam, to give a break to thestudents, the observers and the patients. They alsoallow time to substitute patients at a Clinical station,or to complete the written left over task from theprevious setup: The number of stations can vary from12 to 30 though usually 20 stations suffice(1). Theusual time allotted is 5 minutes for each station;ACGME however recommends station duration of10-15 minutes.

10 Giving more time per station allowsmore competencies to be tested in relation to thegiven task. All students begin simultaneously. Thenumber of students appearing in the exam should notexceed the number of stations. In case, the number ofstudents is more, one or more parallel sessions can beorganized, subject to availability of space, examinersand patients. If facilities do not permit this, then twosessions can be planned. All students shouldcommence the Examination from a procedurestation. The entire exam is usually completed within60-150 minutes. Details of microplanning of OSCE have been described earlier(6).Blueprinting: Preparing the StationsOnce the consensus is reached on the number andtype of stations to be included, the next task is toformulate the questions, model keys, and checklistsfor each station.