Evaluation and Correlation between Multisource Feedback ...

1 Evaluation and Correlation between Multisource Feedback and ObjectiveStructured clinical examination for Trainee Dentists in ClinicalPerformance AssessmentHiroya Gotouda1,5, Kazutaka Kasai2,5, Yasuhiro Okamoto3,5, Seiko Osawa3,5, Hiroyasu Endo3,5,Shinichiro Aoki3,5, Mitsuhiro Ohta3,5, Michiharu Shimosaka4,5, Takanori Ito3,51 Department of Preventive and Public Oral Health, Nihon University School of Dentistry at Matsudo, Matsudo, Japan, 2 Departmentof Orthodontics, Nihon University School of Dentistry at Matsudo, Matsudo, Japan, 3 Department of Oral Diagnostics, NihonUniversity School of Dentistry at Matsudo, Matsudo, Japan, 4 Department of Anesthesiology, Nihon University School of Dentistryat Matsudo, Matsudo, Japan, 5 Research Institute of Oral Science, Nihon University School of Dentistry at Matsudo, Matsudo, JapanAbstractObjective: Although Multisource Feedback (MSF) has been medically reported, there are very few studies concerning MSF indentistry.

2 In addition, compared with undergraduate dental education, no standardized objective structured clinical examinations(OSCE) criteria in dental postgraduate clinical training for trainee dentists have yet been established. The present studycomparatively examined MSF in work place-based assessment (WPBA) as formative Evaluation and OSCE as summativeevaluation to analyze and construct a clinical performance Evaluation in dental postgraduate clinical training. Materials andMethods: The supervising dentist, the dental hygienist, and the receptionist evaluated MSF as formative Evaluation and medicalinterviews for OSCE were administered to trainee dentists at the end of dental postgraduate clinical training for trainee : A positive Correlation was observed between the scores assigned by the supervising dentist and those assigned by the dentalhygienist as well as between the scores assigned by the dentist and those assigned by the receptionist within the MSF (p< ).

3 The median of the total score for MSF was significantly higher than that for medical interview (OSCE) Evaluation (p< ). Thecorrelation between the total MSF score and the total score for the medical interview as OSCE was also observed (p< ).Conclusion: The results of the present study revealed high correlations among items within the MSF Evaluation between theevaluation scores assigned by the supervising dentist and those assigned by other evaluators. Moreover, we noted a significant, butnot high, Correlation between MSF and OSCE, which suggests that these Evaluation methods assess different capabilities withrespect to skills, and attitudes. Compared with medical studies, few dental studies have comprehensively examined MSF and on these findings, further research may aid in establishing a standard clinical performance Evaluation for administration at theend of dental postgraduate clinical Words: Multisource Feedback , Objective structured clinical examination , Dental postgraduate clinical training, Traineedentists, Behavioural science in dental practice, Dental educationIntroductionThe Accreditation Council for Graduate Medical Education(ACGME) has established six core competencies (patientcare, medical knowledge, practice-based learning andimprovement, interpersonal and communication skills,professionalism, and systems-based practice) that residents areexpected to demonstrate [1].

4 Residency programs areexpected to provide a fair and effective Evaluation ofresidents performances regarding each of these Evaluation procedures are likely to be based on thesecompetencies regarding efficiency, validity, reliability, andrelevance to the resident's practice [1-3].Despite a consensus that communication skills, such asmedical interviews, are important for assessing competency ofresidents, opinions regarding the assessment of such skills arediverse [4]. The ACGME recommends five assessmentmethods, including objective structured clinical examinations(OSCEs), standardized patients, Multisource Feedback (MSF),patient surveys, and checklists [1,5]. Several articles havereported on the appropriate protocols for OSCEs andstandardized patient evaluations [6,7]. In addition, patientsurveys and MSF evaluations are standard in most can predict the performances of medical students inwork place-based assessment (WPBA) as well as in the OSCEand licensing examinations [5,8-10].

5 However, although MSF has been medically reported, thereare very few studies concerning MSF in dentistry. In addition,compared with undergraduate dental education, nostandardized OSCE criteria in dental postgraduate clinicaltraining for trainee dentists have yet been established. Thepresent study comparatively examined MSF in WPBA asformative Evaluation and OSCE as summative Evaluation toanalyze and construct a clinical performance Evaluation indental postgraduate clinical and MethodsThe current study included 102 trainee dentists belonging tothe Nihon University Hospital at Matsudo. The study wasapproved by the Ethics Committee of the same institution, andinformed consent was obtained from all participants. Forformative Evaluation as MSF in WPBA, the supervisingdentist, the dental hygienist and the receptionist assessedprofessionalism (five items) and communication skills (fiveitems), and a supervising dentist and hygienist assessedpatient care (five items) and clinical practice (five items).

6 Items used in the clinical performance Evaluation weremodified from those developed previously [1,11,12] and eachCorresponding author: Hiroya Gotouda, Department of Preventive and Public Oral Health, Nihon University School of Dentistryat Matsudo, Matsudo, Chiba 271-8587, Japan, E-mail: was rated on a scale of 1 5 (1=lowest, 5=highest). Asupervising dentist and dental hygienist each assessed fourareas (professionalism, communication, patient care, andclinical practice) wherein a maximum score of 25 points foreach area (100 total points) could be given. A receptionistassessed two areas (professionalism and communication)wherein the maximum score was also 25 points each, totalingto 50 points. The supervising dentist provided comprehensivefeedback to the participants at the end of the interviews for OSCE as summative Evaluation wereadministered to trainee dentists at the end of dentalpostgraduate clinical training.

7 Items of the Evaluation :Practice listening content (three items, a scale of 0 4),problem-solving skills (one item, a scale of 0 4),communication skills(one item, a scale of 0 5), interviewskills(two items ,a scale of 0 3), counseling skills (threeitems, a scale of 0 3), manners(one item, a scale of 0 3),attitude as medical professionals (one item, a scale of 0 3),item related to clinical record (thirteen items, a scale of 0or1), clinical diagnosis (a scale of 0or10), and Summaryevaluation(one item, a scale of 1 5). And Total points wereconverted into 100 points (MSF and OSCE) [1,5-7,11-14].Statistical analysisDescriptive statistics and statistical analyses were performedusing the IBM SPSS Statistics software, (IBM,Armonk, NY, USA). Pearson s Correlation coefficient wasused for Correlation analyses, and the Mann Whitney U-testwas used to compare the scores between the two groups.

8 Dataare presented as mean standard deviation. P< wasconsidered statistically average scores (mean SD) for MSF and OSCE and , respectively. The average scoresfor 54 males and 48 females by MSF were and , respectively, showing no significant differencesbetween the two groups. The average scores for male andfemale by OSCE and ,respectively, showing significant differences between the twogroups (Table1).The median of the total score for MSF and that for medicalinterview (OSCE) evaluations were and ,respectively. These scores were statistically a significantdifference (p< ) (Figure 1).Table 2. Correlation between the scores assigned by the supervisingdentist and those assigned by the dental hygienist as well as thatbetween the scores assigned by the supervising dentist and thoseassigned by the receptionist in Multisource Feedback (MSF).

9 ** ** ** **Dental ** **Receptionist**P< 1. The median (minimum, maximum, first quantile, andthird quantile) for evaluating total Multisource Feedback (MSF)score and total score for medical interview (OSCE) Vol. 17- 20182A positive Correlation was observed between the scoresassigned by the supervising dentist and those assigned by thehygienist (p< ) as well as between the scores assigned bythe dentist and those assigned by the receptionist (p< ). Apositive Correlation was also observed for higher scores inprofessionalism and communication compared with the othercategories (Table 2).Table 1. The average scores (mean SD) for multisourcefeedback (MSF) and objective structured clinical examinations(OSCE).Figure 2. Correlation between the total Multisource Feedback (MSF) score and the total score for medical interviews (OSCE).

10 The Correlation coefficient between the total MSF score andthe total score for the medical interviews (OSCE) was r= (p< ) (Figure 2).DiscussionMedical students recruited for residency programs ideallypossess qualities for delivering high performance duringtraining and subsequent practice [15]. High and effectivecommunication skills are directly associated with patientsatisfaction and the physician's ability to provide quality care[16,17]. The OSCE assesses both communication and clinicalskills during clinical training and has been recommended bythe ACGME as a tool for assessing communication [1]. MSFoccurs when input from peers and a colleague is utilized toassess a person s behavior in the workplace [1-3], because theinformation is aggregated and provided to the individual asfeedback. The ACGME recommends MSF as a key methodfor assessing competencies, such as professionalism andinterpersonal and communication skills [1].