Reliability of diagnoses coding with ICD-10

1 International journal of medical informatics 77(2008)50 57journal homepage: of diagnoses coding with ICD-10J urgen Stausberga, , Nils Lehmanna, Dirk Kaczmarekb, Markus SteincaInstitute for Medical Informatics, Biometry and Epidemiology, Medical Faculty,University of Duisburg-Essen, Hufelandstr. 55, D-45122 Essen, GermanybSankt Marien-Hospital Buer gGmbH, Gelsenkirchen, GermanycClinical Centre Ludwigshafen, Germanyarticle infoArticle history:Received 30 January 2006 Received in revised form1 September 2006 Accepted 20 November 2006 Keywords:CodingData qualityDiagnosesDiagnosis related groupsICD-10 ReliabilityabstractObjective: Reliability of diagnoses coding is essential for the use of routine data in a nationalhealth care system.

2 The present investigation compares Reliability of diagnoses coding withICD-10 between three groups of coding :One hundred and eighteen students coded 15 diagnoses lists, 27 medical managersfrom hospitals 34 discharge letters, and 13 coding specialists 12 discharge letters. Agreementin principal diagnosis was assessed using Cohen s Kappa and the fraction of coincidencesover the number of pairs, agreement for the full set of diagnoses with a previously :Kappa values were fair (managers) or moderate (coders) for terminal codes and (agreement versus ), substantial for the chapter level with (agreement versus ).pomwas lower for the full set of diagnoses thanfor principal diagnoses , for example in case of managers with versus for terminalcodes.

3 Best results were achieved by students coding diagnoses lists. In summary, the resultsare remarkably lowerthan in earlier :The refinement of the ICD-10 accompanied by innumerous coding rules hasestablished a complex environment that leads to significant uncertainties even for of coded data for quality management, health care financing, and health care policyrequires a remarkable simplification of ICD-10 to receive a valid image of health care reality. 2006 Elsevier Ireland Ltd. All rights use of classified and coded medical entities for reim-bursement, quality management, and health care policy hasincreased enormously in the last 30 years.

4 The usefulnessof these data relies basically on an identical coding of thesame entity independent of the coding person and/or the timeof coding . Thirty years ago the Institute of Medicine (IOM)analyzed the Reliability of diagnoses coding from hospitaldischarge abstracts with the 8th Revision of the Interna-tional Classification of Diseases (ICD)[1]. An independent Corresponding author. Tel.: +49 201 723 4512; fax: +49 201 723 Stausberg).re- coding of the principal diagnoses confirmed ofthe original codes. Since then, various studies have raisedissues such as whether hospitals use systematically wrongcodes to increase reimbursement[2]or whether admin-istrative data include the necessary elements for qualitymanagement[3].

5 Many studies have been published concern-ing the validity of coded data[4,5]. But it is still not clearwhether diagnoses coding with ICD is more than a matter established problems raise concerns about thepresent Reliability of diagnoses coding with ICD:1386-5056/$ see front matter 2006 Elsevier Ireland Ltd. All rights journal of medical informatics 77(2008)50 5751 The ICD includes ambiguities and inconsistencies[6]. coding of abstracts and medical reports is influencedbydifferentconclusionsaboutexi stingdiagnoses[7]. Refinement of ICD for reimbursement and a high numberof rules constitute a complex coding system, which is quitedifficult to understand, even for coding of medical entities with classifications is a hot topicin Germany.

6 The codes are used for reimbursement and sys-tem design of the German Diagnosis Related Groups (G-DRGs),introduced on a mandatory basis to hospitals in 2004. Obliga-tory public quality reports from hospitals include performancestatistics comprising codes. These reports were published firstin 2005 for 2004. A system for risk compensation is in insurance companies will establish morbidity scoresderived from coded conducted an investigation on the Reliability ofdiagnoses coding from discharge letters with the Germanmodification of the ICD-10 for health care financing ( ICD-10 -GM)[8]. The ICD-10 -GM is a successor of a pooling of an earlierGerman adaptation of WHO s 10th revision with the ICD-10 Australian Modifications ( ICD-10 -AM) Version 1.

7 Due to theadoption of the Australian Refined DRGs (AR-DRGs) in 2003compatibility with the ICD-10 -AM was required. ICD-10 -GM isrevised each year according to requirements from the coding of procedures a national classification abbreviatedas OPS is used based on WHO s International Classificationof Procedures (ICPM), also adapted to the Australian DRGs. TheICD-10-GM 2004 included 12,983 terminal aimed at calculating the Reliability of diagnoses measures the agreement of different persons cod-ing the same case (inter-rater Reliability ) or the agreement ofone person at different times coding the same case (intra-raterreliability).

8 Reliability is different from validity. Validity mea-sures the agreement with a gold standard. On the one hand itis possible to have high Reliability but weak validity, if all ratersagree in their wrong decisions. On the other hand, low relia-bility can be explained two-fold. It can be the consequence ofinsufficient education and training, and of inadequate stan-dardization of the coders and the coding scenario. But it canalso indicate weaknesses in the classification used for codingmentioned above. In the latter case, low Reliability indicatespoor quality of a coding system and should lead to a majorrevision!The investigation was split into three studies: medical stu-dents coding diagnoses lists from discharge letters, physiciansworking in medical management in hospitals coding from dis-charge letters, and specialists in medical documentation alsocoding from discharge letters.

9 Results from the first study withmedical students were published previously[9]. Objectives ofour study were to learn about the ICD-10 , to find argumentsfor the discussion who should code and to get information onthe quality of data coded in routine and methodsDischarge letters were used as basis for coding . The lettersoriginate from a department of internal medicine of a mediumTable 1 Interpretation of Kappa-valuesKappaGrade of reliabilityLandis and Koch[11]< perfectFleiss et al.[12]< to good> municipal hospital and had been written by one physi-cian in the early 1990s. They cover a full range of medicalproblems with special emphasis on nephrology.

10 Personal datahad been deleted including any datestamps concerning sel-dom events, rare diseases, or pathognomonic length of the letters ranged from 1 to 4 pages ( 1).Participants were asked to code the diagnoses independentof each other, at the time and the location of their ownchoice. No rules were established concerning assisting toolslike software or handbooks. The students were pointed to aWWW-version of the ICD-10 offered ,the other participants were reminded to follow the Ger-man coding Directives[10]. Students results were collectedon paper and subsequently entered into a database; resultsfrom the other participants were stored in simple Excel-fileswith columns for type of diagnosis and code that were dis-tributed by the study center.