Key Performance Indicators to Benchmark Hospital ...

1 508 Schattauer 2009 Original Articles Key Performance Indicators to Benchmark Hospital Information Systems A Delphi Study G. H bner-Bloder; E. Ammenwerth Institute for Health Information Systems, UMIT University for Health Sciences, Medical Informatics and Technology, Hall in Tyrol, Austria ponents in their information-processing Keywords and user satisfaction. Isolated technical indi- roles [3]. The central task of HISs is to sup- Benchmarking, Hospital information systems, cators or cost Indicators were not seen as use- port a high-quality and cost-effective patient medical informatics, Delphi technique, quality ful. The experts favored an interdisciplinary care [4].

2 Indicators , key Performance Indicators , user group of all the stakeholders, led by Hospital As a result of the increasing importance of satisfaction management, to conduct the HIS Benchmark - efficient information logistics, a systematic ing. They proposed benchmarking activities Summary information management, comprising plan- both in regular (annual) intervals as well as at Objectives: To identify the key Performance ning, directing and monitoring, becomes a defined events (for example after IT introduc- Indicators for Hospital information systems central mission for hospitals [3]. While the tion). Most of the experts stated that in their (HIS) that can be used for HIS benchmarking.

3 Planning and directing of information sys- institutions no HIS benchmarking activities Methods: A Delphi survey with one quali- tems seem to be supported by several stan- are being performed at the moment. tative and two quantitative rounds. Forty-four dards and guidelines (for example [3, 5 7]), Conclusion: In the context of IT governance, HIS experts from health care IT practice and the monitoring of information systems is IT benchmarking is gaining importance in the academia participated in all three rounds. often seen as more complex and not well sup- healthcare area. The found Indicators reflect Results: Seventy-seven Performance indi- ported [8, 9].

4 Monitoring comprises, among the view of health care IT professionals and cators were identified and organized into others, the definition and application of those researchers. Research is needed to further eight categories: technical quality, software criteria that should reflect the quality and validate and operationalize key Performance quality, architecture and interface quality, IT efficiency of information logistics. In this Indicators , to provide an IT benchmarking vendor quality, IT support and IT department sense, monitoring stands in close relation framework, and to provide open repositories quality, workflow support quality, IT outcome with a systematic benchmarking of HISs.

5 For a comparison of the HIS benchmarks of quality, and IT costs. The highest ranked indi- According to the Joint Commission [10], different hospitals. cators are related to clinical workflow support benchmarking can be defined as the con- tinuous measurement of a process, product, or service compared to those of the toughest Correspondence to: Methods Inf Med 2009; 48: 508 518 competitor, to those considered industry Dr. Elske Ammenwerth doi: leaders, or to similar activities in the organi- Institute for Health Information Systems received: May 20, 2009 zation in order to find and implement ways to Eduard Walln fer-Zentrum I accepted: July 29, 2009.

6 Improve it . The Joint Commission differenti- 6060 Hall in Tirol prepublished: November 5, 2009. Austria ates between internal benchmarking (similar E-mail: processes within the same organization are compared) and competitive benchmarking (organization's processes are compared with best practices within the industry). 1. Introduction mation system (HIS). A HIS should support Benchmarking in general comprises the the information logistics within a Hospital , choice of an appropriate target, the definition Technological and medical evolution and or- making the appropriate information the of the related Performance Indicators , and the ganizational changes affect health care.

7 As a appropriate knowledge at the appropriate collection of the relevant data that can then consequence, health care has become increas- time at the appropriate location the ap- be used for comparison purposes [11]. ingly more complex, and is today charac- propriate individuals in an appropriate and There exist several approaches to develop terized by more to know, more to do, more to usable form available [2]. In this sense, a HIS Performance Indicators for hospitals. For manage, more to watch, and more people can be defined as the socio-technical subsys- example, the Joint Commission's Annual involved than ever before [1].

8 Tem of an enterprise, which consists of the Report on Quality and Safety 2008 provides To manage these increasing requirements, information-processing activities and re- evidence of hospitals' Performance re- hospitals require an efficient Hospital infor- sponsible human and mechanical com- garding National Patient Safety Goals [12]. Methods Inf Med 6/2009. G. H bner-Bloder; E. Ammenwerth: Key Performance Indicators to Benchmark Hospital Information Systems 509. This report describes how well hospitals fol- in most hospitals, no regular HIS monitoring chose Type 3 ( Fig. 1), comprising one low the pre-defined treatment guidelines.

9 Activities based on the objective and quanti- qualitative and two quantitative rounds: For benchmarking the information sys- fied assessment of HIS quality are conducted. 1. First qualitative round: A written survey tems of a Hospital , fewer approaches seem to Overall, it seems necessary to first define what with five open-ended questions was used exist. The information management stan- useful HIS Performance Indicators are, before to collect ideas for the useful Performance dards of JCAHO [13] define ten major stan- adequate validated methods to measure them Indicators for HIS benchmarking. dards with several sub-standards, in turn fo- can be developed.

10 2. Second quantitative round: Based on the cusing on the general issues of information The objective of the present paper is to results of the first round, a standardized management within a Hospital (including, for develop a prioritized list of the useful per- questionnaire with close questions was example, usage of standardized codes and formance Indicators for HISs, based on a developed, in turn requesting the experts classifications, availability of a complete Delphi-based survey of HIS experts. The to rate the importance of each proposed medical record, or access to knowledge re- study questions were: indicator on a 4-point Likert scale (21).)