Evaluating Performance in Information Technology

1 MANAGEMENT. S T R AT E G Y. MEASUREMENT. M A N AG E M E N T AC C O U N T I N G G U I D E L I N E. Evaluating Performance in Information Technology By Marc J. Epstein and Adriana Rejc Published by The Society of Management Accountants of Canada, the American Institute of Certified Public Accountants and The Chartered Institute of Management Accountants. N OT I C E TO R E A D E R S. The material contained in the Management Accounting Guideline Evaluating Performance in Information Technology is designed to provide illustrative Information with respect to the subject matter covered. It does not establish standards or preferred practices. This material has not been considered or acted upon by any senior or technical committees or the board of directors of either the AICPA, CIMA or The Society of Management Accountants of Canada and does not represent an official opinion or position of either the AICPA, CIMA or The Society of Management Accountants of Canada.

2 Copyright 2005 by The Society of Management Accountants of Canada (CMA Canada), the American Institute of Certified Public Accountants, Inc. (AICPA) and The Chartered Institute of Management Accountants (CIMA). All Rights Reserved. No part of this publication may be reproduced, stored in a retrieval system or transmitted, in any form or by any means, without the prior written consent of the publisher or a licence from The Canadian Copyright Licensing Agency (Access Copyright). For an Access Copyright Licence, visit or call toll free to 1-800-893-5777. ISBN: 1-55302-173-8. S T R AT E G Y. E VA L U AT I N G P E R F O R M A N C E. I N I N F O R M AT I O N T E C H N O L O G Y. INTRODUCTION investments. Decisions were made based on compelling arguments and keeping up While some of the recent publications on with competitors resulting in billions of Information Technology question the dollars of wasted corporate assets. value creating role of IT in today's France Telecom, for example, announced business environments (Carr, 2003 and that they spent 700 million euros on 2004), others assert that, with the external IT services in 2003 which is after economic recovery in many parts of the slashing 138 million euros from the IT.

3 World, innovation especially in spending of the prior year (838 million Information Technology is becoming euros). Almost every organization has even more critical to growth and high stories about unfulfilled promises about Performance . the benefits of new ERP or IT systems. With respect to the first, doubts about In the United States alone, annual the potential payoffs of IT investments expenditure on IT now runs into trillions can be traced to numerous IT projects of dollars, and approaches 50 percent of made without the rigor of measurement new capital investment for most of either the benefits or costs of such organizations with little evidence to CONTENTS EXECUTIVE SUMMARY. Page INTRODUCTION 3 Though there has been significant OBJECTIVES AND TARGET AUDIENCE 5 discussion concerning the importance of ENSURING ACCOUNTABILITY IN Evaluating the payoffs of IT investment, Information Technology 5. IDENTIFYING THE OBJECTIVES. there has been little guidance as to how AND DRIVERS OF to design or implement an appropriate SUCCESSFUL Information Performance evaluation system.

4 Thus, Technology INVESTMENTS 6. investments are often made without the DEVELOPING THE APPROPRIATE METRICS 15. rigor of measurement of either the FURTHER SPECIFICITY ON CALCULATING. BENEFITS (OUTPUTS) FROM IT benefits or costs of such investments. INVESTMENTS 20 Typically costs are much higher than MONITORING THE STRATEGIC anticipated and the benefits are far lower IT OBJECTIVES, DRIVERS AND. Performance MEASURES 23. and harder to achieve. MEASURING THE DISRUPTION COSTS Senior IT managers are also frustrated. OF IT INITIATIVES 23. They are convinced that, if measured MANAGING IT INVESTMENT PROCESSES 24. RECOGNIZING THE RISK ASSOCIATED. properly, IT does create value. WITH IT INVESTMENTS 25. This Guideline will develop an IT. ALIGNMENT OF THE IT CONTRIBUTION. MODEL WITH THE BALANCED Performance measurement framework, SCORECARD, SHAREHOLDER articulate specific measures, describe the ANALYSIS AND ROI 26 causal relationship between various THE APPLICABILITY OF THE IT drivers and measures, and through CONTRIBUTION MODEL AND.

5 MEASURES TO OTHER examples, illustrate how companies can BUSINESS FUNCTIONS 30 identify and measure the payoffs of IT. GUIDANCE FOR MANAGERS 31 investments. BIBLIOGRAPHY 35. 3. MANAGEMENT. S T R AT E G Y. suggest that this expenditure has generated a enthusiastic managers who were over relying MEASUREMENT satisfactory return (Murphy, 2002, Davenport on Technology and did not demand and Prusak, 1997). Typically, the costs of development of the needed skills and the Technology are much higher than anticipated, measures to complete these analyses. Today, the cost of conversion is also higher, whereas the financial managers and other decision the benefits are far lower and harder to makers want the IT requests to be framed in a achieve than expected. In addition, this ignores ROI or shareholder value format so that they the very significant costs related to employee can be effectively compared with alternative time wasted and the disruption to personnel, potential company investments.

6 According to operations, and the revenue stream of the the Forrester Report, 90% of executives make organization. While the claim that some the IT funding decisions based on the financial organizations collapsed because of ineffective impact of IT initiatives; however, the top IT policies may be an exaggeration, many Chief challenge in selecting which IT project to fund Executive Officers (CEOs) and business unit is the lack of objective data (Cameron et al., leaders view IT as a value destroyer or a cost 2000). Senior IT managers are convinced that rather than a value creator implying its they do create value and believe that, if corroding impact on the organization's measured properly and with adequate support, competitive advantage. they would be significant profit centers for their organizations. But without adequate On the other hand, the advocates of the bold Performance evaluation systems, they have and comprehensive new vision of how difficulties proving the value adding role of IT.

7 Organizations can use Information Technology and find themselves continually fighting for and to create value believe IT matters more than justifying the resources that are needed. CEOs ever, yet in a different way. By moving from an and CFOs lack Information to make well era of Technology to an era of Technology informed decisions on the payoffs of these capability, the focus has shifted from individual investments and, as a consequence, corporate technologies to the benefits that can be goals seem to focus on reduction of the costs created with them. Because of this focus on of IT rather than maximizing IT value creation Technology capabilities, innovation is emerging activities. not just from technologists, but from the users of the Technology components who understand how to use IT to deliver higher OBJECTIVES AND TARGET. levels of organizational Performance . By filling AUDIENCE. the gap between the rate of Technology As IT managers must show the payoffs of IT.

8 Innovation and people's understanding and investment to convince key executives that ability to use and implement the Technology , they should be strong supporters of IT efforts, organizations can use these innovations to lead a framework for evaluation of IT Performance to substantial improvements in their is a significant need. Few things are more Performance . Even for organizations that were convincing to top executives than measurable actually quietly making a big difference in their results. When a new project is proposed, markets by leveraging IT, it was still often additional funding is typically based solely on difficult for these Chief Information Officers the results anticipated from the project. (CIOs) to quantify those results, and prove the Currently, IT executives do not have proper benefits. Then, when earnings declined, IT was tools to measure the payoffs of IT. Even an easy target for cost cutting. financial managers that have expertise in A primary reason for doubts about the management control and Performance potential value organizations can derive from measurement, have not focused on the existing and future investments in IT is related benefits of IT and have not developed the to the absence of a proper methodology to appropriate measures.

9 Consequently, the evaluate the payoffs of IT investments. So far, payoffs of IT are not measured, ROI is not there has been little guidance of how to design calculated, and IT investments are not or implement an appropriate IT Performance evaluated with the same rigor as other evaluation system, how to identify and corporate investments. The purpose of this document the contribution of Information guideline is to provide a model and a selection Technology to high- Performance organizations. of measures for Evaluating Performance in Historically, organizations were driven by Information Technology in both for-profit and 4. E VA L U AT I N G P E R F O R M A N C E I N I T. not-for-profit organizations to help CIOs better systematic format, the guideline is also intended justify and evaluate their initiatives and aid CEOs to help CIOs, Chief Training Officers (CTOs). and CFOs in making better resource allocation and senior IT managers better understand how decisions.

10 Information Technology contributes to higher levels of corporate Performance , more easily This Management Accounting Guideline's evaluate the profitability of IT investments, and objectives are as follows: make better resource allocation decisions. In To develop a general model of key factors addition, it is also helpful for the CEOs, CFOs, for organizational success in IT integration (IT and other decision makers that struggle to Contribution Model) that includes four identify, document, measure , and communicate dimensions: the critical inputs and processes the short-term results and long-term impacts of that lead to success in IT outputs and IT investments. This includes both cost savings ultimately to overall organizational success and value creation, and thus provides arguments (outcome). for additional IT resources when appropriate. To articulate each of the key factors (antecedents and consequences of IT ENSURING ACCOUNTABILITY IN. success) as objectives to facilitate further Information Technology .