Evaluating Performance in Information …

1 EvaluatingPerformance inInformationTechnologyByMarc J. EpsteinandAdriana RejcMANAGEMENT ACCOUNTING GUIDELINEP ublished by The Society of Management Accountants of Canada, the AmericanInstitute of Certified Public Accountants and The Chartered Institute ofManagement 2005 by The Society of Management Accountants of Canada (CMA Canada), the American Institute of CertifiedPublic Accountants, Inc. (AICPA) and The Chartered Institute of Management Accountants (CIMA). All Rights part of this publication may be reproduced, stored in a retrieval system or transmitted, in any form or by any means,without the prior written consent of the publisher or a licence from The Canadian Copyright Licensing Agency (AccessCopyright). For an Access Copyright Licence, visit or call toll free to : 1-55302-173-8 NOTICE TO READERSThe material contained in the Management Accounting Guideline Evaluating Performance in Information technology is designed to provide illustrative Information with respect to the subject matter covered.

2 It does not establish standards or preferredpractices. This material has not been considered or acted upon by any senior or technical committees or the board of directorsof either the AICPA, CIMA or The Society of Management Accountants of Canada and does not represent an official opinionor position of either the AICPA, CIMA or The Society of Management Accountants of some of the recent publications oninformation technology question thevalue creating role of IT in today sbusiness environments (Carr, 2003 and2004), others assert that, with theeconomic recovery in many parts of theworld, innovation especially ininformation technology is becomingeven more critical to growth and highperformance. With respect to the first, doubts aboutthe potential payoffs of IT investmentscan be traced to numerous IT projectsmade without the rigor of measurementof either the benefits or costs of suchinvestments. Decisions were made basedon compelling arguments and keeping upwith competitors resulting in billions ofdollars of wasted corporate Telecom, for example, announcedthat they spent 700 million euros onexternal IT services in 2003 which is afterslashing 138 million euros from the ITspending of the prior year (838 millioneuros).

3 Almost every organization hasstories about unfulfilled promises aboutthe benefits of new ERP or IT systems. In the United States alone, annualexpenditure on IT now runs into trillionsof dollars, and approaches 50 percent ofnew capital investment for mostorganizations with little evidence toEVALUATING Performance IN Information TECHNOLOGYCONTENTS EXECUTIVE SUMMARYT hough there has been significantdiscussion concerning the importance ofevaluating the payoffs of IT investment,there has been little guidance as to howto design or implement an appropriateperformance evaluation system. Thus,investments are often made without therigor of measurement of either thebenefits or costs of such costs are much higher thananticipated and the benefits are far lowerand harder to achieve. Senior IT managers are also are convinced that, if measuredproperly, IT does create Guideline will develop an ITperformance measurement framework,articulate specific measures, describe thecausal relationship between variousdrivers and measures, and throughexamples, illustrate how companies canidentify and measure the payoffs of ITinvestments.

4 INTRODUCTION 3 OBJECTIVES AND TARGET AUDIENCE 5 ENSURING ACCOUNTABILITY IN Information technology 5 IDENTIFYING THE OBJECTIVES AND DRIVERS OF SUCCESSFUL Information technology INVESTMENTS 6 DEVELOPING THE APPROPRIATE METRICS 15 FURTHER SPECIFICITY ON CALCULATING BENEFITS (OUTPUTS) FROM IT INVESTMENTS 20 MONITORING THE STRATEGIC IT OBJECTIVES, DRIVERS AND Performance MEASURES 23 MEASURING THE DISRUPTION COSTS OF IT INITIATIVES 23 MANAGING IT INVESTMENT PROCESSES 24 RECOGNIZING THE RISK ASSOCIATED WITH IT INVESTMENTS 25 ALIGNMENT OF THE IT CONTRIBUTION MODEL WITH THE BALANCED SCORECARD, SHAREHOLDER ANALYSIS AND ROI 26 THE APPLICABILITY OF THE IT CONTRIBUTION MODEL AND MEASURES TO OTHER BUSINESS FUNCTIONS 30 GUIDANCE FOR MANAGERS 31 BIBLIOGRAPHY35 PageSTRATEGYMANAGEMENTSTRATEGYMEASUREMEN T4suggest that this expenditure has generated asatisfactory return (Murphy, 2002, Davenportand Prusak, 1997). Typically, the costs oftechnology are much higher than anticipated,the cost of conversion is also higher, whereasthe benefits are far lower and harder toachieve than expected.

5 In addition, this ignoresthe very significant costs related to employeetime wasted and the disruption to personnel,operations, and the revenue stream of theorganization. While the claim that someorganizations collapsed because of ineffectiveIT policies may be an exaggeration, many ChiefExecutive Officers (CEOs) and business unitleaders view IT as a value destroyer or a costrather than a value creator implying itscorroding impact on the organization scompetitive the other hand, the advocates of the boldand comprehensive new vision of howorganizations can use Information technologyto create value believe IT matters more thanever, yet in a different way. By moving from anera of technology to an era of technologycapability, the focus has shifted from individualtechnologies to the benefits that can becreated with them. Because of this focus ontechnology capabilities, innovation is emergingnot just from technologists, but from the usersof the technology components whounderstand how to use IT to deliver higherlevels of organizational Performance .

6 By fillingthe gap between the rate of technologyinnovation and people s understanding andability to use and implement the technology ,organizations can use these innovations to leadto substantial improvements in theirperformance. Even for organizations that wereactually quietly making a big difference in theirmarkets by leveraging IT, it was still oftendifficult for these Chief Information Officers(CIOs) to quantify those results, and prove thebenefits. Then, when earnings declined, IT wasan easy target for cost primary reason for doubts about thepotential value organizations can derive fromexisting and future investments in IT is relatedto the absence of a proper methodology toevaluate the payoffs of IT investments. So far,there has been little guidance of how to designor implement an appropriate IT performanceevaluation system, how to identify anddocument the contribution of informationtechnology to high- Performance , organizations were driven byenthusiastic managers who were over relyingon technology and did not demanddevelopment of the needed skills and themeasures to complete these analyses.

7 Today,the financial managers and other decisionmakers want the IT requests to be framed in aROI or shareholder value format so that theycan be effectively compared with alternativepotential company investments. According tothe Forrester Report, 90% of executives makethe IT funding decisions based on the financialimpact of IT initiatives; however, the topchallenge in selecting which IT project to fundis the lack of objective data (Cameron et al.,2000). Senior IT managers are convinced thatthey do create value and believe that, ifmeasured properly and with adequate support,they would be significant profit centers fortheir organizations. But without adequateperformance evaluation systems, they havedifficulties proving the value adding role of ITand find themselves continually fighting for andjustifying the resources that are needed. CEOsand CFOs lack Information to make wellinformed decisions on the payoffs of theseinvestments and, as a consequence, corporategoals seem to focus on reduction of the costsof IT rather than maximizing IT value creationactivities.

8 OBJECTIVES AND TARGET AUDIENCE As IT managers must show the payoffs of ITinvestment to convince key executives thatthey should be strong supporters of IT efforts,a framework for evaluation of IT performanceis a significant need. Few things are moreconvincing to top executives than measurableresults. When a new project is proposed,additional funding is typically based solely onthe results anticipated from the , IT executives do not have propertools to measure the payoffs of IT. Evenfinancial managers that have expertise inmanagement control and performancemeasurement, have not focused on thebenefits of IT and have not developed theappropriate measures. Consequently, thepayoffs of IT are not measured, ROI is notcalculated, and IT investments are notevaluated with the same rigor as othercorporate investments. The purpose of thisguideline is to provide a model and a selectionof measures for Evaluating Performance ininformation technology in both for-profit and5 Evaluating Performance INITnot-for-profit organizations to help CIOs betterjustify and evaluate their initiatives and aid CEOsand CFOs in making better resource allocationdecisions.

9 This Management Accounting Guideline sobjectivesare as follows: To develop a general model of key factorsfor organizational success in IT integration (ITContribution Model) that includes fourdimensions: the critical inputs and processesthat lead to success in IT outputs andultimately to overall organizational success(outcome). To articulate each of the key factors(antecedents and consequences of ITsuccess) as objectives to facilitate furtheroperationalization of the model. To outline the specific drivers of IT successbased on the objectives related to inputs,processes, outputs, and outcomes, andidentify the causal relationships between thedrivers. To provide the specific measures of ITperformance. Following the cause-and-effectrelationships between the drivers of ITsuccess, measures are developed to trackperformance of IT initiatives along the fourdimensions. The metrics can be used forboth IT project justification prior to its start(planning) as well as for evaluation aftercompletion ( Performance measurement).

10 To provide examples of how to assignmonetary values to non-financial IT outputs(benefits). Although some benefits do notalways easily translate into short term profits,they should ultimately lead to either costsavings or increased revenues. To provide an example of how to calculatethe IT payoffs. Here, the guideline specificallyrecognizes the importance of measuring boththe total costs of an IT initiative includinga range of different disruption costs aswell as the benefits, and additionallyconsiders the risks associated with ITinvestments. To show how the IT Contribution Model isconsistent with other measurementframeworks such as the Balanced Scorecardand Shareholder Value target audienceof this guideline is theaccounting and finance professionals that dealwith the challenges of performancemeasurement and control in IT. Presented in asystematic format, the guideline is also intendedto help CIOs, Chief Training Officers (CTOs)and senior IT managers better understand howinformation technology contributes to higherlevels of corporate Performance , more easilyevaluate the profitability of IT investments, andmake better resource allocation decisions.