Why COSO is - OpRisk Advisory

1 Operational risk is one of the mostsignificant risks that businesses facein today s complex global most of the world s leading institutions ithas become more than apparent thatimplementing an effective operational riskmanagement programme can help reducelosses, lower costs associated with fixingproblems and increase customer andemployee satisfaction, thereby improvingfinancial performance and enhancingshareholder value. Basel II may have forced banks to reviewtheir approach to managing operational risk,but for most leading institutions the questionwas never whether to establish such aprogramme, it was how. But many institutionsare still unsure of the benefits. Some are stillstruggling to decide whether to comply withthe BIS basic indicator, standardised oradvanced measurement , compliance issues aside, mostbanks have come to the conclusion that if theyare going to have to establish an operationalrisk management programme then they want itto be based on a sound framework.

2 What isperhaps surprising though is that while we aremany years into this process, there is still noindustry consensus on what shape or form thisframework ought to take. And while there hasbeen much heated debate on this issue, muchof it has been based on personal opinion andnot fact. This is because, even today, a numberof fundamental misconceptions exist about thetrue meaning of operational risk managementin its modern conception. The purpose of thispaper is to shed light on one of the main issuesthat is driving this people believe that managingoperational risk can be accomplished byfollowing the Committee for SponsoringOrganizations of the Treadway Commission( coso ) recently releasedCOSO framework sets the standards forenterprise-wide risk management (ERM). coso views ERM as a process aimed athelping organisations identify potentiallyadverse events and subsequently manage theassociated risks in furtherance of the entity sbusiness objectives.

3 When applied tooperational risk management this is oftentranslated to mean: begin with acomprehensive survey of the organisation toidentify, define and assess the full spectrum of risks in each business underlying define a series of responses or controlsto mitigate the risks that threaten to preventthe entity from meeting its objective. This isoften accomplished by establishing a list ofissues and follow-up action plans to ensurecompliance with this programme can beverified over time through the audit a macro level, this approach appears bothcomprehensive and sound, but the devil is in thedetails and the specious logic underlying coso becomes evident during implementation. WhileCOSO may help organisations identify andresolve some of their more obvious controlweaknesses, in our view, it is completelyinappropriate for use in operational riskmanagement. Fundamentally, coso isinappropriate for use in operational riskmanagement because the definition of risk usedunder this approach is wholly inconsistent withthe definition of risk used in the riskmanagement industry and by the BIS (see nextsection for a full explanation of this point).

4 Inaddition, the method coso prescribes for anorganisation to assess its risks is highly subjective,overly simplistic and conceptually flawed. coso not only fails to help a firm assess itsrisks, it actually obfuscates the risk assessmentprocess. Because risk assessment is afoundational element in the risk managementprocess, and because coso yields an entirelycounterfeit set of risks, the spurious andmisleading results of the flawed riskassessment stage contaminate everysubsequent stage of the process. As a result,the recommended risk mitigation strategy the set of controls and action plans designedto mitigate the identified risks is likely to benon-optimal at best. In the worst case, it maylead organisations to expand and intensifycontrol structures in areas where they arealready over-controlled, while completelyignoring areas of major control weakness,leaving the organisations both oblivious andvulnerable to huge operational losses thatcould hit them like a bolt from the Samad-KhanWhy coso isflawedCOSO not only fails to help a firm assess itsrisks, it actually obfuscates the riskassessment process.

5 By Ali Samad-KhanJanuary 2005 1In our view, coso as it is currentlyapplied is a wholly inappropriate approachfor managing operational risk; it is a hugewaste of resources and is very likely to domore harm than good. One obvious issue with coso is that it ishugely resource-intensive. This is becauseCOSO requires that all processes be assessed,irrespective of their individual contributions tothe organisation s total risk (because onecannot know the level of contribution to totalrisk without first conducting a risk-assessment). Identifying and documenting therisks in each and every process could takemany person-years. One mid-sized bankrecently estimated that it would require 192person-years to complete such an assessmentacross the entire organisation. Clearly, the costof such massive resource commitment is notsomething an organisation can easily absorb,particularly if the exercise needs to be repeatedon an annual basis, which is necessary becausefor operational risk management to beeffective it must be implemented through adynamic process with continuous , this is not a problem as long as the costcan be serious problem with coso has to dowith the way the risk information iscollected.

6 The starting point under coso ina typical implementation is the identification,definition and assessment of risks in abusiness process. In general, the personsinterviewed are business managers. Whilethese persons may be well qualified to runtheir own businesses, they do not necessarilyknow anything about risk. Yes, they canprobably come up with a long list of potentialrisk scenarios, but that s only half the know which risks are real risks themanager would also have to know the relativeprobability of each risk event that couldaffect his or her business. After all, a tsunamiand a wire-transfer error are both risks, butwithout knowing whether a 99% level tidalwave or 99% level fat-finger error could domore damage in the context of theirexisting control environment they cannotknow which risk poses a greater threat. Andas it turns out there is often a majordiscrepancy between perception and only way one can identify one s real risksis by studying historical loss data.

7 A riskmanager, whose job it is to know abouthistorical losses, is much more likely to beaware of the full range of potential risksaffecting a business (and their relativeprobabilities) than is a business a qualified risk manager ask a businessmanager where his or her major risks are issimilar to having a doctor ask his or herpatient: to which major diseases do you thinkyou are most exposed? Some patients will knowthe answer, but most will not, which is whythey went to see their doctor in the first a well-managed organisation, the riskprofessional should serve as the doctor and thebusiness manager as the patient. For those who still believe the rightapproach is to ask business managers to self-assess the risks within their organisation sunderlying processes, we ask what would theyhave recommended to the Governments ofIndia, Indonesia or Sri Lanka, prior to therecent tragedy, considering that tsunami riskprobably was not a recognised risk in any oftheir processes?

8 Another major problem with coso is thata typical risk-assessment implementationgenerally produces a huge catalogue of risks often in the thousands. Thus, when it comesto actually managing these risks across anorganisation, ie, determining which riskmitigation strategy is optimal, it is verydifficult to prioritise actions because withouta normalised rank ordering of risks onecannot know which controls should be givenprecedence in address this problem, coso developedthe likelihood-impact method of riskassessment. Under this approach, businessescalculate the magnitude of their risks based ona mathematical formula, where risk is equal tothe likelihood that a given event will occurmultiplied by its effect (impact), should itoccur, such that, Likelihood x Impact = who understand theconcept of risk, as it is used in therisk management industry, it is clear that thereis something fundamentally wrong with thisapproach.

9 Using the coso formula theworst-case outcome is characterised by highlikelihood and high impact; however, underthe risk management approach, the worst-caseoutcome is characterised by a low probability(low frequency) high impact (high severity)event, such as a $1 billion dollar unauthorisedtrading loss. In fact, there is no such thing asa high likelihood (high frequency) highimpact (high severity) event. This wouldcharacterise a risk (type of loss) that occurshundreds of times a year and each time causes1 coso s objective was to help standardise procedures for enterprise riskmanagement by developing a conceptually sound framework providingintegrated principles, common terminology and practicalimplementation guidance. For more information on coso , January 2005 billion-dollar losses. This is clearly a phantom risk. What s evenworse is that coso also completely understates the one areaof real risk.

10 In summary, the coso approach to riskassessment will tell you your risk is very high in areas whereyou have no risk, and will also tell you that you have moderaterisk in the very area your risk is of the highest order. Simplystated, coso produces both false positives and false contrast is illustrated in figure 1 (right).Some advocates of coso have suggested that this problemonly exists when the analysis is qualitative or high level. Theyargue that likelihood and impact analysis works well when theinputs are expressed in more quantitative terms, such aspercent probability and dollar magnitude. To examine thisargument, let us express it in the context of a simple businessproblem. Suppose you want to know the risk associated withyour having a car accident during the coming year. If youknow that you have a 10% chance of having an accident andyou expect that accident will cost $10,000, then you wouldcalculate your risk as follows:Likelihood x impact = risk Risk 1: 10% x $10,000 = $1,000 But as you further consider this matter you realise that theproblem is more complex than originally perceived.