Performance measurementforhealth …

1 BAC KGROUN DDOCUMENTPerf orm ancemeasurementfor he al thsy stem improvemen t:exp eriences, chal leng esand pro sp ectsPet er C. Smith, Elias Mossialos andIre ne PapanicolasPerformancemeasurementforhe althsystemimprovement:exp eriences,challengesandpr ospectsPe terC. Smith,EliasMossialos andIr enePapanicolas WorldHealthOrganization2008andWorldHealt hOrganization,onbehalfoftheEuropeanObser vatoryonHealthSystemsandPolicies2008 AddressrequestsaboutpublicationsoftheWHO R egionalOfficeforEuropeto:PublicationsWHO Regional OfficeforEuropeScherfigsvej8DK-2100 Copenhagen ,DenmarkAlternatively,completeanonlinere questformfordo cumentation,healthinformation,orforpermi ssio n toqu ot e or translate,ontheRegionalOfficewebsite(h ttp://w ).Allrights theWorld HealthOrganizationwelcomesrequestsforper missiontoreproduceortranslateitspublicat ions,in partorin in thispublicationdonotimplytheexpressionof anyop inionwhatsoeveronthepartoftheWorldHealth Organizationconcerningthelegalstatusofan ycountry,territ or y, cityorar eaorofitsauthorities,orconcerningthedeli mita eremayno t pr oductsdoesnotimplythattheyareendor sedorrecommendedbytheWorldHealthOrganiza tionin pr ef er encetoothersofa ,thenamesofpr opr ietaryproducts ecautionshavebeentakenbytheWorldHealthOr ganizationtoverifytheinformationcontaine din thispu , thepublishedmaterialis beingdistributedwithoutwarrantyofanykind , e materiallieswiththereader.

2 In noeventshalltheWorldHealthOrganizationbe liable fordamagesarisingfr expressedbyauthors,editors,orexpertgroup sdono t necessarilyrepresentthedecisionsorthesta tedpolicyofth e :DELIVERYOFHEALTHCARE - standardsQUALITYIN DICATORS,HEALTHCAREQUALITYASSURANCE,HEAL THCARE- organizationandadministrationEUROPEA uthorsEl ia s Mo ssia lo s, LSEH ealthandEuropeanObservator y onHealthSystemsandPo licies, LondonSchoolofEconomicsan d PoliticalSc ience,UnitedKingd omIr en e Pap an icol as, LSEH ealth,LondonSchoolofEconomicsan dPoliticalSc ience,UnitedKingdomPe te r C. Smith, CentreforHealthEconomics,UniversityofYo rk,UnitedKingdo mCo ntentsPageKeymessagesiExecutivesummaryii Performancemeasurement forhealthsystemimpr ovement:experie nces,challen gesandprospectsPo licyissue1Pu rp oseof performancemeasurement1 Defi ni ng an d measuringperformance2 Methodolo gicalis suesaboutperformancemeasurement6 Usi ng performancemeasurement: keypolicylevers9Su mmaryandco nclusions15 References18 Perfo rmancemeasurementofferspolicy-makersamaj oropp ortunityforsecuringhealthsystemim provement an d accountability.

3 Perfo rmancemeasurementaimstoimprovethequ alityofdecisionsmadebyal l actorswithin th ehealth system. Securingimprovedperformancemeasurementof tenrequirestheactiveleadership ofgovernmen t. Major improvementsaresti ll neededin datacollection, analyticalmethodologies,policydevelopmen tandimplementationofhealthsystemperfor mancemeasurement. Defin itionsof performanceindicatorsshouldbeclear,consi s tentandfi t intoa cl earconceptualframework. policy -makersshouldpayparticular attentiontothepo liticalandorganizationalcontextwithinwhi c hperfor mancedataarecollectedanddisseminated. Cons iderableprogresshasbeenmadeindevelopingp erforman ceindicatorsforacuteho sp italcare,primary careandpo pulationhealth,bu t formentalhealth,fi nancial protectionan dhealth systemresponsivenessresearchis at a muc hearlierst ag e of development. Thedevelopmentofindividualperformanceind icators requiresconcertedexpertan d politicalat tention, an d these indicatorsshoul d: aimtoprovideinformation thatis relevanttotheneeds ofsp ecific actors; attempttomeasureperformancethatis directlyattributable toanorganizationoractor ; aimtobestatistically sound,easytointerpr etandun ambiguo us ; andbepresentedwithfullacknowledgementofa nydatali mi tations.

4 Thepresentationofperformancemeasurementd ata an d ho w th is influencesitsinterpretation bypatients, providersandpractitionersandthepublicreq uiremore attenti on. Publicreportinghasmanybenefits,butcanlea dtoadverse ou tcomes; mechanismsshould beputinplacetomonitor andcounteracttheseadverseou tcomes. Animport antuseofperfor mance measuremen t isto pr ov ide feedbackto cli nic al practiti oners on theirac ti onsandhowthesecompareto thoseof theirpeers. Perfor mance measurementsy st emssh ouldbemonito redfr equentl y to en sure ali gnment withot herhealth systemmechanis ms an d toidentifyar easfo r improvement. Ex perimentsunderwayto ex amine howperfo rman cemeasurementcanbe usedinconjun ct ion wi thexplici t fi nanci al in centiv estorewardpr ovider per fo rmancear e a pr omising areaforpolicy anda pri or it y forfur th er research.

5 A better ev id encebaseonwhichtounderpinperfo rman cemeasurementpo li cy is needed,an dnewin iti ati vesshould besu bj ecttori gorou sev aluati on .PerformancemeasurementiKeymessagesPerfo r mancemeasurementofferspolicy-makersa majoropportunitytosecurehealthsystemimpr ovementandaccount toimprovethequality ofdecisions made by allac torswithin th e healthsystem,in cludingpatients,practitioners,managers,g overn men ts at alllevels,insurersandotherpayers,politic ians, an d majorad vancesin informationtechnologyandin cr easingdemandsforhealthsystemaccountabili tyandpatientchoicehavedrivenrapidadvance sin healthsy st em ,however,arestillin therel ativelyearlystagesofperfor mancemeasurement,andmajorimprovementsar e st ill neededin datacollection,analyticalmethodologies,a n d policydevelopmentandim numberofaspects in cludingpopulationhealth,healthoutcomesfr omtreatment, clinicalqualityandtheappropri ateness ofcare, respon siveness.

6 Equityandproducti vity andpr ogress is variedin thedevelopmentofperformancemeasuresandda tacollectiontechni quesforthesedif ferent as progresshasbeenmadein such ar easasacutehospitalcare,primarycareandpop ulatio n health,butin suchareasasmentalhealth,financialprotect ion andhealthsystemresponsiveness,researchis at a st requirementofanyperformancemeasurementsy st em is tofo rmulatea performanceindicatorsshouldthenfitin to theframework andsatisfya numberofcri teria,suchasfacevalidity,reproducibility , acceptability,feasibility, reliability,sensitivity , policy -makersshouldpaycarefulat tentiontoth e politicalandorganizationalco ntextwithinwhichperformancedataaretobeco llectedan ddissemin technicalquestionsarise whenanalysingandin ter pretin g portan t are:what hascausedtheobservedperfor manceandtowhatpractitioners,organization soragenciessh ouldvariationsin performancebeat tributed?

7 In so meareas,advancedanalyticalmethodsofriska d justment havebeendevelopedtohelpanswerth e someaspectsofhealthcare,patientsafetyis a majorconcern , an d methodsofstatisticalsurveill anceha ofanomalous,th oughnotnecessarilyunsafe,performanceis theoveruseofa parti cular intervention, andtheneedtofin d ou t whetherit meansso mething unsafeforpatientswoul d thenfol lowtheini ti al find ingof anan at tentionshould bepaid tothepresentation ofperformance-measurement dat a andho w patients,providers,practi ti onersan d the pu bl ic int erpret it andar e infl uencedbyit. Fo r ex ample, a particular lycontentiousissueis theuseofcompositemeasuresofperformance,w hi chseektocombi neseveralperformanceind icators int o a si ngl e measure oforganiz at ional orsy aresu perfici al ly attracti ve,astheycanhelpsu mmarizelevel sof attai nment in anaccessibl e fashion,but theycanalsoleadto fau lty inferencesandshould be us edwithcaution.

8 Po li cy-makers canuseperfor mance measurementin anumber of waystopro mote sy stemim pr ovement. It canbe usedinpub lic rep or ting of per fo rman ce, sometimesin theform ofor ganizationalrepor t cards. Thishasbeenfound to have animportantbeneficial effect,particularlyon pr ov ider organiz ati on s. However,it hassofarhadlitt le di rect ef fect onpatients an d canalsoleadtoad verseoutcomes,such asavoidan ceof patientswithcomplexhealth pr oblems. Mechanismssh ould beputinplace tomon itorandcou nt er actsuchtenden cies .Ex periments areunderwayto ex amine ho wperformancemeasurementcanbe usedwithex plic itfin anci al inc enti vesto rewardhealthcare pr is is a pr omisi ngpo li cy ar ,su chschemesrai sea number of importantquestionsab outdesign, such aswhic h aspec ts ofperformance totarget, how tomeasure at tai nment,how to settargets,whetherto of ferin centivesat thein divid ual orgro uplevel, howst ro ng to maketheli nkbetweenac hi evementan d reward, andhowmuc h moneytoat tachto aninc entive.

9 Sofar,thereis lit tle convincingresearch ev iden ce oftheef fecti venessof such incen tives,an d th is is a pr iori ty forfu rt ets, a qu antitat iveex pr essionof anobjectiveto bemet in thefuture,ar e a particul ar formof eyhave beenparticul arlypr evalentin th ear eaofpublic health. Their ef fecti venessin securingmajor sy st emimprov ements, ho wever , hasbeenquestion ed,an d it is unl ikely th at theywillsecure suc himpro vements unl essal ignedwi thoth er po lic y levers,su chasstr ongdemocrat ic ac co untability, marketmechanismsor di rectfi nancial in cemeasurementcanalsobe usedto pr ov idefeedback to cl ini cal pr act it ioner sontheirperformancerel ati veto thei r pee rs. Th esefeedbacksy stems cansecur e wi despreadimpr ovementsin , tobesu ccessful , theyneedto beownedbyBackgrounddocumentiiEx ecutivesu mmarythe pr actitionersandusually require carefulstatisticalriskadjustmenttocontro lforconfoundingpatientcharacter is tics.

10 Also, theneedtoprovidefeedbackthatdoesno t immediatelythreatenth e reputation orlivelihoodof cliniciansandotherprofession al s canattim esconflictwiththedemandforpublic improvedperformancemeasurementis anim portantstewardshiptaskofgovernment,asman yofthe benefitsof performancemeasurementcannotbere alizedwithouttheactiveleadershipofgovern ment,whetherth roughlaw,regulation,coordinationorper suasion. Stewardshipresponsibilitiesassociatedwit hper for mancemeasurementcanbesummarizedunderthe followingheadings:1. developmentof a clearconceptualframework anda clearvis ion of th e purpose of the performancemeasu rement system: alignment withaccountabili ty relationships ; alignment withotherhealthsystemmechanisms,su chasfin ance,marketstructureandinformatio ntechnology;2. des ign of datacollectionmechanisms: detailedsp ecificationofindividualindicators; alignment withinternationalbest practice;3.