NHS England » The Good Indicators Guide

1 The Good Indicators Guide :Understanding how to use and choose indicatorsDAVID PENCHEON PageForeword21. Introduction32. Indicators : some useful background53. The anatomy of an indicator94. Understanding variation135. Changing hearts and minds186. Frequently asked questions217. Criteria for good Indicators and good indicator sets238. Ten myths about indicators289. Glossary3010. Further reading33 Appendix A: Full anatomy of an indicator351 Contents2 ForewordAs leaders, we have a responsibility toknow the essential data andinformation better than anyone need our teams and organisationsto be able to capture, interpret andcommunicate the essence of anysituation in order to make the rightdecisions at the right time. Theindicators we use and choose thereforeneed to be carefully designed to bepractical, complete, valid and robust sowe can concentrate on those areas thatneed further investigation.

2 In short, weneed to sort the wheat from the chaffin the information overload world wenow live in. This short Guide focuses onthe key principles behind developing,understanding and using Indicators . It is designed to be an essential andreadable Guide to those in seniorpositions who may not always feelentirely comfortable with this importantarea in healthcare CrumpChief ExecutiveNHS Institute for Innovation andImprovement31. What is this guidedesigned to do?This Guide is intended to be a short,practical resource for anyone in anyhealth system who is responsible forusing Indicators to monitor andimprove performance, systems oroutcomes. A Guide this size can t giveall the answers to Indicators . However,it can help you ask the if you are working with imposedmandatory Indicators , you needto be sure that your measured performance isn t falling short ofwhat is really true, either becauseyou don t understand them orhow to make the best of them if you have a choice over whichindicator to use, or anopportunity to introduce a newindicator to your organisation you need to be competent inchoosing the ones that willgenuinely help you measure localsystems, services and reading this Guide , you should beable to assess the validity of theindicators you are working with,allowing you to exert more controlover the way your organisation isproperly judged, regulated, and all this is the reality thatanyone working in a health system isworking in a complex and politicalenvironment.

3 This Guide aims tobalance what is desirable in terms ofusing Indicators in the most correctand most rigorous way, with what ispractical and achievable in Why focus on Indicators ?Why devote a whole Guide to justindicators? They are, after all, just onevery specific tool for measurement andimprovement (see definitions in theGlossary on page 30). The answer is that we all love indicatorswhen they seem to summarise and bringsummary/simplicity, but not when theyjudge us, or something dear to us. It is then that we realise that people and organisations can be unfairlyIndicators are a fact of life for every NHS local authority, and socialcare organisation. For the most part, you will have little or no choiceover the Indicators that are used to measure performance in yourorganisation. If you don t fully understand the Indicators you have towork with; if you think they paint an inaccurate picture of howthings really are on the ground; or suspect they are fundamentallyflawed in some way you can find yourself locked into an unhappyrelationship with a performance measurement tool that you canneither challenge nor and rewarded on the basis ofindicators that may tell a wrong orincomplete think of it like this: of all thecomments from new non-executivedirectors that join the boards of healthorganisations, one of the mostcommon seems to How to use the guideThis brief Guide is NOT the last word onindicators.

4 Nor is it designed to provideoff-the-shelf Indicators for anyparticular service or Good Indicators Guideshould beused alongside other related resourcesoffered by the National Health ServiceInstitute for Innovation andImprovement ( ),and the Association of Public HealthObservatories (APHO)( ), as well as realexamples from the Department ofHealth ( ), theInformation Centre ( ),the National Centre for HealthOutcomes Development( ), the HealthcareCommission( )and particular, you should read: the NHS Institute s ImprovementLeaders Guides on Measurementfor Improvement and APHO s Annual Health And if you read hope is that the Guide is shortenough and important enough to beread right through. However, there aresome key sections that will be useful bythemselves especially as quick-reference resources you can revisit.

5 These are: section 2. Indicators the importantprinciples (page 5) and section 7. Criteria for goodindicators and good indicator sets.(page 23).If you only read two sections from thisguide, read these! How can we ensure that theorganisation is actually doingwhat it says it is doing? Because we use good indicatorsto measure the right things andfeed these with the best data wecan get. The answer should be; What are Indicators , and why are they soimportant? Indicators are succinctmeasuresthat aim to describeas much abouta system as possible in as few pointsas possible. Indicators help us understandasystem, compareit and are extremely importantforms of measurement, but they canalso be controversial (see below). Like all powerful tools, they can easilydo as much harm as good. The world is becoming a moretransparent and competitive place,where people want instant summaryinformation.

6 Indicators appear to fitthis need and are therefore becomingan increasingly important part of howeverybody The three key roles ofmeasurementIndicators, like many other forms ofmeasurement, can be used in threebroad ways:1. for understanding: to know how asystem works and how it might beimproved (research role) 2. for performance: monitoring if andhow a system is performing to anagreed standard (performance/managerial/improvement role)3. for accountability: allowing us tohold ourselves up to patients, the government and taxpayers and beopenly scrutinised as individuals, teams and organisations(accountability/democratic role). Why are people suspiciousand mistrustful towardsindicators?Mistrust and ambivalence towardsindicators is hardly surprising whenconsidering what they are designed todo. While they play an increasinglyfundamental role in monitoring ourperformance, they can only everindicateand summarisethe complexsystems in which we Indicators : some useful backgroundObjectives of this section: to help you understand why Indicators have assumed such importance to explore why Indicators cause so much anxiety and resentment to discuss the different ways Indicators can be A particularhospital has a high death rateamongst its patients.

7 This isimportant and needs may be because it is a specialisthospital and admits many very illpatients from other hospitals, or itmay be because there is room forimprovement in the way thehospital treats and cares forpatients. Lesson: Indicators rarelygive definitive answers but theynearly always suggest the next bestquestion to ask that ultimatelyWILL give the answer , Indicators very oftenmake people and organisations feelvulnerable, exposed and defensive. This feeling is not likely to changeunless more is done to help peopleunderstand and accept the strengths,as well as the limitations, of theseimportant measurement Four things to know andaccept about Indicators Indicators only indicate:an indicator will never completelycapture the richness and complexity ofa system . This makes people nervousthat they will be judged unfairly onthe basis of only one (or a few) set of Indicators will usually notimprove things much.

8 Indicators aredesigned to give slices of reality. Theymight provide the truth, but theyrarely give the whole truth. This leadsto people s understandable fear thatthey are being unfairly measured andjudged. Like any reductionistapproach, an indicator must beunderstood in context Indicators encourageexplicitness: Indicators force us tobe clear and explicit about what weare trying to do. Not everyone feelscomfortable with this as there isoften a desire to retreat to nonspecific agreements rather than faceimportant differences inunderstanding. It can be difficultattaining a true agreement andunderstanding of the work, thoughindicators can be very helpful inachieving this by asking suchquestions as What would successlook like if we could only measurethree things? Indicators usually rely on numbersand numerical techniques:A lot ofpeople even competentprofessionals - fear numbers.

9 Yet, inorder to be able to use indicatorsproperly or challenge them, we needa basic understanding of elementarystatistics (rates, ratios, comparisons,standardisation etc). Although manystatistical methodologies can bedifficult and counter-intuitive to learn(especially if they are only usedoccasionally), Indicators don t alwaysuse complex methods. Indicators should not just beassociated with fault-finding:people often assume that indicatorsare designed to find fault. In fact,they can help us understand ourperformance be it good or measurementsystems identify high performers(from whom we can learn), as wellas systems (or parts of systems),that may warrant furtherinvestigation and Measuring to improveWhile Indicators do come (deservinglyor not), with a lot of baggage , usingthem properly can yield enormousbenefits particularly in terms ofimproving systems and outcomes forpatients, staff and useful analogyImagine a car dashboard:an indicatoris a warning lightflashing on the dashboard.

10 It is fedby one of many streams of data maybe oil level, temperature, flashes when all is not well,suggesting we stopthe car. Theindicator alerts us to somethingworthy of further investigation. Improving any system ( a carepathway), depends on goodmeasurement and good understanding: we measure to understand howthings work ( a care pathway) so: we can understand how we might dothings is deliberate emphasis on the wordunderstandin this Guide manyapproaches to improvement assume thatsimply by measuring something, it can beproperly improved. Using Indicators wellis an important part of understandingand therefore improving. Butmeasurement on its ownvery rarelyleads to improvement - you can t makepigs fatter just by measuring them!Measurement is necessary forimprovement but is not sufficient on it improvement depends onmore than good measurement not leastunderstanding how to turn measures intothe sorts of messages that will changehearts and minds.