The immunization data quality self-assessment …

1 WHO/ : ENGLISHDISTR.: GENERALThe immunizationdata quality self - assessment (DQS) tooliiThe Department of immunization , Vaccines and Biologicalsthanks the donors whose unspecified financial supporthas made the production of this publication publication was produced by theVaccine assessment and Monitoring teamof the Department of immunization , Vaccines and BiologicalsOrdering code: WHO/ : March 2005 This publication is available on the Internet may be requested from:World Health OrganizationDepartment of immunization , Vaccines and BiologicalsCH-1211 Geneva 27, Switzerland Fax: + 41 22 791 4227 Email: World Health Organization 2005 All rights reserved. Publications of the World Health Organization can be obtained from Marketingand Dissemination, World Health Organization, 20 Avenue Appia, 1211 Geneva 27, Switzerland(tel: +41 22 791 2476; fax: +41 22 791 4857; email: Requests for permission toreproduce or translate WHO publications whether for sale or for noncommercial distribution shouldbe addressed to Marketing and Dissemination, at the above address (fax: +41 22 791 4806.))

2 Email: designations employed and the presentation of the material in this publication do not imply theexpression of any opinion whatsoever on the part of the World Health Organization concerning the legalstatus of any country, territory, city or area or of its authorities, or concerning the delimitation of itsfrontiers or boundaries. Dotted lines on maps represent approximate border lines for which there maynot yet be full mention of specific companies or of certain manufacturers products does not imply that they areendorsed or recommended by the World Health Organization in preference to others of a similar naturethat are not mentioned. Errors and omissions excepted, the names of proprietary products are distin-guished by initial capital reasonable precautions have been taken by WHO to verify the information contained in this publica-tion.

3 However, the published material is being distributed without warranty of any kind,either express or implied. The responsibility for the interpretation and use of the material lies with thereader. In no event shall the World Health Organization be liable for damages arising from its by the WHO Document Production Services, Geneva, viiExecutive ixA. 1B. immunization data quality self - assessment options: overview .. of reporting .. 1 the quality of the monitoring system .. 2 the quality of immunization card recording(health unit level) .. 2 of wastage .. 2 of immunization safety .. 2 of immunization coverage .. 2 6C. Where to conduct a DQS? .. 2 7D. Present the DQS findings .. 3 the DQS results .. 3 Excel to enter and represent the data .

4 3 3E. Conduct a DQS workshop .. 3 4 Some proposed workshop principles .. 3 5F. Integrate DQS results into the routine activities .. 3 7 Annex A: Sample chart for monitoring doses administeredand drop-outs in children less than one year of age .. 3 9 Annex B: Example of a completeness/timeliness reporting table .. 4 1 Annex C: Standard questions to assess the quality of themonitoring system .. 4 3 Annex D: Child immunization card exercise (example for 20 infants) .. 5 7 Annex E: Sampling of health units .. 5 9 Annex F: data quality self - assessment workshop 6 2ivvSpecial thanks is given to Abdallah Bchir, Craig Burgess, Jan Grevendonk,Fran ois-Xavier Hanon, Stephen Hadler and Ezzedine Mohsni for assisting withthe technical content of the data quality self - assessment (DQS) has been developed subsequently to theimmunization data quality audit procedure (WHO/V& ), which was designedfor use for the Global Alliance for Vaccines and immunization (GAVI).

5 The DQShas been tested in a number of countries (Nepal, Morocco and Togo) in which localsupport and feedback was extremely useful and appreciated. Respective WHOregional and country offices and ministries of health ( immunization divisions) ofthese countries are deeply acknowledged. Financial support from GAVI hascontributed to the design and testing of the (syringe)AEFI adverse events following immunizationBCGbacille Calmette-Gu rin (existing TB vaccine)DTPdiphtheria tetanus pertussis vaccineDQSdata quality self -assessmentHUhealth unitMOHministry of healthNGOnongovernmental organizationNIDnational immunization dayOPVoral polio vaccineQIquality indexQQquestions on qualityREDR eaching Every DistrictSEstandard errorTTtetanus toxoidUNICEFU nited Nations Children s FundVVMvaccine vial monitorVPDvaccine-preventable diseaseviiiixWhat is the DQS?

6 The DQS is a flexible toolbox of methods to evaluate differentaspects of the immunization monitoring system at district and health unit (HU) monitoring refers to the regular ongoing measurement of the levelof achievement in vaccination coverage and other immunization system indicators( safety, vaccine management). Monitoring is closely linked with reporting becauseit involves data collection and audience. This document is to be used primarily by staff who will adapt thetoolbox for a specific area (usually staff at national and regional levels). The adaptedtool should then be used by staff collecting and using immunization data at the national,provincial or district of the DQS. The DQS aims to assist countries in diagnosing problems and toprovide orientation to improve district monitoring, as highlighted in the ReachingEvery District (RED) DQS aims to determine: the accuracy of reported numbers of immunizations, and the quality of the immunization monitoring assessment includes a review of data accuracy at different levels and aself-designed questionnaire reviewing monitoring quality issues ( availability ofvaccination cards, use of tally sheets, directly-observed recording and reportingpractices).

7 These are then analysed, strengths and weaknesses identified,conclusions reached and practical recommendations made. These recommendationsaim to improve the use of accurate, timely and complete data for action at all is a DQS performed? One approach is to hold an initial national participatoryDQS workshop involving key people from the national and district levels to reviewcountry monitoring practices and design a self - assessment . This workshop isimmediately followed by a practical assessment in a number of districts andhealth units to provide a self -diagnosis of the monitoring system of the approaches can be developed and self -assessments can be designed andconducted without this preliminary summaryxThe final goal of the DQS is to integrate into routine practice the tool options thatare most relevant for a country so that constant attention is given to improvemonitoring practices and management of immunization to use this document?

8 A number of options for evaluating monitoring processesare presented in this document. They should be explored, selected and refinedaccording to specific needs. The DQS does not aim to be standardized acrosscountries. The same flexibility is required when selecting where to conduct the DQSin a data quality self - assessment (DQS) consists of a flexible toolbox, designed forstaff at the national, provincial or district levels to evaluate different aspects of theimmunization monitoring system at district and health unit (HU) level in order todetermine the accuracy of reported numbers of immunizations and the quality ofthe immunization monitoring this manual, monitoring refers to the measurement of the level of achievement invaccination coverage and other system indicators ( safety, vaccine management,etc).

9 Monitoring is linked closely with reporting because it involves data collectionand options described in the toolbox (Section B) should be explored, selected andrefined according to specific needs. The tool does not aim to be standardized acrosscountries. The same flexibility should be applied for the selection of DQS sites,which is discussed in Section DQS aims to diagnose problems and provide orientation to improve districtmonitoring and use of data for action, as highlighted in the Reaching Every District(RED) Basic knowledge of Excel is helpful when entering and analysingcollected data but the self - assessment can be conducted without computerized date, two Excel workbooks are available for different components of the toolbox(Section D).

10 The approach described here to introduce the DQS concept in one country is througha national participatory workshop (see Section E) involving key people from thenational and district levels. This workshop is immediately followed by an assessmentin a number of districts and HUs that provides a self -diagnosis on the monitoringsystem of the country. Other approaches can be developed and self -assessments canbe conducted la final goal of this assessment tool is to integrate the options that are most relevant for onecountry into routine practice (Section F) so that constant attention can be given to improvemonitoring practices and management of immunization Introduction1 Increasing immunization coverage at the health facility level. Geneva, WHO, 2002 (WHO/V& ).