Developing Data Collection Instruments - World Bank

1 103 Developing data Collection Instruments88888 Monitoring and evaluation rely on data Collection Instruments to elicitand record information. There are various types of instrumentsavailable for M&E including summary report forms used to aggregateproject data , questionnaires, more informal interview guides, inventoryforms, and observation checklists. Several formats can be combined onone form. For example, a household survey may include questions for themother about her family s health as well as a checklist for observationsabout the size, construction, and cleanliness of the development of clear, easy to read Instruments is a vital componentof both monitoring and evaluation. Knowing what to include and how toorganize the questions and answers requires skill, common sense, andpractice.

2 Considering how the collected data will be analyzed and usedbefore designing the data Collection instrument, creates a focused,efficient instrument. The following section outlines considerations tobe taken when designing data Collection Instruments . (Clear languageand an easy to follow format are essential to collect accurate, usefulinformation).Steps in Designing M&E Instruments26In designing a questionnaire, report form, or other type of instrument,M&E staff should follow the following concrete steps:1) Draft the content of the instrument based on pre-determined informa-tion ) Pay attention to ) Craft questions ) Optimize the sequence of the These guidelines have been formatted into an easy-to-use checklist that can befound in Annex ) Make the format easy to ) Check for consistency between )Pre-test the )Revise the these tasks are accomplished, the Instruments can then be printed,distributed, and put to use in the the ContentKeep it shorKeep it shorKeep it shorKeep it shorKeep it short and simplet and simplet and simplet and simplet and Information should be collected and recordedin the simplest way possible while maintaining reliability.

3 An M&E systemshould collect only what is needed. Those designing the data collectionsystem should determine in advance how each piece of information col-lected will be used. It is useful to keep in mind that the Collection , pro-cessing, and analysis of information is costly in terms of financial andhuman project evaluation teams have found it particularly useful to con-struct, at the point of survey questionnaire design, dummy tables corre-sponding to the filled in tables anticipated in the evaluation report. Thevery process of constructing these tables often clarifies the type andform of data required and can significantly reduce the proportion of infor-mation collected but unutilized often over half of the data in an evalua-tion subjects and sourIdentify subjects and sourIdentify subjects and sourIdentify subjects and sourIdentify subjects and The instrument must provide enumera-tors with an easy way to identify the child, household, clinic, or commu-nity being studied.

4 Codes are often used to identify the physical locationof the subject ( a province, state, district, county, village, cluster,house) and the particular unit of analysis ( a clinic, a household, anindividual). In addition, the persons filling out the forms must be identifiedon the filter questionsUse filter questionsUse filter questionsUse filter questionsUse filter questions to avoid asking inappropriate questions. For ex-ample, ask a woman if she has children before asking the age of heryoungest laterCalculate laterCalculate laterCalculate laterCalculate Forms to be filled out in the field should not require cal-culations. data collectors should record information as provided to manipulations ( ranks, percentages, etc.)

5 Should be done many cases, easy-to-use computer programs, such as EpiInfo27 or27. EpiInfo is a word-processing, database and statistical software package providedfree by the Centers for Disease Control and Prevention. Dean AG, Dean JA,Coulombier D, Brendel KA, Smith DC, Burton AH, Dicker RC, Sullivan K, Fagan RF,Arner TG. EpiInfo, Version 6: A Word-Processing, Database, and Statistics program forPublic Health on IBM-compatible computers. Centers for Disease Control and Preven-tion, Atlanta, GA, USA. Insight: Identification Codes that Have WorkedA household survey conducted in Nicaragua that focused onunder-twos, combined various codes to form one long identi-fication number for each child. ProvinceTownHouseholdChildSeveral monthly reports for the Bangladesh Integrated NutritionProgram have separate lines to identify the thanas, unions, andcommunity nutrition centers (CNC).

6 The numbers are then com-bined into a single code when entered into a : _____Union: _____CNC: _____106basic spreadsheet or statistical packages, can be used to make Attention to LanguageCommunicate apprCommunicate apprCommunicate apprCommunicate apprCommunicate When designing monitoring and evaluationinstruments, it is important to use the local spoken language and in wayswhich capture local concepts. Sometimes pictures or symbols that areeasily recognized by respondents are useful to convey ideas, particularlywhere level of literacy is low. When discussing past events or seeking toestimate the age of a person, it may be helpful to use local holidays,seasons ( , the harvest or the rainy season), and special occurrences( , a flood or construction of a school) to help respondents with not comprDo not comprDo not comprDo not comprDo not compromise on translationsomise on translationsomise on translationsomise on translationsomise on translations.

7 When working in foreign lan-guages, a good translation is essential. If the original instrument is to bewritten in English, it is useful to translate the English version into the locallanguage and then have someone else translate it back into English. Notonly will this prevent translation mistakes, but it will also allow investiga-Field Insight: Defining Terms to Avoid MisinterpretationSometimes what is clear to those designing Instruments may beincomprehensible or misinterpreted by others. This is particu-larly true when translating Instruments into foreign languages. Evenin the same language, however, there may be different understand-ings of concepts. In Ghana, for example, the term short-term hun-ger , which was introduced by a workshop facilitator to discuss theconsequences of skipping one or two meals, was interpreted byworkshop participants as the seasonal hunger that local people expe-rience six months of the to understand which concepts do not translate well in the local lan-guage and vice-versa.

8 In Bangladesh, for example, an ORS campaignrecommending use in all cases of diarrhea, mistakenly used the Banglaword for watery stools, understood locally as a small subset of simple languageUse simple languageUse simple languageUse simple languageUse simple language. Avoid using double negatives such as, Do youbelieve that a mother should not have to attend feeding sessions if herchild is not at risk ? Be specificBe specificBe specificBe specificBe specific. Language used, whether in interviews or on reporting forms,must be specific to prevent confusion. Instruments should avoid generalterms such as small , big , frequently , or often . The examples pro-vided in Table may be useful in judging whether the language in aninstrument is sufficiently specific or too vs.

9 Specific Language for InstrumentsGeneral LanguageDoes the school have adequate sanitation?Number of functioning latrines _____ yes_____ noNumber of people who use latrine (teachers,students, and others) _____[Note: these figures can be used to laterdecide if there are adequate sanitationfacilities given the ratio of users to latrines.]Is the house clean?Inside the house, there is evidence of:_____ yes_____ noAnimal feces _____Rotten food _____Garbage on the floor _____How often does your childHow many loose or watery stools has yourhave diarrhea? _____ child had in the past five days? _____108 Craft Questions CarefullyCarCarCarCarCarefully craft questions. efully craft questions. efully craft questions. efully craft questions.

10 Efully craft questions. Inappropriate wording of questions can influ-ence responses and may lead to information which do not accuratelyreflect the situation. Employing the following strategies when craftingquestions can prevent this double barvoid double barvoid double barvoid double barvoid double barrrrrreled questionseled questionseled questionseled questionseled questions that present two ideas simulta-neously such as, Do you think children should receive deworming medi-cine and micronutrient supplements?. Make sure that each questioncovers a single questions neutralKeep questions neutralKeep questions neutralKeep questions neutralKeep questions neutral and refrain from using words that could influ-ence answers.