Designing Household Survey Samples: Practical Guidelines

1 Department of Economic and Social Affairs Statistics Division Studies in Methods Series F Designing Household Survey samples : Practical Guidelines Logo United Nations New York, 2005. The Department of Economic and Social Affairs o f the United Nations Secretariat is a vital interface between global policies in the economic, social and environmental spheres and national action. The department works in three main interlinked areas: (i) it compiles, generates and analyses a wide range of economic, social and environmental data and information on which States Members of the United Nations draw to review common problems and to take stock of policy options; (ii) it facilitates the negotiations of Members States in many intergovernmental bodies on joint courses of action to address ongoing or emerging global challenges; and (iii) it advises interested Governments on the ways and means of translating policy frameworks developed in United Nations conferences and summits into programmes at the country level and, through technical assistance, helps build national capacities.

2 NOTE. Symbols of the United Nations documents are composed of capital letters combined with figures. ST/ESA/ UNITED NATIONS PUBLICATION. Sales No. ISBN. Copyright @ United Nations 2005. All rights reserved Preface The main purpose of the handbook is to include in one publication the main sample Survey design issues that can conveniently be referred to by practicing national statisticians, researchers and analysts involved in sample Survey work and activities in countries. Methodologically sound techniques that are grounded in statistical theory are used in the handbook, implying the use of probability sampling at each stage of the sample selection process. A well designed Household Survey which is properly implemented can generate necessary information of sufficient quality and accuracy with speed and at a relatively low cost.

3 The contents of the handbook can also be used, in part, as a training guide for introductory courses in sample Survey design at various statistical training institutions that offer courses in applied statistics, especially Survey methodology. In addition, the handbook has been prepared to complement other publications dealing with sample Survey methodology issued by the United Nations, such as the recent publication on Household Surveys in Developing and Transitional Countries and the series under the National Household Survey Capability Programme (NHSCP). More specifically, the objectives of the Handbook are to: a. Provide, in one publication, basic concepts and methodologically sound procedures for Designing samples for, in particular, national-level Household surveys, emphasizing applied aspects of Household sample design.

4 B. Serve as a Practical guide for Survey practitioners in Designing and implementing efficient Household sample surveys;. c. Illustrate the interrelationship of sample design, data collection, estimation, processing and analysis ;. d. Highlight the importance of controlling and reducing nonsampling errors in Household sample surveys. While having a sampling background is helpful in using the handbook, other users with a general knowledge of statistical and mathematical concepts should also be able to use and apply the handbook with little or no assistance. This is because one of the key aims of the handbook is to present material in a Practical , hands-on format as opposed to stressing the theoretical aspects of sampling. Theoretical underpinnings are, however, provided when necessary. It is expected that a basic understanding of algebra is all that is needed to follow the presentation easily and to apply the techniques.

5 Accordingly, numerous examples are provided to illustrate the concepts, methods and techniques. In the preparation of this handbook the United Nations Statistics Division was assisted by Mr. Anthony Turner, Sampling Consultant, who drafted chapters 3 to 5 and finally reviewed the consolidated document. Mr. Ibrahim Yansaneh, Deputy Chief of Cost of living Division of the International Civil Service Commission, drafted chapters 6 and 7 and Mr. Maphion Jambwa, Technical Adviser, Southern African Development Community Secretariat, drafted chapter 9. The draft chapters were reviewed by an Expert Group Meeting organized, in New York, by the i United Nations Statistics Division from 3 to 5 December 2003. List of experts is given in the Appendix. ii i Chapter 1. Sources of Data for Social and Demographic Statistics.

6 1. 1. Data sources .. 1. Household surveys .. 1. Population and housing 4. Administrative 5. Complementarities of the three data sources .. 6. Concluding remarks .. 7. References and further reading .. 8. Chapter 9. Planning and Execution of Surveys .. 9. Planning of 9. Objectives of a 9. Survey 10. Information to be collected .. 10. Survey budget .. 11. 2. 2. Execution of 16. Data collection 16. Questionnaire 18. Tabulation and analysis 20. Implementation of field work .. 21. References and further reading .. 26. Chapter 27. Sampling Strategies .. 27. 27. 27. Glossary of sampling and related 28. Notations .. 31. 3. 2 Probability sampling versus other sampling methods for Household surveys .. 32. Probability sampling .. 32. Non-probability sampling 34. Sample size determination for Household 36. Magnitudes of Survey 37.

7 Target population .. 37. Precision and statistical confidence .. 38. analysis groups - 39. Clustering effects .. 41. Adjusting sample size for anticipated 42. Sample size for master samples .. 42. Estimating change or 43. Survey budget .. 43. Sample size 44. 46. iii Stratification and sample 47. Rules of stratification .. 48. Implicit 49. Cluster sampling .. 50. Characteristics of cluster sampling .. 51. Cluster design 51. Cluster 53. Calculating deff .. 54. Number of clusters .. 54. Sampling in stages .. 54. Benefits of sampling in stages .. 54. Use of dummy stages .. 55. The two-stage design .. 58. Sampling with probability proportional to size (PPS) .. 60. PPS sampling .. 60. PPES sampling (probability proportional to estimated size) .. 63. Options in sampling .. 65. Epsem, PPS, fixed-size, fixed-rate sampling.

8 65. Demographic and Health Survey (DHS) .. 69. Modified Cluster Design - Multiple Indicator Cluster Surveys (MICS) .. 70. Special topics two-phase samples and sampling for trends .. 72. Two-phase sampling .. 72. Sampling to estimate change or 74. When implementation goes wrong .. 77. Target population definition and coverage .. 77. Sample size too large for Survey budget .. 78. Cluster size larger or smaller than expected .. 78. Handling non-response cases .. 79. Summary 79. References and further reading .. 81. Chapter 83. Sampling Frames and Master samples .. 83. Sampling frames in Household 83. Definition of sample frame .. 83. Properties of sampling frames .. 84. Area frames .. 86. List frames .. 87. Multiple frames .. 88. Typical frame(s) in two-stage designs .. 89. Master sample frames .. 90. Common problems of frames and suggested 90.

9 Master sampling 94. Definition and use of a master sample .. 94. Ideal characteristics of PSUs for a master sample 94. Use of master samples to support surveys .. 95. Allocation across domains (administrative regions, etc.) .. 97. iv Maintenance and updating of master samples .. 98. Rotation of PSUs in master samples .. 98. Country examples of master samples .. 100. Summary 106. References and further reading .. 108. Chapter 109. Documentation and Evaluation of Sample 109. 109. Need for, and types of, sample documentation and evaluation .. 109. Labels for design variables .. 110. Selection 112. Response rates and coverage rates at various stages of sample 112. Weighting: base weights, non-response and other 113. Information on sampling 114. Evaluation limitations of Survey data .. 115. Summary 117. References and further reading.

10 118. Chapter 119. Construction and Use of Sample Weights .. 119. 119. Need for sampling weights .. 119. 120. Development of sampling 120. Adjustments of sample weights for unknown eligibility .. 121. Adjustments of sample weights for duplicates .. 122. Weighting for unequal probabilities of 123. Case study in construction of weights: Vietnam National Health Survey 127. Self-weighting 128. Adjustment of sample weights for 128. Reducing non-response bias in Household 129. Compensating for non-response .. 129. Non-response adjustment of sample 130. Adjustment of sample weights for non-coverage .. 132. Sources of non-coverage in Household surveys .. 133. Compensating for non-coverage in Household surveys .. 134. Increase in sampling variance due to weighting .. 135. Trimming of 136. Concluding 138. References and further reading.