EVALUATION OF AGRICULTURAL EXTENSION AND …

1 EVALUATION OF AGRICULTURAL . EXTENSION AND ADVISORY SERVICES. A MEAS Training Module By Murari Suvedi September 2011. EVALUATION OF AGRICULTURAL . EXTENSION AND ADVISORY SERVICES. A MEAS Training Manual Professor Murari Suvedi Michigan State University Murari Suvedi, Michigan State University and the MEAS Project This work is licensed under a Creative Commons Attribution Non Commercial Share Alike License. This license lets others remix, tweak, and build upon this work non-commercially, as long as they credit Murari Suvedi and license their new creations under the identical terms.

2 This training module was produced as part of the United States Agency for International Development, USAID, project Modernizing EXTENSION and Advisory Services , MEAS. Leader with Associates Cooperative Agreement No. AID-OAA-L-10-00003. The module was made possible by the generous support of the American people through USAID. The contents are the responsibility of the authors and do not necessarily reflect the views of USAID or the United States Government. ACKNOWLEDGEMENTS. EVALUATION practitioners and international AGRICULTURAL development experts have contributed to the development of this training manual by offering guidance and/or reviewing the manuscript.

3 Mary Andrews, Michigan State University Kirk Heinze, Michigan State University Diane Ruonavaara, Michigan State University Gail Vander Stoep, Michigan State University R. Kirby Barrick, University of Florida Shawn Morford, Program Manager, Rural Development Initiatives, Inc. Angela Hobson, Graduate Assistant in the Department of Community, Agriculture, Recreation and Resources Studies at Michigan State University as well as Andrea Bohn, MEAS Project Manager at the University of Illinois at Urbana-Champaign helped in editing and formatting the manual.

4 Resources from diverse research and EVALUATION literature have been utilized to develop this training manual. Samples of EVALUATION instruments, reports and presentation materials are drawn from author's work at Michigan State University. Specifically, this manual draws heavily from the following previous digital and print publications: Suvedi, M., & Morford, S. (2003). Conducting Program and Project Evaluations: A Primer for Natural Resource Program Managers in British Columbia. Forrex-Forest Research EXTENSION Partnership, Kamloops, Forrex Series 6.

5 Suvedi, M., Heinze, K., & Ruonavaara, D. (2008). How to conduct EVALUATION of EXTENSION programs. In Suvedi, M., Singh, B., Vijayaragavan, K., Padaria, R., & Wason, M. (Eds.) EVALUATION capacity building in rural resource management: A manual. India: Golden Printers & Publishers. i TABLE OF CONTENTS. About the EVALUATION Training Manual .. v Chapter 1. History and Development of AGRICULTURAL EXTENSION and Advisory Services .. 1. AGRICULTURAL EXTENSION and Advisory Service Models and Approaches .. 2. AGRICULTURAL EXTENSION in the 21st Century .. 5.

6 Challenges to EVALUATION of EXTENSION Programs and Services .. 6. Chapter 8. Introduction to Program EVALUATION .. 8. Why Evaluate a Program?.. 10. What is the Role of the Evaluator? .. 11. What are Common EVALUATION Approaches and Types? .. 12. Evaluative Studies Conducted at the Planning Stage .. 14. Evaluative Studies Conducted During Program Implementation .. 15. Evaluative Studies Conducted at the End of the Program or After .. 16. Qualitative versus Quantitative Studies .. 16. Evaluative Studies to Measure Long-term Changes .. 19. Does EVALUATION Involve Research?

7 19. Which Experimental Designs are Useful for EVALUATION ? .. 21. Ethics of EVALUATION .. 25. Popular Program EVALUATION Frameworks and Designs .. 25. Program Logic Model .. 25. Hierarchy of Evidence (Hierarchy of Program EVALUATION ) .. 27. Program EVALUATION Cycle .. 30. Results 32. Chapter 33. Step 1: Creating a Climate for EVALUATION .. 33. Chapter IV .. 36. Developing an EVALUATION Plan .. 36. ii Step 2: Identifying the Program Phase and EVALUATION Type .. 36. Step 3: Assess the Feasibility of Implementing an EVALUATION Study .. 37. Step 4: Identify and Consult Key Stakeholders.

8 38. Clarify EVALUATION Issues, Questions, Indicators, and Criteria .. 39. Step 5: Identify Approaches to Data Collection .. 41. Chapter V .. 45. Managing EVALUATION Data Collection .. 45. Step 6: Selecting Data Collection Techniques .. 45. Secondary Data (Existing Information).. 47. Data from Testing and Examination .. 47. Questionnaire Surveys and Interviews .. 49. Creating Quality Surveys by Avoiding Errors .. 55. Focus Group .. 63. Rapid Rural Appraisal .. 66. Case Study .. 68. Semi-structured Interviews with Key Informants .. 68. Participant 69.

9 Economic Analyses for EVALUATION .. 71. Benefit/Cost Analysis .. 72. Step 7: Sampling for Program EVALUATION .. 74. Chapter VI .. 81. Step 8: Collecting and Analyzing EVALUATION data .. 81. Organizing EVALUATION Data .. 82. Analyzing and Interpreting 82. Chapter VII .. 98. Step 9: Communicate EVALUATION Findings .. 98. Reporting Negative Findings .. 99. Step 10: Use of EVALUATION Results .. 100. Reflecting on the EVALUATION .. 100. iii Appendix A: Online Resources on Program EVALUATION .. 102. Appendix B: Sample Cover Letter to a Mail Survey .. 103.

10 Appendix C: Sample Questionnaire .. 104. Appendix D: Sample Short Course EVALUATION 107. Appendix E: Sample Introduction to Focus Group .. 108. Appendix F: Sample Focus Group Questions .. 109. Appendix G: Sample Case Study .. 110. Appendix H: Example of Qualitative Data Generated from Open-ended Survey Questions .. 111. References .. 113. iv ABOUT THE EVALUATION TRAINING MANUAL. Planners and policy-makers in EXTENSION systems are busy professionals who may lack the skills and core competencies to guide a systematic EVALUATION of AGRICULTURAL EXTENSION initiatives and programs.