Basic Concepts in - psc.gov.za

1 Basic Concepts inMonitoring and EvaluationFebruary 2008 Published in the Republic of South Africa by:THE PUBLIC SERVICE COMMISSION (PSC)Commission HouseCnr. Hamilton & Ziervogel StreetsArcadia, 0083 Private Bag x121 Pretoria, 0001 Tel: (012) 352-1000 Fax: (012) 325-8382 Website: Anti-Corruption Hotline Number:0800 701 701 (Toll-Free)Compiled by: Branch Monitoring and EvaluationDistributed by Directorate: Communication and Infomation ServicesISBN: 978-0-621-37612-8RP: 14/2008iiFOREWORD ivCHAPTER 1 INTRODUCTION The purpose of this document Intended audience Definition of Monitoring and Evaluation Importance of Monitoring and Evaluation Purposes (and uses) of Monitoring and Evaluation Outline of this document 6 CHAPTER 2.

2 THE CONTEXT OF MONITORING AND EVALUATION The Context of the Developmental State Location of M&E in the policy process Location of M&E in the planning process The Government-wide M&E System Important institutions with a role in M&E Departments at the centre of government on the national level 14 The Presidency 14 National Treasury 14 Department of Public Service and Administration (DPSA) 14 Department of Provincial and Local Government (DPLG) 15 Statistics South Africa (Stats SA) 15 South African Management Development Institute (SAMDI) Departments at the centre of government on the provincial level Line departments Constitutional institutions 15 Public Service Commission (PSC) 16 Auditor-General 16 Human Rights Commission 16 CHAPTER 3.

3 EVALUATION PERSPECTIVES The idea of evaluation perspective The Balanced Scorecard of Kaplan and Norton Programme performance perspective Financial perspective Governance perspective Human Resource Management (HRM) perspective Ethics perspective The perspectives adopted by National Treasury Guidelines 23 CHAPTER 4: VALUES The value basis of monitoring and evaluation Deriving standards of performance from values The values and principles governing public administration The Eight Principles of Batho Pele Other Concepts / principles expressing some dimension of public service performance 35 CHAPTER 5: EVALUATING PROGRAMME PERFORMANCE Programme evaluation Logic models Results-Based Management Theory-based evaluation 44ivCHAPTER 6.

4 APPLYING THE Concepts Focusing monitoring and evaluation Designing monitoring frameworks Framing evaluation questions Examples of monitoring and evaluation of different dimensions 54 CHAPTER 7: SUMMARY 60 LIST OF SOURCES CONSULTED 64 INDEX 68 Figures1 The policy life cycle 92 Planning and review cycle 103 Policy frameworks of the Government-wide Monitoring and Evaluation System 124 Components of the logic model 425 Example of an outcome chain 45 Tables1 Deriving indicators and standards from values 272 Examples of objectives and indicators according to the logic model 503 Examples of evaluation questions: Housing programme 524 Types of monitoring and evaluation in relation to evaluation perspectives and values 55 Boxes1 Example of a theory underlying causal relationship between outputs and outcomes 29v2 Examples of evaluative findings with regard to the efficiency of programmes 303 Examples of evaluative findings with regard to the effectiveness of programmes 364 Examples of evaluative findings with regard to the sustainability of programmes 375 Examples of evaluative findings with regard to the scale of engagement of programmes 376 Examples of evaluative findings with regard

5 To the targeting of programmes 407 Illustrations of types of evaluations: Housing programme 448 Examples of a theory underlying the causal relationship between outputs and outcomes 459 Illustrations of types of evaluations: Housing Programme 58viThe growth of Monitoring and Evaluation (M&E) units in government, together with an increased supply of M&E expertise from the private sector, calls for a common language on M&E. M&E is a relatively new practice, which tends to be informed by varied ideologies and Concepts . A danger for government departments is that these diverse ideological and conceptual approaches can exacerbate confusion and misalignment.

6 The standardisation of Concepts and approaches in government is particularly crucial for the enhancement of service delivery. The PSC s mandate requires of it to monitor and evaluate the organization and administration, and the personnel practices, of the Public Service. Taking this mandate and the need for a common understanding of Concepts and approaches into account, the PSC decided to produce this text on Basic M&E very Basic question asked when a monitoring system must be developed or when an evaluation is planned is: What to monitor or evaluate, that is, what should the focus of the monitoring or the evaluation be? This document tries to answer this Basic question by introducing Concepts and involves a value judgement.

7 Many of the Concepts discussed in the document have the status of values. The PSC has the specific constitutional responsibility to promote the application of these values in the Public document is by no means definitive, but the PSC hopes that it will contribute to better understanding and enriched debate about the utility of M&E as a tool for improving the performance of the Public Service. We hope that it will fill the gap that currently exists for an accessible document that caters for managers in the Public Service, whilst also providing a common point of reference to the more advanced practitioner. It is hoped that readers will feel compelled to delve more deeply into the trust that this document helps you to deepen your interest and understanding of monitoring and sincerelyPROFESSOR STAN S SANGWENICHAIRPERSON: PUBLIC SERVICE COMMISSIONF orewordChapter OneIntroduction122 INTRODUCTIONThis Chapter deals with the following.

8 Purpose of this document Intended audience Definition of monitoring and of evaluation Importance of monitoring and evaluation Purposes (and uses) of monitoring and evaluation Content The purpose of this documentThe purpose of this document is to clarify Basic M&E Concepts and ideas as they apply in the context of the SA Public Service; put the Concepts in a framework showing the interrelationships between them; contribute to the development of a coherent and dynamic culture of monitoring and evaluation in the Public Service; and contribute to a better understanding and enriched debate about the different dimensions of public sector this document will complement and enhance the work being done as part of the Government-Wide Monitoring and Evaluation System, particularly the Framework for Managing Programme Performance Information (published by the National Treasury in 2007) and the Policy Framework for the Government-Wide Monitoring and Evaluation System (published by the Policy Coordination and Advisory Services in the Presidency, in 2007).

9 Intended audienceThis document is intended for use by M&E practitioners; Senior Management in the Public Service; and managers of service delivery units, who produce performance information and statistics, and who are also required from time to time to reflect on and evaluate the success of their Definition of Monitoring and EvaluationMonitoring and evaluation have been defined as:Monitoring A continuing function that uses systematic collection of data on specified indicators to provide management and the main stakeholders of an ongoing development intervention with indications of the extent of progress and achievement of objectives and progress in the use of allocated funds The systematic and objective assessment of an on-going or completed project, programme or policy, its design, implementation and results.

10 The aim is to determine the relevance and fulfillment of objectives, development efficiency, effectiveness, impact and sustainability. An evaluation should provide information that is credible and useful, enabling the incorporation of lessons learned into the decision-making process of both recipients and also refers to the process of determining the worth or significance of an activity, policy or programme. An assessment, as systematic and objective as possible, of a planned, on-going, or completed development : Evaluation in some instances involves the definition of appropriate standards, the examination of performance against those standards, an assessment of actual and expected results and the identification of relevant lessons above definitions are widely used by the development assistance community.