Multi-criteria analysis: a manual - GOV.UK

1 Multi-criteria analysis : a , opportunity, prosperityMulti- criteria analysis : a manualJanuary 2009 Department for Communities and Local Government: LondonDepartment for Communities and Local GovernmentEland HouseBressenden PlaceLondon SW1E 5 DUTelephone: 020 7944 4400 Website: Crown copyright 2009 Copyright in the typographical arrangement rests with the publication, excluding logos, may be reproduced free of charge in any format or medium for research, private study or for internal circulation within an organisation. This is subject to it being reproduced accurately and not used in a misleading context. The material must be acknowledged as Crown copyright and the title of the publication other use of the contents of this publication would require a copyright licence.

2 Please apply for a Click-Use Licence for core material at , or by writing to the Office of Public Sector Information, Information Policy Team, Kew, Richmond, Surrey TW9 4 DUe-mail: you require this publication in an alternative format please email and Local Government PublicationsPO Box 236 WetherbyWest YorkshireLS23 7 NBTel: 0300 123 1124 Fax: 0300 123 1125 Email: via the Communities and Local Government website: 2009 Product Code: 08 ACST05703 ISBN: 978-1-4098-1023-0 Multi-criteria analysis : a manual | 3 Contents Preface 5 Chapter 1 The scope and objectives of this manual 6 Chapter 2 Appraisal and evaluation in government 9 Introduction 9 The decision making process 10 Identifying objectives 10 Identifying options for achieving the objectives 11 Identifying the criteria to be used to compare the options 11 analysis of the options 11 Making choices 12

3 Feedback 13 Sequences of decisions through time 13 Chapter 3 Monetary-based techniques 14 Introduction 14 Financial analysis 14 Cost-effectiveness analysis 15 Cost-benefit analysis 15 Inclusion of the results of monetary analyses in an MCA framework 17 Chapter 4 An overview of Multi-criteria analysis techniques 19 Introduction 19 criteria for selecting MCA techniques 20 Key features of MCA 20 Different types of MCA 23 Direct analysis of the performance matrix 24 multi -attribute utility theory 24 Linear additive models 25 The Analytical Hierarchy Process 26 Outranking methods 26 Procedures that use qualitative data inputs 27 MCA methods based on fuzzy sets 28 Other MCA methods 29 Application of MCA in government 294 | Multi-criteria analysis : a manualChapter 5 Decisions without weights: construction of a performance matrix 30 The stages of a Multi-criteria analysis 30 Step 1.

4 Establishing the decision context 32 Step 2: Identifying options 32 Step 3: Identifying criteria and sub- criteria 32 Step 4 and beyond (without scoring and weighting) 39 Step 4: Assessing performance levels (with scoring) 41 Chapter 6 Multi-criteria decision analysis 46 What is MCDA? 46 Stages in MCDA 49 Uncertainty, Risk and MCDA 71 Chapter 7 Case studies 73 The new approach to appraisal for transport 73 UK applications of MCDA 77 The NAO case study: evaluation of overseas trade services 78 The Nirex case study: appraisal of sites for further investigation as potential repositories for radioactive waste 90 Medium term revenue budget planning 101 Lessons from the MCDA case studies 109 Appendix 1 Continuous MCA models (infinitely variable alternatives)

5 112 Appendix 2 Non-compensatory methods 115 Appendix 3 multi -attribute utility models 118 Appendix 4 Linear additive models 121 Appendix 5 The analytical hierarchy process 127 Appendix 6 Outranking methods 131 Appendix 7 Outranking methods with qualitative data 138 Appendix 8 Fuzzy MCA 140 Appendix 9 Using MCA to explore stakeholder viewpoints 141 Appendix 10 Probabilities to preferences 146 Glossary 149 Bibliography 154 Software 161 Multi-criteria analysis : a manual | 5 PrefaceThis manual was commissioned by the Department for the Environment, Transport and the Regions in 2000 and remains, in 2009, the principal current central government guidance on the application of Multi-criteria analysis (MCA) techniques.

6 Since 2000 it has become more widely recognised in government that, where quantities can be valued in monetary terms, MCA is not a substitute for cost-benefit analysis , but it may be a complement; and that MCA techniques are diverse in both the kinds of problem that they address (for example prioritisation of programmes as well as single option selection) and in the techniques that they employ, ranging from decision conferencing to less resource intensive processes. 6 | Multi-criteria analysis : a manualChapter 1 The scope and objectives of this manual This manual provides guidance for Government officials and other practitioners on how to undertake and make the best use of Multi-criteria analysis (MCA) for the appraisal of options for policy and other decisions, including but not limited to those having implications for the environment.

7 It covers a range of techniques which can be of practical value to public decision makers and are increasingly being used in the UK and in other countries. They are described in this manual as Multi-criteria analysis (MCA) techniques. The document Policy Appraisal and the Environment was published by DETR in 1997 a report Experience with the Policy Appraisal and the Environment Initiative concluded that there was still room for The Policy Appraisal and the Environment: Policy Guidance leaflet was published by the DETR in April In the same year a scoping study was commissioned on techniques and published guidance on environment One suggestion for future action was the production of general guidance on Multi-criteria analysis .

8 This increasingly popular set of techniques typically combines a range of project, policy or programme option impacts into a single framework for easier assimilation by decision makers . The present manual implements this recommendation. The manual is about techniques which do not necessarily rely on monetary valuations. It therefore complements guidance on those techniques which primarily use monetary valuations, namely financial analysis , cost effectiveness analysis (CEA), and cost-benefit analysis (CBA). These monetary techniques have been extensively used in UK government circles, and are the subject of a number of guides and manuals. Most officials are familiar with monetary techniques, but less so with the MCA techniques described in this manual .

9 Nonetheless highway investments, for example, have for many years been appraised using procedures that take account both of impacts measured in monetary units, such as construction costs, time savings and reductions in accident costs, and of social and environmental impacts that may be quantified but not valued (such as the number of houses suffering specified increases in noise) or assessed only in qualitative terms (such as impacts on landscape). In 1998 DETR developed an MCA approach in the form of the New Approach to Appraisal for transport projects. This provides an improved way of presenting monetised and non-monetised impacts of transport projects to decision makers . Other MCA procedures, including scoring and weighting are also already used 1 Department of the Environment (1991) Policy Appraisal and the Environment HMSO, DETR (1997), Experience with the Policy Appraisal and the Environment Initiative KPMG for DETR, DETR (1998a) Policy Appraisal and the Environment: Policy Guidance DETR, DETR (1998b) Review of Technical Guidance on Environmental Appraisal DETR, London, reporting work produced by analysis : a manual | 7from time to time elsewhere in government.

10 For example, the Environment Agency have been developing the techniques to appraise improvements in water quality, while scoring and weighting systems have been used to assess Structural Fund This manual provides practical guidance on the application of these techniques, in non-technical language. It is designed to help non-specialist staff to gain an overview of the advantages offered by MCA and what its requirements may be in terms of resources for undertaking appraisals. The manual also has more detailed appendices on various MCA methodologies which will be more accessible to economists and other analytical specialists. One source of confusion to those new to the field is the variety of different techniques, often with rather similar sounding titles, that appear to be available, such as Multi-criteria decision analysis , multi -attribute utility theory, the analytic hierarchy process, and fuzzy set theory.