Handbook on Constructing Composite Indicators: …

1 Subscribers to this printed periodical are entitled to free online access. If you do not yet have online access via your institution s network, contact your librarian or, if you subscribe personally, send an email to on Constructing Composite Indicators METHODOLOGY AND USER GUIDEISBN 978-92-64-04345-930 2008 25 1 P-:HSTCQE=UYXYZ^: Handbook on Constructing Composite IndicatorsMETHODOLOGY AND USER GUIDEThis Handbook is a guide for Constructing and using Composite indicators for policy makers, academics, the media and other interested parties. While there are several types of Composite indicators, this Handbook is concerned with those which compare and rank country performance in areas such as industrial competitiveness, sustainable development, globalisation and innovation.

2 The Handbook aims to contribute to a better understanding of the complexity of Composite indicators and to an improvement of the techniques currently used to build them. In particular, it contains a set of technical guidelines that can help constructors of Composite indicators to improve the quality of their outputs. Handbook on Constructing Composite IndicatorsMETHODOLOGY AND USER GUIDEAVAILABLE ON LINEH andbookon ConstructingComposite IndicatorsMETHODOLOGY AND USER Page 1 Tuesday, August 19, 2008 7:41 AMORGANISATION FOR ECONOMIC CO-OPERATIONAND DEVELOPMENTThe OECD is a unique forum where the governments of 30 democracies work together toaddress the economic, social and environmental challenges of globalisation. The OECD is also atthe forefront of efforts to understand and to help governments respond to new developments andconcerns, such as corporate governance, the information economy and the challenges of anageing population.

3 The Organisation provides a setting where governments can compare policyexperiences, seek answers to common problems, identify good practice and work to co-ordinatedomestic and international OECD member countries are: Australia, Austria, Belgium, Canada, the Czech Republic,Denmark, Finland, France, Germany, Greece, Hungary, Iceland, Ireland, Italy, Japan, Korea,Luxembourg, Mexico, the Netherlands, New Zealand, Norway, Poland, Portugal, the Slovak Republic,Spain, Sweden, Switzerland, Turkey, the United Kingdom and the United States. The Commission ofthe European Communities takes part in the work of the Publishing disseminates widely the results of the Organisation s statistics gathering andresearch on economic, social and environmental issues, as well as the conventions, guidelines andstandards agreed by its to OECD publications may be found on line at: OECD 2008 You can copy, download or print OECD content for your own use, and you can include excerpts from OECD publications, databases and multimediaproducts in your own documents, presentations, blogs, websites and teaching materials, provided that suitable acknowledgment of OECD as sourceand copyright owner is given.

4 All requests for public or commercial use and translation rights should be submitted Requests forpermission to photocopy portions of this material for public or commercial use shall be addressed directly to the Copyright Clearance Center or the Centre fran ais d'exploitation du droit de copie work is published on the responsibility of the Secretary-General of the OECD. Theopinions expressed and arguments employed herein do not necessarily reflect the official views ofthe Organisation or of the governments of its member countries or of the European the European Commission nor any person acting on behalf of the Commission isresponsible for the use which might be made of this ON Constructing Composite INDICATORS: METHODOLOGY AND USER GUIDE ISBN 978-92-64-04345-9 - OECD 20083 FOREWORDThis Handbook aims to provide a guide to the construction and use of Composite indicators,for policy-makers, academics, the media and other interested parties.

5 While there are several types ofcomposite indicators, this Handbook is concerned with those which compare and rank countryperformance in areas such as industrial competitiveness, sustainable development, globalization andinnovation. The Handbook aims to contribute to a better understanding of the complexity of compositeindicators and to an improvement in the techniques currently used to build them. In particular, itcontains a set of technical guidelines that can help constructors of Composite indicators to improve thequality of their has been jointly prepared by the OECD (the Statistics Directorate and the Directorate forScience, Technology and Industry) and the Econometrics and Applied Statistics Unit of the JointResearch Centre (JRC) of the European Commission in Ispra, Italy.

6 Primary authors from the JRC areMichela Nardo, Michaela Saisana, Andrea Saltelli and Stefano Tarantola. Primary authors from theOECD are Anders Hoffmann and Enrico Giovannini. Editorial assistance was provided by CandiceStevens, Gunseli Baygan, Karsten Olsen and Sarah people contributed to improve this Handbook with their valuable comments andsuggestions. The authors wish to thank especially Jochen Jesinghaus from the European Commission,DG Joint Research Centre; Tanja Srebotnjak from the Yale Center for Environmental Law and Policy;Laurens Cherchye and Tom Van Puyenbroeck from the Catholic University of Leuven; Pascal Rivi refrom INSEE ; Tom Griffin, Senior Statistical Adviser to the UNDP Human Development Report; andAri LATVALA from the European Commission, DG Enterprise and special thank goes to Giuseppe Munda (Universitat Aut noma de Barcelona and EuropeanCommission, DG Joint Research Centre) who supplied all the material for the chapter on aggregationmethods and the box on measurement scales and to Eurostat (Unit B5-'Methodology and research',DDG-02 ' Statistical governance, quality and evaluation' and other members of the Task Force onComposite Indicators) for the useful comments and the improvements they suggested.

7 We are alsograteful to the Statistical Offices of the OECD Committee on Statistics whose comments contributedto enhance the quality of this information on the topics treated in this Handbook and on other issues related tocomposite indicators can be found in the web page: research was partly funded by the European Commission, Research Directorate, underthe project KEI (Knowledge Economy Indicators), Contract FP6 No. 502529. In the OECD context,the work has benefited from a grant from the Danish Government. The views expressed are those ofthe authors and should not be regarded as stating an official position of either the EuropeanCommission or the ON Constructing Composite INDICATORS: METHODOLOGY AND USER GUIDE ISBN 978-92-64-04345-9 - OECD 20085 TABLE OF CONTENTSINTRODUCTION.

8 13 Pros and cons of Composite indicators .. 13 Aim of the Handbook .. 14 What s next .. 17 PART I. Constructing A Composite INDICATOR .. 191. Steps for Constructing a Composite indicator .. Developing a theoretical framework .. Selecting variables .. Imputation of missing data .. Multivariate analysis .. Normalisation of data .. Weighting and aggregation .. Robustness and sensitivity .. Back to the details .. Links to other variables .. 391. 10 Presentation and dissemination .. 402. A quality framework for Composite indicators .. Quality profile for Composite indicators .. Quality dimensions for basic data .. Quality dimensions for procedures to build and disseminate Composite indicators .. 48 PART 2.

9 A TOOLBOX FOR CONSTRUCTORS .. 51 Step 3. Imputation of missing data .. Single imputation .. Unconditional mean imputation .. Regression imputation .. Expected maximisation imputation .. Multiple imputation .. 58 Step 4. Multivariate analysis .. Principal components analysis .. Factor analysis .. Cronbach Coefficient Alpha .. Cluster analysis .. Other methods for multivariate analysis .. 796 Handbook ON Constructing Composite INDICATORS: METHODOLOGY AND USER GUIDE ISBN 978-92-64-04345-9 - OECD 2008 Step 5. Normalisation .. Scale transformation prior to normalisation .. Standardisation (or z-scores) .. Min-Max .. Distance to a reference .. Indicators above or below the mean .. Methods for cyclical indicators.

10 Percentage of annual differences over consecutive years .. 86 Step 6. Weighting and aggregation .. 89 Weighting methods .. Weights based on principal components analysis or factor analysis .. Data envelopment analysis (DEA) .. Benefit of the doubt approach (BOD) .. Unobserved components model (UCM) .. Budget allocation process (BAP) .. Public opinion .. Analytic hierarchy process (AHP) .. Conjoint analysis (CA) .. Performance of the different weighting methods .. 99 Aggregation methods .. Additive aggregation methods .. Geometric aggregation .. On the aggregation rules issue: lessons learned from social choice and multi-criteria decision analysis .. Non-compensatory multi-criteria approach (MCA) .. Performance of the different aggregation methods.