Cost Effectiveness and Productivity KPIs

1 PERFORMANCE REVIEW UNITW orking Paper Cost Effectiveness and Productivity KPIs Prepared by the KPI DRAFTING :October 2001 PERFORMANCE REVIEW UNITKPI Drafting Group Working PaperOctober 2001(i)KPI Working Paper( ) Summary .. of ATM Performance .. Performance Environment .. Difficulties .. for Cost Effectiveness and Productivity .. Physical Capital .. Expenses .. Staff Costs .. Operating Costs .. Depreciation & Amortisation Costs .. The Cost of Capital: The Return on Fixed Assets and Working .. En-route .. Terminal .. Other .. En-route .. Terminal .. 20 PERFORMANCE REVIEW UNITKPI Drafting Group Working PaperOctober 2001(ii)KPI Working Paper( ) .. En-route .. Terminal .. of .. Efficiency of Production .. Efficiency of Allocation .. Staff Productivity .. Asset Productivity .. Multi-factor Interesting Indicators.

2 Looking KPIs .. Work .. 27 Annex 1: KPI Drafting Group Attendance Lists .. 29 Glossary of Definitions ..30 ILLUSTRATIONS Figure 1: Context of Work .. 5 Figure 2: A Performance 7 Figure 3: Metrics for Cost Effectiveness and Productivity 11 Figure 4: ATCO Hours Worked Per 13 Figure 5: Different Sources of En-route 18 TABLES Table 1: KPIs for Cost Effectiveness .. 23 Table 2: KPIs for Efficiency of Production .. 24 Table 3: KPIs for Efficiency of Allocation .. 24 Table 4: KPIs for Staff Productivity .. 25 PERFORMANCE REVIEW UNITKPI Drafting Group Working PaperOctober 2001(iii)K P I W o r k i n g P a p e r ( v 1 . 0 ) f i n a l 1 . d o c Table 5: KPIs for Asset 25 Table 6: Other Interesting Indicators .. 26 Table 7: Specific Forward Looking Information .. 27 PERFORMANCE REVIEW UNITKPI Drafting Group Working PaperOctober 2001 Page 1K P I W o r k i n g P a p e r ( v 1 . 0 ) f i n a l 1.

3 D o cEXECUTIVE SUMMARYA Validation Phase of the Pilot Project on Information Disclosure was launched on 28 March2001 with a meeting of the participating air navigation service providers (ANSPs) from Phase1, together with representatives from six additional ANSPs. The airspace users communitywas also represented by participants from IATA, British Airways, and this meeting it was decided that the previous participants and the representatives of thenew participating ANSPs would form the Steering Group for the Validation Phase onInformation Disclosure and that they should put forward nominees to form a KPI DraftingGroup (KPIDG) which would work at a more detailed level. The objectives of the KPIDG wereto develop a performance framework for the measurement of cost Effectiveness andproductivity, and to identify a set of high level KPIs that could be used to measure differentaspects of performance within this KPIDG has met five times between 23 April and 12 September 2001 and has beenattended by participants from 17 organisations representing ANSPs, Civil Aviation Authorities(CAAs) and airspace users.

4 Each meeting has been a working session, led and facilitated bythe Performance Review Unit (PRU). A major output has been this working paper whichidentifies a selection of KPIs at the gate-to-gate, en-route, terminal and ACC levels. Toensure consistency in both the collection of data and the interpretation of indicators, astandard set of definitions relating to inputs, outputs and operations has been definitions have been used where possible, but novel definitions have beennecessary in certain areas. Care has been taken to harmonise these definitions with thosebeing developed by scope of the work of the KPIDG has been limited to the development of precise KPIs forcost Effectiveness , Productivity and efficiency. The KPIDG has not considered otherimportant, but indirect components of value-for-money such as the cost of ATM delay and thecost of inefficient flight sample indicators that have been prepared using the 2000 data display considerablevariation in performance between ANSPs.

5 In this working paper, the KPIDG draw attention tothe many and various exogenous factors that contribute to the variation and create difficulty inmaking fair comparisons between ANSPs unless they can be quantified. These exogenousfactors are related to the institutional and operational environment in which the ANSP operates and are mainly outside an ANSP s control. Some of these factors , such ascomplexity, are difficult to measure and will clearly require further work and modelling beforeagreement is reached on a satisfactory definition that takes into account the variousdimensions involved. Since many of the exogenous factors that affect the performance of aparticular ANSP will remain fairly constant from year to year, KPIs are particularly relevant totrack the evolution of performance of a given KPIDG has recognised that complementary benchmarking tools such as multivariateregression analysis and total factor Productivity should also be used to provide a richer andfairer picture of performance and give further insight, but such tools will require further this context, full information disclosure should be encouraged in order to throw light onsome of the factors that can cause KPIDG has identified several situations where care is needed when measuringperformance.

6 Difficulties can arise when one state/ANSP delegates air navigation services toanother state/ANSP, when civil and military OAT are partially or fully integrated, and whendifferent outsourcing (contracting out) policies are used. In all cases it is necessary to ensurethat consistency is maintained between the measurement of inputs and the measurement following figure illustrates a framework that has been identified by the KPIDG for themeasurement of cost Effectiveness and Productivity . The framework includes the principlePERFORMANCE REVIEW UNITKPI Drafting Group Working PaperOctober 2001 Page 2K P I W o r k i n g P a p e r ( v 1 . 0 ) f i n a l 1 . d o cmetrics that are required to measure performance and the arrows show the way in which themetrics can be connected to measure different aspects of working paper provides an in depth discussion of each part of the framework, exploresdifferent metrics that could be used to measure output, production and capacity, and suggestsindicators for each aspect of en-route, terminal and gate-to-gate performance.

7 High levelindicators are summarised in a table on the following working paper has been approved by the Steering Group who have also prepared of aSpecification for Information Disclosure (Version 2) which will provide all the necessary datato populate the indicators and will provide additional information to enable analysis of theseindicators to be the KPIDG and the Steering Group recognise that while historical indicators areinteresting, they are insufficient on their own to predict future performance. Informationdisclosure will, therefore, require specific forward-looking information to be provided coveringthe key inputs, outputs and working paper concludes that similar indicators can and should be used at the level of anoperational unit (ACC, TWR, etc.) where benchmarking is more meaningful and appropriatebecause operational units operating within a similar environment can be grouped togetherand compared.

8 It also concludes that since the majority of costs fall under the heading ofATM/CNS, they should be further split into the major service components (Communications,Navigation, Surveillance, MET, AIS, etc.) so that the cost Effectiveness of significant servicescan be sers chargesOutputs/ServicesProduction/Capaci tyCost effectivenessEfficiencyofallocationEffic iency of productionStaff productivityAssets productivityUnit rateC o st s O perating costsFi xed Asset sSt a f fSu r p l u s, t a xandfinancialcost sD epreciationStaff costsO perating expensesCapital expensesCost of capitalRevenuePERFORMANCE REVIEW UNITKPI Drafting Group Working PaperOctober 2001 Page 3K P I W o r k i n g P a p e r ( v 1 . 0 ) f i n a l 1 . d o cEn-routeANST erminalANSO therGate-to-GateCOST EFFECTIVENESST otal cost / Km controlled (or flight hours) Operating expenses / Km controlled (or flight hours) total terminal cost / total airport movements Terminal operating expenses / total airport movements EFFICIENCY OF PRODUCTIONT otal cost / Sector hours (or Number of sectors) Operating expenses / Sector hours (or Number ofsectors) total cost (or operating expenses) / Flight hourscontrollable EFFICIENCY OF ALLOCATIONF light hours controlled / Flight hours controllable STAFF PRODUCTIVITYT otal ATCOs hours on duty in OPS / Number of ATCOsin OPS Sector hours / Number of en-route ATCOs in OPS total airport movements / Number of terminal ATCOsin OPS Flight hours controlled / Number of ATCOs in OPS ASSET PRODUCTIVITYS ector hours (or number of sectors) / NBV fixed assetsin operation Km controlled (or Flight hours controlled)

9 / NBV fixedassets in operation Number of movements / NBV fixed assets in operation Table: Summary of Key Performance IndicatorsPERFORMANCE REVIEW UNITKPI Drafting Group Working PaperOctober 2001 Page 4K P I W o r k i n g P a p e r ( v 1 . 0 ) f i n a l 1 . d o c1. INTRODUCTIONAs part of the work undertaken in 2001 during the Validation Phase of Information Disclosure,a Key Performance Indicator (KPI) Drafting Group (KPIDG) was set up to establish a linkbetween performance measurement and economic information disclosure. More specifically,the KPIDG has been formed to carry out the following tasks:(i) To develop a performance framework for measuring cost Effectiveness andproductivity;(ii) To use the performance framework to identify suitable high level indicators tomeasure different aspects of cost Effectiveness and Productivity ;(iii) To identify exogenous factors influencing performance;(iv) To propose a set of KPIs of cost Effectiveness , Productivity and efficiency to theSteering Group on Information Disclosure KPIDG consists of representatives from a number of European Air Navigation ServiceProviders (ANSPs) and airspace users, and is led and facilitated by the Performance ReviewUnit (PRU).

10 By working together in a collaborative manner the KPIDG have been able todevelop a mutual understanding of how best to measure performance and have also beenable to identify joint solutions to problems in defining and collecting data in a KPIDG met five times and attendance lists are provided in Annex BACKGROUNDIn 1997 the Member States of EUROCONTROL jointly took the decision to establish anindependent performance review system that will address all aspects of air trafficmanagement, including policy and planning, safety management at and around airports andin the airspace, as well as financial and economic aspects of services rendered, and settargets that will address these aspects Member States also decided to study and promote measures for improving costeffectiveness and efficiency in the field of air navigation a result of these decisions the Performance Review Commission (PRC) was establishedto ensure the effective management of the European air traffic management system through astrong, transparent and independent performance review and target setting the request of the ECAC Transport Ministers, a comprehensive, gate-to-gate oriented ATMS trategy for 2000+ has been developed.