VOLUME M – METHODOLOGY - eebestpractices.com

1 NATIONAL energy EFFICIENCY BEST PRACTICES STUDY VOLUME M METHODOLOGY Submitted to California Best Practices Project Advisory Committee Kenneth James Contract Manager Pacific Gas and Electric Company Box 770000,N6G San Francisco, CA 94177 Submitted by PRIME CONTRACTOR QUANTUM CONSULTING INC. 2001 Addison St, Suite 300 Berkeley, CA 94704 With Assistance from Megdal Associates, Research Into Action, Frontier Associates, and Shel Feldman Management Consultants December 2004 2004 Quantum Consulting Inc. All Rights Reserved. ConsultingQuantumQCQuantum Consulting Inc.

2 I Table of Contents METHODOLOGY TABLE OF CONTENTS SECTION PAGE 1. INTRODUCTION M-1 2. APPROACH M-2 Overview M-2 Definition of Terms M-5 Program Decomposition Model M-6 Cross-Cutting Outcome Metrics M-9 Program Context Characteristics M-10 Program Benchmarking M-12 Process Benchmarking Considerations M-13 3. PROGRAM SELECTION M-15 Selection of Program Categories M-15 Program Screening & Selection M-17 4.

3 DATA COLLECTION M-22 Needs Assessment Meetings M-22 Literature Review M-24 Program Population Data Collection M-24 Data Collection Instrument M-25 Program Benchmarking Data Collection M-26 Data Collection Challenges M-27 APPENDIX A LITERATURE REVIEW FOR NATIONAL energy EFFICIENCY BEST PRACTICES STUDY M-31 APPENDIX B DATA COLLECTION LETTER AND FORMS M-43 Quantum Consulting Inc. ii Best Practices - METHODOLOGY ACKNOWLEDGEMENTS The Best Practices Study team would like to thank the many individuals that participated in the user needs focus groups conducted at the outset of the project.

4 The METHODOLOGY for this study was developed through the joint effort of the consultant and client team. We gratefully thank the members of the Best Practices Study s Project Advisory Committee for their inspiration, insights, tireless review and thoughtful direction throughout the project: Kenneth James Pacific Gas & Electric Company Pierre Landry Southern California Edison Company Rob Rubin Sempra Utilities Jay Luboff California Public Utilities Commission, energy Division Eli Kollman California Public Utilities Commission, energy Division Sylvia Bender California energy Commission Quantum Consulting Inc.

5 M-1 Best Practices - METHODOLOGY 1. INTRODUCTION The overall goal of the Best Practices Study is to develop and implement a method to identify and communicate excellent programmatic practices in order to enhance the design of energy efficiency programs in California. In particular, program implementers supported through Public Goods Charge ( PGC ) funds will be encouraged to use this Study s products, along with other resources and their own knowledge and experience, to develop and refine energy efficiency programs.

6 This Study is intended to be a first, not final, step in a process that would seek to identify and communicate best practices on an on-going or periodic basis. The Study does not expect to produce a census of best practices across all types of programs. Such an approach would be neither practical nor useful given the number of programs that exist; the many differences in policies, goals, and market conditions around the country; the unique needs and market conditions in California; and the importance of encouraging innovation, which by its nature sometimes requires attempting approaches that are not yet proven.

7 Although a few studies and papers exist in the energy efficiency literature that seek to identify exemplary programs and summarize best practices, none of these efforts have done so in the manner intended by the Project Team, the Project Advisory Committee (PAC), and the CPUC originators of the current Unique aspects of the current Study are its comprehensiveness, its use of a program decomposition approach, and its focus on development of a database and user-driven website.

8 The large scope and changing nature of energy efficiency programs and energy markets require that a dynamic approach be employed. Like any study of this type, resource and schedule constraints must limit the scope of the effort. In the current Study, data was collected from roughly 90 programs in total across a range of program Thus, readers and users should recognize that the intent is not to cover all types of programs with this first effort and that the depth of coverage will vary even among the program types that are addressed.

9 If the framework and results of the Study prove useful, it is anticipated that future phases of the work can expand the number and types of programs covered. Because this is one of the first efforts of its type in the energy efficiency program industry, there is a strong methodological focus to the project. The purpose of this chapter is to document and describe the Project Team s approaches to achieving the Study s objectives. 1 See California Public Utilities Commission Opinion ( ), filed August 23, 2001. Quantum Consulting Inc.

10 M-2 Best Practices - METHODOLOGY 2. STUDY APPROACH This section presents the Best Practices Study METHODOLOGY . Specifically, this section details the benchmarking approach developed for this Study which involves decomposing programs into their components and comparing those program elements across selected programs. OVERVIEW An overview of the Study process is shown in Exhibit M-1. Key aspects of the Study include a user needs assessment, secondary research, development of the benchmarking methods, identification and selection of programs to benchmark, development of the program database, data collection and program benchmarking, analysis, and preparation of the Study s best practices report and final database.