Framework for Program Evaluation in Public Health

1 Inside: Continuing Education ExaminationSeptember 17, 1999 / Vol. 48 / No. DEPARTMENT OF Health & HUMAN SERVICESC enters for Disease Control and Prevention (CDC)Atlanta, Georgia 30333 Framework for Program Evaluationin Public HealthInside: Continuing Education ExaminationCopies can be purchased from Superintendent of Documents, GovernmentPrinting Office, Washington, DC 20402-9325. Telephone: (202) of trade names and commercial sources is for identification only and does notimply endorsement by the Department of Health and Human MMWR series of publications is published by the Epidemiology Program Office,Centers for Disease Control and Prevention (CDC), Department of Health and Hu-man Services, Atlanta, GA for Disease Control and P.

2 Koplan, , The production of this report as an MMWR serial publication was coordinated in Epidemiology Program Barbara Holloway, Director Office of Scientific and Health Communications .. John W. Ward, Editor, MMWR Series Recommendations and Suzanne M. Hewitt, Editor C. Kay Smith-Akin, Editor Morie M. Higgins Peter M. JenkinsVisual Information Specialists SUGGESTED CITATIONC enters for Disease Control and Prevention. Framework for Program Evaluation inpublic Health . MMWR 1999;48(No. RR-11):[inclusive page numbers].References to sites of non-CDC organizations on the Internet are provided as a serv-ice to MMWR readers and do not constitute or imply endorsement of theseorganizations or their programs by CDC or the Department of Health and Hu-man Services.

3 CDC is not responsible for the content of pages found at these ..iiIntroduction ..1 Background ..2 Procedures for Developing the Key Concepts ..3 Integrating Evaluation with Routine Program Value To Program Activities ..3 Framework For Program Evaluation in Public Health ..4 Steps in Program Evaluation ..5 Step 1: Engaging 2: Describing the 3: Focusing the Evaluation 10 Step 4: Gathering Credible 14 Step 5: Justifying Conclusions .. 18 Step 6: Ensuring Use and Sharing Lessons Learned .. 22 Standards for Effective Evaluation ..26 Standard 1: Utility .. 27 Standard 2: Feasibility .. 27 Standard 3: 27 Standard 4: 29 Applying The Optimal Evaluations ..31 Assembling an Evaluation Team ..32 Addressing Common Concerns ..33 Evaluation Trends.

4 Information .. 48 / No. RR-11 MMWRiFOREWORDH ealth improvement is what Public Health professionals strive to achieve. To reachthis goal, we must devote our skill and our will to evaluating the effects of publichealth actions. As the targets of Public Health actions have expanded beyond infec-tious diseases to include chronic diseases, violence, emerging pathogens, threats ofbioterrorism, and the social contexts that influence Health disparities, the task ofevaluation has become more complex. CDC developed the Framework for programevaluation to ensure that amidst the complex transition in Public Health , we willremain accountable and committed to achieving measurable Health integrating the principles of this Framework into all CDC Program operations, wewill stimulate innovation toward outcome improvement and be better positioned todetect Program effects.

5 More efficient and timely detection of these effects willenhance our ability to translate findings into practice. Guided by the steps and stand - ards in the Framework , our basic approach to Program planning will also from prevention research will lead to Program plans that are clearer andmore logical; stronger partnerships will allow collaborators to focus on achievingcommon goals; integrated information systems will support more systematic meas-urement; and lessons learned from evaluations will be used more effectively to guidechanges in Public Health of this Framework also emphasizes CDC s continuing commitment toimproving overall community Health . Because categorical strategies cannot succeedin isolation, Public Health professionals working across Program areas must collabo-rate in evaluating their combined influence on Health in the community.

6 Only then willwe be able to realize and demonstrate the success of our vision healthy people in ahealthy world through P. Koplan, , , Centers for Disease Control and PreventionAdministrator, Agency for Toxic Substances and Disease RegistryiiMMWRS eptember 17, 1999 The following CDC staff members prepared this report: Robert L. Milstein, Office of Program Planning and Evaluation Office of the Director Scott F. Wetterhall, , Chair, CDC Evaluation Working Group Office of Program Planning and Evaluation Office of the Directorin collaboration withCDC Evaluation Working Group MembersGregory M. Christenson, Dennis-FlaglerDivision of Health Education and PromotionAgency for Toxic Substances and Disease RegistryJeffrey R.

7 Harris, L. Higgins, A. Schachter, , of Prevention Research and Analytic MethodsNancy F. Pegg, of the DirectorEpidemiology Program OfficeJanet L. Collins, , of Adolescent and School HealthDiane O. Dunet, of Cancer Prevention and ControlAliki A. Pappas, , of Oral HealthNational Center for Chronic Disease Prevention and Health PromotionAlison E. Kelly, of the DirectorNational Center for Environmental HealthPaul J. Placek, of Data Standards, Program Development, and Extramural ProgramsNational Center for Health StatisticsMichael Hennessy, , of STD PreventionDeborah L. Rugg, of HIV/AIDS Prevention Intervention, Research, and SupportNational Center for HIV, STD, and TB PreventionVol. 48 / No. RR-11 MMWRiiiApril J.

8 Bell, A. Deppe, I. Meltzer, , D. Wiley, of the DirectorNational Center for Infectious DiseasesThomas A. Bartenfeld, III, of the DirectorNational Center for Injury Prevention and ControlRoger H. Bernier, , of the DirectorNational Immunization ProgramMax R. Lum, of the DirectorNational Institute for Occupational Safety and HealthGalen E. Cole, , of CommunicationKathy Cahill, Carmack, E. Cheal, of Program Planning and EvaluationHope S. King, D. Moulton, of the DirectorEunice R. Rosner, , of Laboratory SystemPublic Health Practice Program OfficeWilliam Kassler, Hampshire Department of Health and Human ServicesConcord, New HampshireJoyce J. Neal, , of State and Territorial EpidemiologistsAtlanta, GeorgiaivMMWRS eptember 17, 1999 Additional CDC ContributorsOffice of the Director: Lynda S.

9 Doll, , ; Charles W. Gollmar; Richard , , ; Wilma G. Johnson, ; Marguerite Pappaioanou, , , ; David J. Sencer, , (Retired); Dixie E. Snider, , ; Marjorie A. Speers, ; Lisa R. Tylor; and Kelly O Brien Yehl, (Wash-ington, ).Agency for Toxic Substances and Disease Registry: Peter J. McCumiskey andTim L. Tinker, , Program Office: Jeanne L. Alongi, ( Public Health PreventionService); Peter A. Briss, ; Andrew L. Dannenberg, , ; Daniel B. Fishbein, ; Dennis F. Jarvis, ; Mark L. Messonnier, , ; Bradford A. Myers; RaulA. Romaguera, , ; Steven B. Thacker, , ; Benedict I. Truman, , ; Katherine R. Turner, ( Public Health Prevention Service); Jennifer , ( Public Health Prevention Service); G.

10 David Williamson, ; andStephanie Zaza, , Center for Chronic Disease Prevention and Health Promotion: Cynthia , ; Marshall W. Kreuter, , ; R. Brick Lancaster, ; ImaniMa at, , , ; Elizabeth Majestic, , ; David V. McQueen, , ; Diane M. Narkunas, ; Dearell R. Niemeyer, ; and Lori B. de Ravello, Center for Environmental Health : Jami L. Fraze, ; Joan ; William C. Parra, ; Judith R. Qualters, ; Michael J. Sage, ;Joseph B. Smith; and Ronald R. Center for Health Statistics: Marjorie S. Greenberg, and Jennfier , Center for HIV, STD, and TB Prevention: Huey-Tsyh Chen, ; JanetC. Cleveland, ; Holly J. Dixon; Janice P. Hiland, ; Richard A. Jenkins, ; JillK. Leslie; Mark N. Lobato, ; Kathleen M.