Measuring Impact of Educational Programs

1 Measuring Impact ofEducational ProgramsWhat is Impact ? Impact is the difference we make in people s lives as aresult of Programs we conduct. These Programs may beresearch or teaching and may or may not involve thepublic directly while they are being delivered. Yet theresults must ultimately change people s attitudes or be-havior, or benefit society in other is Impact important? To justify the investment of time and effort, as well asthe dedication of public and private funds. To earn and build professional, organizational, andpolitical credibility and support.

2 To yield tangible results that serve as the base forscholarly publications, as well as awards and recogni-tion. To satisfy the requirements of political bodies andfunding begin, start with the end !The program development model used by CooperativeExtension incorporates needs assessment, developmentof program objectives based on the organization s mis-sion to meet those needs, program planning and deliv-ery, and evaluation. Measuring Impact is part of theevaluation component. Despite what is commonlybelieved and typically practiced, however, evaluationneeds to be considered while a program is being devel-oped, as well as during its delivery and after its are some points to consider when developing pro-grams so that Measuring Impact becomes an integral partof program development:1.

3 Develop goals for Programs basedon are the audiences to be served? What are theoutcomes to be sought? Are these outcomes attainableand measurable? Do potential Programs fit local orclientele interests, state strategic plans, national initia-tives, or federal performance goals?2. Assess resources available toconduct surely influences objectives and affects programviability and success. Are you relying on public funds,grant monies, or user fees? How stable is the funding?Consider what funding agencies will require when youreport results.

4 Be sure to build evaluation needs and costsinto your Determine time and staff available. If choices must bemade, which Programs are likely to have greater impacton more people? Which ones are likeliest to grow?Which might be turned over to volunteers or advisorygroups to maintain? Which might provide the greatestpositive media attention and other forms of recognitionfor you and the organization? Which are likely togenerate the most scholarly publications? Which havethe support of clientele and advisory groups?

5 Involvestakeholders people who have an interest in programresults in planning and decision making G. Diem, , Program Leader in Educational DesignFor a comprehensive list of our publications visit sheetFS869*From Up the Hierarchy, by Claude Bennett, Specialist, Educational Methodology and Evaluation, Staff Development, ExtensionService-USDA, from Journal of Extension: March/April, 1975, pp. Determine specific, measurableobjectives for the Programs that your objectives are measurable and attain-able. If you are unable to list your objectives in writing,you are probably not ready to plan or deliver the objectives state what will happen.

6 However, Educational objectives are preferable - they state what theprogram participant or target audience will do, learn, orgain as a result of the program. Objectives will reflect themany different levels of program outcomes that might besought. Below is Bennett s Hierarchy,* which depicts therange of outcomes that might be desired in programdelivery: End Results Practice Change KASA (Knowledge, Attitudes, Skills,Aspirations) Change Reactions People Involvement Activities InputsNote that true Impact increases as you go up the the cost of seeking these higher outcomes is that theyare often more difficult to measure or require a longer timeto do so.

7 An additional challenge of proving end resultsis making a feasible connection between a program of-fered and the results realized. Program planners must beable to answer the question: How do you know thisprogram was responsible for these impacts? Below are examples of objectives for each of the levels ofthe hierarchy, starting from the bottom. Before you startconducting a program, analyze your listed objectives tomake sure they are attainable and measurable, and at whatlevel they are likely to yield Impact . If your objectivesdon t go beyond the Reactions level, then your results areunlikely to time, funds, staff invested A budget of $2,000 will be allocated.

8 One hundred staff hours will be dedicated. A two-year study surveying 384 citizens will be events, activities, Programs , sessions offered A six-week beef production course will be offered. 4-H Summer Camp is scheduled for July 5 10. A home-study course on financial management will bepresented. A subscription-based newsletter will be offered tointerested commercial Involvement number of participants involved Two hundred welfare recipients will be enrolled. Two-thirds of Chamber of Commerce members willattend.

9 All elected officials from county government willparticipate. Enrollment at 4-H Summer Camp will be doubledwithin two years. Twenty volunteers will be trained to assist with pro-gram what participants thought of the program,its organization, its leader, etc. Seventy-five percent of workshop attendees will ratethe program as Very Good or Excellent. The instructor will attain an average Teacher Instructorrating of at least 8 on a scale of 1 10. Ninety percent of conference attendees will agree thatthey would recommend it to others.

10 Seventy-five percent of participants will be satisfiedwith meeting facilities, food service, and (Knowledge, Attitudes, Skills, Aspirations)Change Two-thirds of farmers attending will learn how to applyherbicides properly. Ninety percent of citizens participating in the votereducation course will report being likelier to vote in thenext election. Seventy percent of fourth grade students will be moreinterested in science careers. Sixty percent of home gardeners will understand thevalue of composting lawn clippings.