Environmental Education Workshop Series: Evaluation
The San Diego STEM Ecosystem’s Environmental Education Working Group hosted their second professional development workshop on Thursday, August 23, at the Living Coast Discovery Center. This workshop centered around evaluation and recommended approaches to measuring processes and successes.
Jim Marshall, Ph.D., from the Department of Education Leadership at San Diego State University, presented on various tactics of evaluation, from the early planning stages of a program to its completion. He reminded the group—made up of over 20 Environmental Education professionals—to build data collection into programs from the start and to focus on the outcomes to frame the evaluation.
A key framework that was highlighted throughout the workshop was the Kirkpatrick Model. Developed by Dr. Donald Kirkpatrick in the 1950s, this model provides a framework that can be used for analyzing and evaluating the results of educational and training programs which ultimately increase the effectiveness and impact of an organization’s program.
The Kirkpatrick Model can be implemented before, during and after a program or training and utilizes the following four levels as frames for evaluation questions:
Level 1: Reaction
The first level measures the participants’ engagement and whether they found the content relevant or favorable in their own jobs. Did they attend? Did they like it? Will they use or apply the training?
Level 2: Learning
Level 2 analyzes the participants to gauge how much they gained from their participation, whether that is in the form of an increase in knowledge, skills, attitude, commitment, etc. Did they learn the content? Can they perform the skill? Do they show new skills and knowledge?
Level 3: Behavior
The Behavior level involves looking at the degree to which participants apply what they learned during the training at work. Does it get used? Are they doing it consistently and appropriately? Are they successful?
Level 4: Results
The last level involves determining whether or not the targeted outcomes occurred and whether the material had a positive impact on the participant’s organization. Does it advance the organizational strategy or mission? Do the outcomes match the investment? Is there return on investment?
The evaluation process may be demanding and time-consuming from time to time, but understanding that there are resources such as the Kirkpatrick Model may be beneficial for your organization in the long run. Analyzing the measurements and outcomes of the training/program for your organization is a great start, but determining what actions and changes as a result of the evaluation process is where improvements are likely to shine.
A big thank you to The Living Coast Discovery Center for hosting the Evaluation Workshop!