Evaluation Design
Review Exhibit 2.1 (Evaluation
Design: Company-Based Wellness Center) on pp. 41-42.
Discuss it in terms of chapters 1-2.
Although
it's a little challenging to understand the scope of the wellness center
program without all the details, one of the first things that struck me about
the evaluation summary was the program objectives. Main program goals of
reduced depression and on-the-job productivity seemed a little unusual for a
(presumably "extracurricular") company-based wellness center. I
couldn't help but wonder why those were chosen (above others) to be big goals
of the program, and I wondered if they had been articulated to the staff as
well. (And, if so, what they thought of them.) That being said, the
evaluation questions fit the stated program goals well, and at the end of the
day the role of a good evaluator is to frame guiding questions based on stated
(vs. perceived) program objectives.
Another potential issue
is the timeline that is implied in the program goals. Twelve months may not be
enough to conclusively determine program "impact", however, may be
too long to give credence to shorter-term outcomes (unless those kind of
formative benchmarks are built into the evaluation process, which is difficult
to tell from this chart). Equally, one might deem this program effective
because "95% of participants will have improved cardiovascular
endurance", but could that be a case of "focusing on trivia"?
(i.e. "the processes are effective but have little or no impact within the
work environment" (p. 36.)
A limitation that might
skew results of this evaluation is wide disclosure which leads to scrutiny and
criticism. As Chapter 2 explained, people do not care to reveal their
shortcomings, and this might lead them to behave differently during the
evaluation or to provide inaccurate responses. This is especially significant
due to the sensitive nature of the activities and data sources in the wellness
evaluation. Reflective journals, psychologist reports, physical examinations,
and personnel evaluations are extremely private (and potentially
stress-inducing) documents. An employee would rightfully feel quite reluctant
to divulge such intimate information to an employer, as there may be other
risks in doing this (which are tied to the social and political atmosphere of
the company). Therefore, if an employee knew they were part of the
evaluation, they might lie on surveys or in meetings to avoid any kind of
negative impressions or feelings of failure/humiliation. I think the evaluation of the wellness center is
a great example of the need for "involvement of the consumers during the
formation of the evaluation design and their input into the evaluation
questions [to] help ensure that the results will be used" (p. 37). Dealing
with sensitive data could otherwise lead to significant evaluation limitations.
Reply to peer on discussion board about anticipated and unanticipated outcomes when evaluating:
I agree with all of your thoughts on the "pre-ordained" change idea. I feel like, in general, many of us (maybe it's a societal thing) lack the patience that is required to make judicious steps toward change. I know that I, perhaps like you, feel that "thinking through" a decision is sufficient to merit action. We already kind of know what we want, and it's all a matter of rationalization. We make things FIT our vision of change.
Reply to peer on discussion board about anticipated and unanticipated outcomes when evaluating:
I agree with all of your thoughts on the "pre-ordained" change idea. I feel like, in general, many of us (maybe it's a societal thing) lack the patience that is required to make judicious steps toward change. I know that I, perhaps like you, feel that "thinking through" a decision is sufficient to merit action. We already kind of know what we want, and it's all a matter of rationalization. We make things FIT our vision of change.
Systematic evaluation is cumbersome and tedious, but methodical, fair, and forward-thinking. It offers a chance to view perspectives or angles of a problem/scenario that may have been hidden before, and it makes data objective versus opinionated, emotional, or tied to "the loudest voice in the crowd". Hopefully as we come to learn more about "formal" evaluation processes, we can find points of compromise where our impatient desire for immediate change or "the next best thing" is tempered with a systematic review of data. Maybe there are some viable shortcuts for more here-and-now decision-making!
Discussing Chapter Two as a whole:
The section "Benefits to Sponsors and Staff" stood out to me because of its optimistic promise of collaboration and group reflection. Ideally, in an educational setting, teachers would embrace objective evaluation activities because they would back up their "inklings" about a program's value, effectiveness, and impact. It would be a reason to gather with colleagues to discuss the program and its processes as well as the "evaluation's philosophy, its procedures, and its anticipated outcomes" (p. 34). In other words it would provide an staging ground for staff to collaboratively stop and reflect on their program's design and activities in a natural, conversational way. It could rekindle feelings of pride, ownership, and advocacy. It could also lead to building a framework for colleagues to jointly select and organize "standards, indicators, evidence, and resources". Evaluation serving as impetus for discussion, colleague appreciation, and program scrutiny is huge benefit in its own right... not to mention the hard data and "next steps" that will ultimately come from it.
Discussing Chapter Two as a whole:
The section "Benefits to Sponsors and Staff" stood out to me because of its optimistic promise of collaboration and group reflection. Ideally, in an educational setting, teachers would embrace objective evaluation activities because they would back up their "inklings" about a program's value, effectiveness, and impact. It would be a reason to gather with colleagues to discuss the program and its processes as well as the "evaluation's philosophy, its procedures, and its anticipated outcomes" (p. 34). In other words it would provide an staging ground for staff to collaboratively stop and reflect on their program's design and activities in a natural, conversational way. It could rekindle feelings of pride, ownership, and advocacy. It could also lead to building a framework for colleagues to jointly select and organize "standards, indicators, evidence, and resources". Evaluation serving as impetus for discussion, colleague appreciation, and program scrutiny is huge benefit in its own right... not to mention the hard data and "next steps" that will ultimately come from it.
No comments:
Post a Comment