Evaluation Models
Here is the quiz (from assignment above) that I generated to help others review the content from Chapter 5, aiding in their understanding of the differences and similarities between various evaluation models.
Metaphorical Story: Recently, I was talking with three exceptional (special) education teachers at a technology conference. These three colleagues described their classrooms to me. They invited me to visit. So I did. I went into several classrooms. I approached one of the teachers and asked, "What are you doing?" "I'm teaching reading," he replied. Then I asked another teacher, "What are you doing?" "I'm showing these students how to have good study skills," she said. Then, I asked the third teacher, "What are you doing?" The woman put down her pen and said, "I'm helping all my students achieve their maximum potential in academics and social skills so that when they go out into the world they will be magnificent contributors."
Now, all three of these teachers had the same job, but only the last teacher had vision. She could see beyond the daily grind of teaching and see her students contributing mightily to our society. In our lives and in our jobs, sometimes it's hard for us to stay focused on the larger vision, to rise above the mundane, above the day-to-day.
In history, special people had that vision, one that has benefited us all. In my own work, I, too, sometimes get caught up in the details of the the daily grind. I go to meetings, read reports, and talk to colleagues. But there are times when the big picture is as clear as day, when I feel truly connected to issues and ideas much larger than myself, larger than any job, larger than any single organization.
How is this story related to EDTECH 505 and, more specifically, to the readings for this week? Do you have to have vision to be successful evaluator? How does vision fit with choosing the most appropriate evaluation model for a particular program?
My response: When approached, the three teachers in this story demonstrated that they were in full project implementation mode. They were in the process of putting program goals in action, whether teaching reading skills, study skills, or general academic/social skills. Nevertheless, the third teacher was the only one to use the phrase "so that..." This is what makes her stand out as focusing on the program's (in this case, education in general's) impact. My students and I do X so that Y. There is a clear cause and effect relationship implied in current activities and future behaviors. Focusing on the goal (the reason for instruction) makes teacher #3's job more efficient and effective from a program implementation standpoint. There is a sense of forward progress and being part of a "bigger picture" program cycle. The teacher's actions are not the endpoint.
That being said, I'm not convinced that this inherently makes teacher #3 the better educator. While it's good practice to articulate program objectives (as they align to program goals as well as activities), the fact that these two stakeholders (teachers #1&2) did not speak these impact objectives out loud does not mean they are not working towards them. Perhaps the same "so that..." statement (as teacher #3) guides their lesson planning, assessment design, and conversations with their students. We would have to talk more with these two teachers to get to know their motivations before assuming an absence of vision. The same is true for a program evaluator -- we need to get to know all stakeholders, their beliefs and understanding of the program, before making conclusions based on observations of their behavior or brief one-off conversations.
Another point that interested me in this story was the fact that teacher #3's reported action was the most general/generic of the three, and therefore probably the hardest to evaluate. "Helping students achieve their maximum potential in academics and social skills" is a lofty and broad program goal that may need to be broken into more tangible sub-goals or objectives. In order to measure and draw conclusions about teacher #3's program effectiveness, an evaluator would have to define and tease out evaluation questions regarding "maximum potential", "academic skills", "social skills", and "magnificent contributors". Yes, this teacher has great vision (which is of supreme importance in program development and evaluation), but the vision cannot stand alone without tangible activities and measurable outcomes.
Reply #2: In my opinion, being a visionary and having vision are quite similar. Teacher #3 has demonstrated this personality trait ("being a visionary" perhaps being more perpetual and noticeable than "having vision"). Teacher #3 is motivated by his/her drive to create impact. They see the bigger picture and the purpose behind painting the picture. Having vision often accompanied by passion and an ability to motivate others to action. The evaluator's role is to help them look critically at the various components of this grandiose picture-painting exercise and determine if they are efficient and effective in leading to the desired impact (change in behavior/attitude/mental structure, in the case of education). An evaluator's role is to help communicate the vision in objective, easy-to-understand ways. An evaluator must also communicate the success of the current plan to attain this vision.
While it's terrific to be a visionary (with wisdom and creative plans for the future), it's not a precursor to having a good program plan. Having A vision is a great start, and is necessary if one wishes to have meaning/purpose in their actions (their career?). Even if there is only one single objective -- one carrot at the end of the stick -- there is a sense of achievement that propels these teachers forward. There is a program cycle in place and in perpetual motion. They might not have vision in other areas of their job, but for the purposes of one class or program, having a vision means there is something to work towards and something to evaluate and reflect upon at the end.
To be honest, I'm not sure if an evaluator really needs to be visionary or have vision. From my understanding and perspective, they are more in charge of restating, recording, and analyzing the various components of someone else's "vision plan". As on objective bystander, they aren't creating or changing the objectives, they are reporting on their fulfillment. Yes, they need to "have a vision" of what's all going on, but they don't necessarily need to be active in determining what the "big picture" (goals) look like.
Response to a peer: Derick, I must admit I feel the same way as you. To me an evaluator's role is not really to have a vision. Yes, they must have a wide scope of what's going on and a "vision" of program goals, activities, and measurements. However, this is different from the vision of the program in the first place. This kind of broader, step-way-back vision embodies the goals and beliefs that led the group to adopt a certain program in the first place. Like you say, as an outside entity, It's not really the evaluator's place to change that vision. They are working with more deductive reasoning, whereas the organization is working more open, generative, creative (not sure if it's quite "inductive") thinking.
This goes back to a previous week's discussion of the AEA, where I was very impressed but a little confused by the vision and value statements. Where does this kind of "change-causing" fit into their job? Isn't it against good evaluation practice to include measuring sticks and suggestions beyond the stated program goals and expected outcomes? I'm all for evaluators being visionaries, but how would that actually work?
No comments:
Post a Comment