- About Us
- Statewide Resources
- Get Involved
- For Employees
- Find Us
Evaluation is an Everyday Activity
Program Evaluation Discussions
Updated: 9 hours 50 min ago
I was reading Harold Jarche’s blog, Perpetual Beta and he is talking about the interface of the digital and analog worlds; he is talking about connections. I find that that concept applies to evaluators. Here’s how.
I was trained as an evaluator in the 1980s; we didn’t have access to the web, internet, email, FB, or many of the other high technology options available today. I did an NDE (wasn’t called that then) volume (Vol. 62) which was all done by hard copy and the USPS (a tedious and laborious process). I just completed another NDE (now called that) volume (Vol. 145) that was done electronically (no longer tedious, though still a laborious process). This last volume was quick. Although most of the authors entered the field after I did, my co-editor didn’t; he entered before I did. These authors had the luxury of electronics that we didn’t have. I have had to learn how to use electronics; I know my co-editor did, too. (I remember saying to myself and my colleagues, what will my secretary [yes, we used that title] do now that I’m composing on a key-board?) Now I do all my composing and other things on a keyboard; most of my work is augmented with electronics (i.e., the digital world). But I am truly a digital immigrant; learning how to use this new technology, to be in this digital world, is hard for and on me. (My children laugh at me and get exasperated; it is so simple to them.)
Today’s evaluators are highly connected, largely due to the electronic capabilities. Yet there is still evaluator isolation. Makes me wonder if evaluators really connected? Sheila Robinson (the only evaluator in her organization bemoans this fact here). She advocates for connections through EvalTalk and AEA’s LinkedIn account. I still see these as digital, albeit, opportunities to connect. Social media is also mentioned for connections. Still I wonder–are you really connected? With colleagues scattered around the world, this may truly be the only way to stay connected today. Letters and phone calls are truly analog and perhaps passe. Still they are appreciated and sometimes welcome as the only way to connect. What will this world look like if the only way to connect is via digital?
Jarach advocates changing the way we organize. To me that talks about changing the way we work. Maybe evaluators will work in isolation. In order to “see as many possible roads ahead”, perhaps we need to “work in self-managing networks”. “If those who are educated, knowledgeable, and experienced do not push for a better world of work, then who will? An effective knowledge network cultivates the diversity and autonomy of each worker. Knowledge networks function best when each person can choose with whom and when they connect. Solving problems together is becoming the real business challenge.” (From Jarach)
And that affects evaluators!
To whom are you connected? How do you connect? Are you caught in-between?
A reader made the comment that “blogging is like doing case studies”. Made me think about the similarities and differences. Since case study is a well known qualitative method used in evaluation with small samples, I think this view is valid.
Blogs are small samples of something observed in depth, with great scrutiny, and often with serious time commitment, although Brinkerhoff in his book, “The Success Case Method” advocates a case study method that is “valid, practical, fast, and credible”. I must provide the caveat that the book was developed for organizations which put out innovations and improvements (as in products, like Ford Motor Company, Gap International) not educational organizations (like universities or school districts), so a bit of translation will be needed.
There are many sources for case study research. Perhaps Bob Stake and Robert Yin are the best known in the evaluation field. Bob Stake’s first book (that I have on my shelf; there may be others that I don’t have) is “The Art of Case Study Research” . Google books says this about that volume, “This book presents a disciplined, qualitative exploration of case study methods by drawing from naturalistic, holistic, ethnographic, phenomenological and biographic research methods.” (And it is less than 200 pages.) Robert Yin has a fifth edition of the “Case Study Research” book available now from Sage .
After you have been blogging for a while (several months, for example), the blog will have categorized the posts. The tags will help determine what are the most relevant points. Use those tags and categories to help develop the “case”. You will find posts that form the case, more cases than you realized. So multiple posts on the same subject will provide you with multiple observations; one post may provide you with a slice of something in-depth, an individual case study, if you will.
Scriven, in the 4th edition of the Evaluation Thesaurus says this of case-study method: “The case study method is at the opposite end of one dimension of the spectrum of methods from the survey method, the micro end rather than the macro end.” Blogs are a microcosm of information as well. He goes on, “…observing…is more characteristic of case-study method than large-scale surveys…Causation is usually determined in case studies by the modus operandi method, rather than by comparison of an experimental with a control group.” He refers the reader to “Naturalistic” where he references Ernie House, Egon Guba, Yvonna Lincoln, Bob Stake, and Bob Wolf and states that naturalistic approaches “…stress contextual factors, unstructured interviewing, observation rather than testing…”
All this reminds me of blogging.
What do you think? Is blogging a form of case study? Let me know.
The website, mymande.org/evalyear, , has devoted an entire page to the announcement. They are calling it “EvalYear”. You are invited to join the global year by visiting the mymande website. This web site explains more about the international year of evaluation. Check it out.
The year becomes important when one advocates for evaluation, when one does evaluation, when one supports an evaluator, and/or when one is an evaluator.
What can you do to contribute to 2015; to make 2015 truly an evaluation year?