Contents
|
Interpreting the lack of connection between the organizational change
interventions and the outcome indicators
The OSU innovations are associated with impressive positive changes with
respect to integration, scholarship, and flexibility. It would be reasonable
to hypothesize that these changes would be sufficient to generate corresponding
positive shifts in the study’s outcome measures. This was not the
case; although most of the outcome measures received relatively high ratings,
they showed small and statistically insignificant changes in their ratings
for 1995 vs. 2001. There are several plausible interpretations for this.
- (1) Factors that are useful or necessary but not sufficient
-
The interventions address factors which might be valuable and perhaps
even necessary for improvement in the outcome measures, but might
not be sufficient to achieve a positive change in any particular one
of the outcome measures. For example, a closer integration of Extension
professionals with the research base found in academic departments
may provide a useful degree of access to an essential program input
(knowledge), but easy access to the relevant knowledge base is only
one component of outstanding Extension programming.
- (2) Countervailing variables
-
The organizational changes at Oregon State were not a controlled
experiment. Much has changed within the University, the state of Oregon,
the nation, and the world between 1995 and mid-2001. For example,
major restructuring and globalization within agriculture and forestry
have presented major challenges to those sectors of the Oregon economy,
and to the Extension professionals who work with those sectors. That
there is “No change” in an outcome indicator could be the
net result of positive forces (such as those that the interventions
have contributed) and negative forces impinging from the broader context.
- (3) “Resolution” or “granularity” of analysis
-
The interventions may have positive impacts on specific programming
areas, whereas the questions about outcome addressed the overall Extension
effort. There may be positive outcomes that are not apparent when
the respondents are asked to give average ratings for the whole organization
and its programming overall. This interpretation finds some support
in the breakout findings by program area: there were significant differences
in the Average Change for a key outcome item (3b: Extension’s
success in making a worthwhile contribution to the quality of life
in Oregon).
- (4) Lack of “headroom” or regression toward the mean
-
The 1995 average ratings on the outcome measures were already relatively
high – perhaps toward the upper end of the range of ratings which
the respondents, collectively, were willing to attribute to any item.
- Five of the six items (all but “financial well-being”)
had average 1995 ratings that were above the mean of means for all
29 items.
- One of the outcome measures had the highest 1995 average rating
of any of the 29 items – the item most closely associated with
the overall purpose for the interventions (to better address the
needs of the people of Oregon), namely “Extension’s success
in making a worthwhile contribution to the quality of life in Oregon.”
Of these four possible explanations for the lack of a significant shift
in most of the outcome measures, the first two – “necessary
but not sufficient” and “countervailing variables” seem
indisputably pertinent. The “resolution” or “granularity”
rationale is possible but uncertain. “Lack of headroom” is the
weakest of the four: none of outcome indicators had a mean above 5.6 on
a seven-point scale, so there is a potential headroom of at least 1.4
points on the scale. The statistical notion of regression toward the mean
is a more meaningful concern: any further increase of an already-high
score is, in a sense, “swimming upstream” against the tendency
for a second rating (for “NOW”) to revert to something closer
to the “mean of means” (about 4.7).
The four possible explanations above do not change the reality that ongoing
positive changes in the outcome measures would be desirable. The Recommendations
section that follows focuses on how to sustain the changes that have been
achieved and how to achieve broader changes.
The Recommendations section of this report also
proposes several approaches to achieving sustainable improvements at the
outcome level.
|
