Third Mission Innovations

The Questionnaire

Contents


A single questionnaire was used in an ex post facto design: respondents were asked to respond to most questions twice—once for “NOW” (mid-2001) and once for 1995 (“THEN”). The “treatment” to be assessed—the third mission innovations—took place in the intervening six years, beginning in 1995. Although there is no plausible control group, different sub-groups among the respondents (by program area, location, and administrative role) experienced somewhat different versions of the “treatment” and therefore provided an opportunity to test the consistency of results across groups and across treatment variations. Questionnaire text

Because of the lack of a control group and because of the non-random assignment of individuals to the “treatment variations,” this is properly understood as a quasi-experimental design. The primary precaution associated with this is that where between-group differences are found – for example, between two program areas – it is impossible to be certain whether they are the result of treatment differences or characteristics of the individuals. Findings are presented with this in mind.

Description of questionnaire sample and respondents

OSU Extension professionals were treated as the appropriate population to address the variety of questions selected for inclusion in the questionnaire. For some questionnaire items, the respondents report on their own attitudes; for other items, they are serving as an “expert panel” to assess the organization’s performance and impacts.

The questionnaire was distributed by email on July 27, 2001. In order to maximize the response, it was distributed to a 100 percent sample of county Extension faculty, campus-based Extension faculty, Extension program leaders, and also OSU department heads with Extension responsibilities. This was done by means of existing email lists that are actively maintained by OSU Extension.

The questionnaire was returned by approximately 56% of the potential respondents. The rate of response was similar across all breakout groups: breakouts by program area, location (county/campus/other), hire decade, and administrative role all produced no group with a response rate lower than 48%. The response rate – overall and within each breakout group – is adequate for the intended analyses. Response details

Respondents

The respondents are representative of all program areas in similar proportions to the overall staff mix

Respondents by Program Area (Data table)

The Agriculture program area is a near-majority of the respondents, as it is of the total Extension faculty. Because of this preponderance of respondents from the Ag program area, averages for questions that specify “… in my program area …” (questionnaire items 4a-f and 5a-k) are weighted toward agriculture. See program area breakouts for each questionnaire theme (in the Results by Theme section) for a complete understanding of the results.

County staff chairs, department heads with Extension responsibilities, and Extension program leaders make up 20% of the staff and 22% of the respondents.

(Data table)

Interpreting the Results

To fully understand the results of the questionnaire, the following information will be useful. Items 4a - 4f and 5a - 5k refer to the respondent’s program area. Items 3a - 3f and 6a - 6f refer to OSU Extension overall.

Twenty-nine of the questionnaire’s items (3a - 6f) solicit two ratings, one for “THEN” and one for “NOW.”

  • “NOW” means mid-2001
  • “THEN” means 1995

However, 76 of the respondents (40%) were hired since 1/1/95. Special instructions in the questionnaire gave them two options.

  1. Skip the “THEN” rating if they felt incapable of giving a response that would be distinct from their “NOW” rating. On average, across the 29 “THEN” ratings, about 37% of recently hired respondents skipped the “THEN” response.
  2. Provide a “THEN” rating, using their date of hire as “THEN.” On average, across the 29 THEN ratings, about 63% of recently hired respondents did that.

“Average Change, 1995-2001” is the mean difference between “THEN” and “NOW” ratings for all respondents who provided both ratings for the item. For most of the questionnaire items, the “Average Change, 1995-2001” is trivially different from the difference of the NOW mean minus the THEN mean—i.e., not excluding missing scores. For the items that do show a meaningful discrepancy between these two approaches, the shift over time as indicated by the Average Change is less positive or more negative than the NOW-minus-THEN approach. Since earlier-hired Extension professionals constitute a greater proportion of the Average Change responses (which exclude some newer hires), the discrepancies usually reflect a more conservative assessment of the “NOW” situation on the part of earlier hires.


previous page

next page