|
Session Title: Evaluation as a Learning Tool: Maximizing Outcomes Using Strategic Formative Evaluation
|
|
Panel Session 659 to be held in Liberty Ballroom Section A on Friday, November 9, 4:30 PM to 6:00 PM
|
|
Sponsored by the Organizational Learning and Evaluation Capacity Building TIG
|
| Chair(s): |
| Linda Thurston,
Kansas State University,
lpt@ksu.edu
|
| Discussant(s):
|
| Jan Middendorf,
Kansas State University,
jmiddend@ksu.edu
|
| Abstract:
A vital aspect of evaluation work within higher education is to assist academic programs and externally funded projects develop successful programs and/or continuously improve program outcomes. The Office of Educational Innovation and Evaluation at Kansas State University utilizes formative evaluation to provide feedback to program personnel as they focus on program development and improvement. To provide the most focused and useful information for program development and improvement, we use strategic formative evaluation practices which focus on four basic practices: understanding clients' long-term expected outcomes; understanding clients' intended use of the data we collect; asking the right evaluation questions; and reporting our findings in a usable form. This panel will provide case studies that describe these strategic formative evaluation practices with several projects. A discussant will make recommendations for future research and practice.
|
|
Impact of Formative Evaluation on Service Learning Projects to Restore Water Quality in Kansas
|
| Christa Smith,
Kansas State University,
christa2@ksu.edu
|
| Bill Hargrove,
Kansas State University,
bhargrove@ksu.edu
|
| Christopher Lavergne,
WaterLINK,
lavergne@ksu.edu
|
|
The goal of WaterLINK is to engage Kansas' colleges with local communities as partners in water quality restoration and protection through service learning, with the ultimate goal of improving water quality in high priority watersheds. The client's primary goal was to implement 20 to 30 service-learning projects and maintain or increase that number on an annual basis. The process of establishing the WaterLINK project in campuses and communities proved to be more challenging than expected. Formative evaluation strategies that helped to identify these challenges and provided feedback on successes facilitated the client's goal. Evaluators focused on project impact on participants through administering Web-based surveys to students and community partners, interviewing faculty, and conducting site visits. Providing formative reporting with recommendations for project improvement to WaterLINK stakeholders helped project directors improve and maintain their project.
|
|
|
Informing the Development of Graduate Coursework Through Formative Evaluation
|
| Jennifer McGee,
Kansas State University,
jemcgee@ksu.edu
|
| Amy Conner,
Kansas State University,
amcabe@ksu.edu
|
| Marsha Dickson,
University of Delaware,
quattro.oet.udel.edu
|
|
Concern about socially responsible business practices have increased, in part due to the media attention of worst practices such as sweatshop conditions in the apparel industry. Apparel businesses and their relevant stakeholders need a shared framework upon which to base socially responsible solutions. The Social Responsibility in Textile, Apparel, and Footwear Industry project contracted us to help with developing a conceptual framework and relevant definitions for the foundation of Internet-based graduate courses providing competencies related socially responsible textile, apparel, and footwear industry supply chains. Multiple formative evaluation activities shaped this first phase of the project, including:
(1) industry expert interviews to produce definitions and framework; and
(2) on-line reviews of course syllabi by industry experts to evaluate the appropriateness of course objectives.
This paper will discuss the multi-layered approach we designed to provide current, relevant information to form the foundation for graduate preparation of personnel for the industry.
| |
|
The Evolution of Formative Evaluation for a Statewide Multi-year Initiative
|
| Cindy Shuman,
Kansas State University,
cshuman@ksu.edu
|
| Jan Middendorf,
Kansas State University,
jmiddend@ksu.edu
|
| Cindi Dunn,
Kansas State University,
ckdunn@ksu.edu
|
|
As the evaluation unit for Kansas' statewide broadband initiative, Kan-ed, for the last four years, OEIE has provided formative evaluation feedback to the network. Given the scope of the initiative, the evaluation uses a variety of methodologies, both quantitative and qualitative, to document and report results. Over time, as the implementation has moved into different phases, our evaluation procedures have also had to evolve in order to be responsive to the needs of the director, staff, and stakeholders, including the state legislature. The focus of this case study will be on how the evaluation team has continually assessed how the client will use the data. The presentation will also discuss the system OEIE has developed to report timely, relevant and useful information to identify and address effective strategies and challenges during the implementation of the initiative.
| |
|
Learning From Evaluation: Capacity Building in a Geoscience Education Project
|
| Sheryl Hodge,
Kansas State University,
shodge@ksu.edu
|
| Iris M Totten,
Kansas State University,
itotten@ksu.edu
|
|
Clients who were leaders of a National Science Foundation - funded geology education project used OEIE services to evaluate changes in students' knowledge, skill, and attitude that resulted from a geoscience digital tutorial. Although the formative evaluation provided valuable information for the development of this innovative curriculum project, the most significant learning outcome was considerable capacity building outcome that emerged throughout the collaborative process. The evaluation team listened to the PIs explain their methodology and expected outcomes and melded this information with what was presented in the original funded proposal, thereby operationalizing the 'what' that the PIs were hoping to achieve in valid, reliable, and measurable means. The result of this relationship not only fostered improvements in data collection tools and administrations, but it also provided comprehensible strategic formative feedback from complex analyses that presented new and different paths for the PIs to consider in their successive research endeavors.
| |