|
Session Title: Program Evaluation and "Scientific" Research: One Community's Serendipitously Successful Approach Via Character Education - Nixus
|
|
Panel Session 599 to be held in Suwannee 14 on Friday, Nov 13, 3:35 PM to 4:20 PM
|
|
Sponsored by the Pre-K - 12 Educational Evaluation TIG
|
| Chair(s): |
| David Hough, Missouri State University, davidhough@missouristate.edu
|
| Abstract:
Presented here are findings from the evaluation of a United States Department of Education (USDE) - funded character education program that examined the program's impact on school climate, student behavior, attendance, and achievement. Forty-seven schools in a large, county-wide district in the Deep South participated in this four-year quasi-experimental wait-list control design, along with university personnel and five community organizations. The multi-agency character education model implemented is described in detail, along with the development of a School Climate Index (tm) and observation protocols used by the evaluation team. Qualitative and quantitative data document a number of positive, negative, and problematic issues associated with program integrity, fidelity of data collection, and reporting. Many character education activities were found to be associated with improved school climate and student outcomes, findings evaluators were able to report while advancing sound evaluation methods combined with an "evidence-based" research design.
|
|
Quantitative Methods in Support of Program Evaluation and "Scientific" Research: An Example of Meeting All Stakeholders' Needs in a Character Education Program
|
| Vicki Schmitt, University of Alabama, vschmitt@bamaed.ua.edu
|
|
This presentation describes quantitative methods used to examine a four-year, grant-funded character education project designed as a quasi-experimental, wait-list control study. Data collection methods are described, and findings are presented along with detailed explanations of how evaluation theory and practice can coincide with randomization and controls imposed by "evidence-based" research, per USDE mandates. While the character education program examined herein was required to employ a "rigorous, evidence-based, scientific evaluation," both PI's and evaluators were able to find a nexus between evaluation and research that met school, community, and USDE requests and requirements. This approach resulted in program improvement, refinement, documented outcomes, and scientific evidence of value and use to all stakeholders.
|
|
|
Qualitative Methods Used to Inform, Improve, and Address Program Integrity in One Community's Character Education Initiatives
|
| David Hough, Missouri State University, davidhough@missouristate.edu
|
|
While opinions differ as to where the line between evaluation and so-called "scientific" research should be drawn, the integrity of program implementation remains a subject of interest and concern to both. This presentation describes a number of qualitative methods used to examine the level and degree of implementation of a character education program in a county-wide community located in the Deep South. In addition to focus groups and personal interviews, 547 separate field notes written by ten different data collection "coaches," over a four-year time frame were collected and content analyzed. In addition to presenting findings, this presentation focuses on the positive, negative, and problematic issues associated with this data collection approach. Recommendations for systemic improvements and rigorous protocols in qualitative evaluation and research are offered.
| |