|
Session Title: Determining What is Critical in the Program Evaluation of A Multi-Site College Access Program
|
|
Panel Session 417 to be held in the Quartz Room Section A on Thursday, Nov 6, 4:30 PM to 6:00 PM
|
|
Sponsored by the College Access Programs TIG
|
| Chair(s): |
| Imelda Castaneda-Emenaker,
University of Cincinnati,
castania@ucmail.uc.edu
|
| Discussant(s):
|
| Jennifer E Williams,
JE Williams and Associates LLC,
jew722@zoomtown.com
|
| Abstract:
This panel illustrates how a rigorous program evaluation can be built upon the conceptualization of key constructs for particular components of a college access program called Gaining Early Awareness and Readiness for Undergraduate Program (GEAR UP). The presenters are all part of the evaluation team for Ohio's multi-site GEAR UP program. The panel shares how the identification of relevant evaluative criteria and determination of their importance can lead to meaningful program expansions, strong support of stakeholders, and program sustainability. We discuss the development and application of evaluative criteria then narrow the focus onto one important construct: school climate as condition for students' success. Presentations feature lessons learned along with the theoretical foundations, process of merit determination, and results that contributed to the utility of the findings. Tentative suggestions for changes in local, state, and federal policies are discussed.
|
|
Upping the Ante: Establishing Focused Evaluative Criteria for a Program Evaluation of a College-Access Program
|
| Janet Matulis,
University of Cincinnati,
janet.matulis@uc.edu
|
|
This presentation focuses on the use of evaluative criteria driving the program evaluation of a college access program. Program goals are typically used as benchmarks for evaluation purposes; however, our experience as the evaluation team of Ohio's multi-site Gaining Early Awareness and Readiness for Undergraduate Program (GEAR UP) is consistent with Davidson's (2005) claims regarding the importance of establishing focused evaluative criteria. This develops a systematic process for honing in on valid conclusions about program performance. This presentation discusses how the Ohio GEAR UP evaluation has become more purposive by focusing on the underlying constructs and evaluative criteria used in our surveys of teachers, students, and parents. Evaluative criteria drove the revisions to GEAR UP's surveys that further clarified constructs for different components considered for the success of the program. Policy implications of this process are explored.
|
|
|
School Climate and Student Achievement: Theoretical Considerations in Evaluation
|
| Hsin-Ling Hung,
University of Cincinnati,
hsonya@gmail.com
|
|
This presentation centers on theoretical concerns related to one of the key constructs in the program evaluation of Ohio's multi-site Gaining Early Awareness and Readiness for Undergraduate Program (GEAR UP): the effects of school climate on student achievement. Our evaluation team's concept is grounded in the view that school climate is the total environment quality within the school building (Anderson, 1982) and that school climate defines the school level learning environment (Johnson & Stevens, 2006). Tagiuri's (1968) taxonomy of climate is presented followed by his three school climate theories. Variables in Tagiuri's model of climate and research on the relationship between school climate and student achievement are examined. Lessons learned from the theories and researches about the assessment of school climate as an important influence on student academic success are addressed. Policy implications related to school climate, student achievement, and college access are discussed.
| |
|
Merit Determination in Evaluating Conditions for Students' Success in a College Access Program
|
| Imelda Castaneda-Emenaker,
University of Cincinnati,
castania@ucmail.uc.edu
|
|
In this panel, the presenter shares the perspectives of an experienced evaluator of Ohio's Gaining Early Awareness and Readiness for Undergraduate Program (GEAR UP). The presentation focuses the merit determination processes used in the design of a major data collection instrument for this multi-site college access state program: the surveys for teachers, students, and parents. Although the surveys are meant to gather data for all program components, the school climate as condition for students' success is highlighted as an example of one of the major prioritized constructs. The presentation illustrates the use of some strategies posited by Davidson (2005) in determining the importance of this particular construct drawing on the knowledge of key stakeholders, using evidence from literature, and using program theory. This process ensures that data of significance to the project stakeholders are gathered, analyzed, and used for appropriate program improvements and reports for accountability. Policy implications are discussed.
| |
|
An Illustration of Utility of Evaluation Findings: Addressing Teachers' Perceptions
|
| Jerry Jordan,
University of Cincinnati,
jerry.jordan@uc.edu
|
|
As part of the multi-site Ohio Gaining Early Awareness and Readiness for Undergraduate Program (GEAR UP) evaluation team, this presenter focuses on the characteristics of school climate brought to light as a result of the team's evaluative process. The merit determination phase of our evaluation highlighted student expectations as central to school climate. Our measures discerned a gap between teachers' perceptions of student college-going potential and the students' own perceptions of their college-going potential. This finding helped the program focus ongoing program objectives and allowed individual sites to target changes in teachers' perceptions as one short term goal. This presentation highlights how the program has adjusted to these findings. Finally, this presentation addresses the difficult issue of how policy makers might influence teacher perceptions and expectations.
| |