| Surveymonkey and Zoomerang: How Do You Choose Which One to Use? |
| Poster Presentation 139 to be held in Sebastian Section J on Wednesday, Nov 11, 6:30 PM to 8:00 PM |
| Presenter(s): |
| Lois Ritter, Walter R McDonald and Associates Inc, lritter@wrma.com |
| Lori Nascimento, The California Endowment, lnascimento@calendow.org |
| Abstract: SurveyMonkey and Zoomerang are two commonly used web-based online survey development tools, but few people know the attributes and differences between these two tools. In this poster, learners will be shown the key features, similarities, and differences between these programs. |
| Assessing Mass Sport and Sport Development Initiatives in Egypt |
| Poster Presentation 139 to be held in Sebastian Section J on Wednesday, Nov 11, 6:30 PM to 8:00 PM |
| Presenter(s): |
| David Waters, University of Wisconsin at La Crosse, waters.david@uwlax.edu |
| Sherin Elmahdy, University of Wisconsin at La Crosse, elmahdy.sher@uwlax.edu |
| Ezzeldin Aly, University of Wisconsin at La Crosse, aly.ezze@uwlax.edu |
| Abstract: Egypt has been long considered as one of the most attractive countries to visit because of its historical. People's needs mass sport with the Egyptian population up to75milion. Sport for all opportunities, events, and or organized programs for large groups including the general population ,participating in clubs , on play fields in gymnasiums , community centers . Enough for the Egyptian mass sport future. (ISMQ), created by Waters (1996). Was translated in Arabic then distributed to 90 Egyptian mass sport leaders at 28 city of Egypt, . Raters were asked to provide answers to questions (n=51) on the ISMQ, which ultimately resulted in scoring (Maximum=33 points) of eight factor components: objectives, legislation, organization, implementation, physical resources, human resources, research and evaluation, and contribution to nation building. Analysis of data will reveal descriptive statistics highlighting Egyptian mass sport that are effectively being developed and managed in the present, with recommendations. |
| Getting Ahead of the Course Evaluation Curve |
| Poster Presentation 139 to be held in Sebastian Section J on Wednesday, Nov 11, 6:30 PM to 8:00 PM |
| Presenter(s): |
| Brian Hopewell, Academic Management Systems Inc, bhopewell@academicmanagment.com |
| Abstract: If we believe students have valuable information to offer about what they observe and absorb from their courses and instructors, doesn't it makes good sense to ask about these things and pay attention to what's offered? Can this be done well electronically or are golf pencils and paper surveys the pinnacle of course evaluation technology? This poster will explore these issues, present the case for and against online survey delivery, and explore one online tool's features. |
| FRAMES: Using Images to Track Life Stories of Youngsters Under Social Exclusion |
| Poster Presentation 139 to be held in Sebastian Section J on Wednesday, Nov 11, 6:30 PM to 8:00 PM |
| Presenter(s): |
| Daniel Brandao, Instituto Fonte, daniel@fonte.org.br |
| Abstract: Frames is a method created in Brazil (2007) to investigate the life path of youngsters marked by involvement in crime. Its original application throughout the evaluation of an educational program revealed the possibility of using Frames with any youngster in a social exclusion context. The method was inspired by comic books and consists of 27 drawings that display situations that might occur in the lives of these youngsters. The drawings are represented by scenes that allow multiple interpretations and have the power of triggering a dialogue with the youngster. The goal of Frames was to break the institutionalized discourse of this population and reach real and in-depth information about their lives. Applications were made individually. All the drawings were presented to the youngsters as conversation began and using questions such as "What catches your eye?", among other questions. |
| Evaluating a Multi-level Capacity Building Program to Promote Evidence-based Teen Pregnancy, HIV and STI Prevention |
| Poster Presentation 139 to be held in Sebastian Section J on Wednesday, Nov 11, 6:30 PM to 8:00 PM |
| Presenter(s): |
| Catherine A Lesesne, Centers for Disease Control and Prevention, clesesne@cdc.gov |
| L Duane House, Centers for Disease Control and Prevention, lhouse1@cdc.gov |
| Alison Spitz, Centers for Disease Control and Prevention, aspitz@cdc.gov |
| Heather Tevendale, Centers for Disease Control and Prevention, htevendale@cdc.gov |
| Diane Green, Centers for Disease Control and Prevention, dcgreen1@cdc.gov |
| Abstract: With increasing attention to accountability, the government PART (OMB) assessments, and a desire to make optimal use of taxpayer dollars, government evaluators and their funded partners must develop evaluation methods and techniques appropriate to capacity building programs. Both internal and external evaluators are met with limitations in the science and evaluation practice as it relates to capacity building. This poster will demonstrate how the Centers for Disease Control and Prevention's Promoting Science-based Approaches (PSBA) program has approached evaluation of a primarily capacity building program in the area of teen pregnancy prevention (Lesesne et al, 2008). CDC is currently evaluating this multi-level, multi-site capacity building program and will share our methods, preliminary findings, and new strategies for monitoring capacity building programs. |
| Navigating the Information Collection Request Process in Government Evaluation: Perspectives From the United States Environmental Protection Agency |
| Poster Presentation 139 to be held in Sebastian Section J on Wednesday, Nov 11, 6:30 PM to 8:00 PM |
| Presenter(s): |
| Terell Lasane, United States Environmental Protection Agency, lasane.terell@epa.gov |
| Michelle Mandolia, United States Environmental Protection Agency, mandolia.michelle@epa.gov |
| Yvonne Watson, United States Environmental Protection Agency, watson.yvonne@epa.gov |
| Abstract: The Paperwork Reduction Act, administered by the Office of Management and Budget, requires that all statistically valid data collections be subjected to an Information Collection Request Review. All federal agencies using federal monies for data collection must navigate this protracted process that often results in the ultimate disapproval of the proposals submitted. As accountability demands are raised by government oversight bodies and as evaluation agencies throughout the federal government strive to build capacity for evaluations that meet ever broadening demands for methodological rigor, creative solutions must be developed in circumventing this formidable obstacle to program evaluation. US Environmental Protection Agency personnel will present their unique challenges with Information Collection Requests. |
| Evaluation of Comprehensive Cancer Control: National and State Perspectives Methods, Techniques and Tools |
| Poster Presentation 139 to be held in Sebastian Section J on Wednesday, Nov 11, 6:30 PM to 8:00 PM |
| Presenter(s): |
| Phyllis Rochester, Centers for Disease Control and Prevention, phyllis.rochester@cdc.hhs.gov |
| Deborah Porterfield, RTI International, dporterfield@rti.org |
| Angela Moore, Centers for Disease Control and Prevention, angela.moore@cdc.hhs.gov |
| May Yassine, Michigan Public Health Institute, myassine@mphi.org |
| Debra Holden, RTI International, debra@rti.org |
| Garry Lowry, Centers for Disease Control and Prevention, garry.lowry@cdc.hhs.gov |
| Cindy Soloe, RTI International, csoloe@rti.org |
| Abstract: This poster will describe the role of evaluation in the development of comprehensive cancer control (CCC) at national and state levels. Beginning in the nineties through the work of state health departments and partners as well as national cancer control organizations including the Centers for Disease Control and Prevention (CDC), CCC has become a national movement with demonstrated successes and accomplishments. This has been possible in part because of the importance of evaluation as a defined component of planning and implementation activities. Specific tools, techniques, materials and processes developed at the national level by CDC and evaluators at the Research Triangle Institute will be shared to illustrate how evaluation and data has been used to demonstrate success in cancer control outcomes. Additionally, a representative from an experienced CCC program, the Michigan Cancer Coalition, will describe their use of evaluation to guide program development and to demonstrate successes and accomplishments. |
| Stepping Up to The Challenge: Building Capacity and Setting Standards for the Evaluation of Foreign Affairs Policy and Program Evaluation - A Canadian Perspective With Global Reach |
| Poster Presentation 139 to be held in Sebastian Section J on Wednesday, Nov 11, 6:30 PM to 8:00 PM |
| Presenter(s): |
| Jennifer Jordan-Saifi, Department of Foreign Affairs and International Trade Canada, jennifer.jordan-saifi@international.gc.ca |
| Stephen Kester, Department of Foreign Affairs and International Trade Canada, stephen.kester@international.gc.ca |
| Abstract: The Evaluation Division of the Department of Foreign Affairs and International Trade Canada has initiated a professional community of practice in the evaluation and oversight of its own policies and programs while simultaneously building capacity in organizational development, e-collaboration and information & knowledge management to enhance lessons learned and best practices to achieve results excellence for the Government of Canada. Its efforts have expanded to facilitate a Government-wide professional dialogue through a Federal Evaluation Community of Practice focusing on cross-cutting evaluation areas of best practice and methodology. The question that this Think Tank session seeks to address is how can Communities of Practice effectively translate into Organizational Learning and Evaluation Capacity Building that aim at enhancing quality and results, particularly in the complex world of Foreign Affairs and International Trade that span international environments and rapidly changing, and at times competing, policy frameworks (both internationally and domestically). |
| An Evaluation of Changes in Policy, Practice, and Attitudes: Tobacco-Free Substance Abuse Facilities in New York State |
| Poster Presentation 139 to be held in Sebastian Section J on Wednesday, Nov 11, 6:30 PM to 8:00 PM |
| Presenter(s): |
| Elizabeth Brown, Research Triangle Institute, bettybrown@rti.org |
| Abstract: For the comprehensive evaluation of the New York Tobacco Control Program, Research Triangle Institute implemented a study addressing tobacco dependence treatment within an addictions framework. This study assesses substance abuse facilities' tobacco policies, tobacco dependence treatment, and related beliefs and attitudes. We conducted key informant interviews in 2004 with substance abuse facility directors, followed by a 2005 statewide survey of substance abuse facility program directors and clinical directors. We identified existing policies and treatment practices and assessed perceptions regarding implementing stricter regulations. Findings helped determine technical assistance to ease transition to new substance abuse facility tobacco regulations. After implementation of new regulations in 2008, we are conducting a follow-up survey in 2009 to assess changes in policy, systems, practices, and attitudes among substance abuse facility program and clinical directors. This poster will share evaluation priorities, methods, and analysis, and describe the study's influence on practice. |
| Making Decisions: Helping Foundation Trustees to Use Evaluation |
| Poster Presentation 139 to be held in Sebastian Section J on Wednesday, Nov 11, 6:30 PM to 8:00 PM |
| Presenter(s): |
| Samantha Nobles-Block, FSG Social Impact Advisors, samantha.nobles@fsg-impact.org |
| Tim Wilmot, Marin Community Foundation, twilmot@marincf.org |
| Srik Gopalakrishnan, Ball Foundation, srik@ballfoundation.org |
| Eva Nico, FSG Social Impact Advisors, eva.nico@fsg-impact.org |
| Abstract: This poster will address foundation trustee usage of evaluation and its connection to foundation decision making and performance. The poster will cover themes from the recent foundation trustee and CEO interviews conducted by FSG on the subject of evaluation, the recently published trustee evaluation toolkit that resulted from that research, and the experiences of the panelists in working with foundation trustees on issues of evaluation. The poster will cover several topics: (1) The importance of evaluation in foundation decision making (2) Key insights for evaluators around working with trustees, and integrating evaluation results into the discussion (3) Ways to engage trustees in the evaluation process, and educate them about the evaluation options available |
| Changing Horses Mid-Stream: Lessons Learned From Evaluator Transitions During Two Innovative Technology Experiences for Students and Teachers (ITEST) Science, Technology, Engineering, and Mathematics (STEM) |
| Poster Presentation 139 to be held in Sebastian Section J on Wednesday, Nov 11, 6:30 PM to 8:00 PM |
| Presenter(s): |
| Ginger Fitzhugh, Evaluation & Research Associates, gfitzhugh@psctlt.org |
| Karen Manuel, Puget Sound Center for Teaching, Learning and Technology, kmanuel@psctlt.org |
| Denice Hood, University of Illinois at Urbana-Champaign, dwhood@illinois.edu |
| Mary White, Arizona Grants Management, mwhite@azgm.org |
| Abstract: Utilizing the cases of two NSF (National Science Foundation) ITEST (Innovative Technology Experiences for Students & Teachers) funded STEM (Science, Technology, Engineering, and Mathematics) middle and high school educational programs, this interactive session will explore the issues and best practices associated with changing evaluators during the funding cycle. As a coordinated poster, the presenters bring important and complementary perspectives (incoming/outgoing & evaluator/PI) to this important issue within the field of evaluation, particularly related to K-12 educational programs. |
| Interorganizational Network Changes Among Health Organizations in the Brazos Valley, Texas |
| Poster Presentation 139 to be held in Sebastian Section J on Wednesday, Nov 11, 6:30 PM to 8:00 PM |
| Presenter(s): |
| Monica Wendel, Texas A&M Health Science Center School of Rural Public Health, mlwendel@srph.tamhsc.edu |
| John D Prochaska, Texas A&M University, jdprochaska@srph.tamhsc.edu |
| Heather Clark, Texas A&M Health Science Center School of Rural Public Health, hrclark@srph.tamhsc.edu |
| Abstract: Community health development is a process by which a community identifies factors influencing population health, assesses available resources to build the capacity to plan and take action, and implement interventions to address identified needs. At its core, community health development targets structural change and infrastructure development to facilitate more efficient and effective health service delivery systems and environmental changes to support improvements in community health. One indicator of structural change and common measure of community capacity is the relationships among the network of organizations that comprise that system. The Brazos Valley has employed a community health development approach to population health improvement in partnership with the Center for Community Health Development. Changes in interorganizational networks illustrate progress in the Brazos Valley. Contextual factors provide some insight into how the process has unfolded. |
| The Role of Transformational Learning in Evaluation: Helping to Increase the Sustainability of Programs |
| Poster Presentation 139 to be held in Sebastian Section J on Wednesday, Nov 11, 6:30 PM to 8:00 PM |
| Presenter(s): |
| Barbra Portzline, GenQuest Inc, bportzline@comcast.net |
| Patsy Boverie, University of New Mexico, pboverie@unm.edu |
| Abstract: Utilizing transformative learning in the field of program evaluation can help programs grow and develop without the dread often inherent in having to be evaluated. Transformative learning can help guide evaluators and programs through the change process while respecting the values of the program(s), helping them to transform practices, and to develop positive regard for both evaluators and those being evaluated. In this paper the use of transformative learning for evaluation is discussed in terms of helping evaluation of programs, developing sustainable changes, and maintaining positive attitudes for future evaluations. |
| Evaluation Utilization in the Context of Teacher Education Program Review |
| Poster Presentation 139 to be held in Sebastian Section J on Wednesday, Nov 11, 6:30 PM to 8:00 PM |
| Presenter(s): |
| Georgetta Myhlhousen-Leak, University of Iowa, gmyhlhousenleak@hotmail.com |
| Abstract: This research investigated the types of use and the factors affecting use in the context of teacher education program accreditation in Iowa. The study explored program administrators', and faculty member's perceptions and experiences with use in the context of teacher education program review (i.e., accreditation). As potential evaluators and/or intended users of the assessment process faculty members represent key stakeholders in the evaluative efforts of program review. This study identified the types of use (i.e. instrumental, conceptual, political, and process) and factors effecting use (i.e. human, context, procedural) that administrators and faculty, found most salient. The population for the study was drawn from four volunteering four-year liberal arts institutions in the state of Iowa. Respondents completed a structured open-ended interview and five related scales. Findings suggest that both the nature of the political context and specific variables of context are instrumental in achieving use of the evaluation and its process. |
| Assessment of Local Community Needs and Resources to Reduce Substance Abuse and HIV Among Youth: Methods and Challenges Encountered |
| Poster Presentation 139 to be held in Sebastian Section J on Wednesday, Nov 11, 6:30 PM to 8:00 PM |
| Presenter(s): |
| Charles Brown, Centerstone Research Institute, charles.brown@centerstoneresearch.org |
| Celia Larson-Pearce, Independent Consultant, celialarson@comcast.net |
| Heather Cobb, Centerstone Research Institute, heathercobb@centerstoneresearch.org |
| Abstract: In the recent past, many federally funded community-based substance abuse and HIV prevention programs targeting youth have been asked to conduct a community needs assessment to develop prevention strategies and program outcomes. This paper provides an overview of the methods used in conducting a community needs assessment and an illustration of how the information was presented to the funding agency. The presenter will also discuss particular challenges encountered throughout the process of collecting information and data involving youth substance abuse and HIV transmission. |
| Geography is More Than Just Maps: Geographical Contributions to Understanding Context, Conducting Analysis, and Representing Findings for Program Evaluation |
| Poster Presentation 139 to be held in Sebastian Section J on Wednesday, Nov 11, 6:30 PM to 8:00 PM |
| Presenter(s): |
| Douglas Grane, University of Iowa, douglas.grane@gmail.com |
| Abstract: This paper outlines existing and potential contributions to program evaluation from Geography. The discipline of Geography is sometimes thought to be limited to locating places on a map, yet Geography provides a unique understanding of context, various analytical tools, and methods for representing the evaluand and its context. Geographers are trained to analyze both place and space, and this training produces transdisiplinary thinking that brings forth an awareness of political jurisdiction, issues related to the physical environment, and cultural and social differences within place and across space. Geography also captures spatial distribution and disparities of important human and physical phenomena. Recent advances in Geographical information Science (GIS) is part of a long tradition of spatial analysis and representation in Geography, yet Geographers also contribute a great deal to the specific application of theories borrowed across the political, psychological, social, and educational, health, and natural sciences as well as the humanities. |
| Special Education Placement in Early Grades: the Role of School Climate, Child Behavior Problems and Race |
| Poster Presentation 139 to be held in Sebastian Section J on Wednesday, Nov 11, 6:30 PM to 8:00 PM |
| Presenter(s): |
| Eric Slade, University of Maryland, eslade@psych.umaryland.edu |
| Sergio Prada, University of Maryland Baltimore County, sprada1@umbc.edu |
| Abstract: African Americans' rate of special education placement during elementary school (grades K-5) was compared to placements among whites, using nationally representative data from the Early Childhood Longitudinal Study - Kindergarten Cohort (ECLS-K). The sample included (N=1,723) African American and (N= 6,242) White first time Kindergartners attending public schools in the U.S. In longitudinal multilevel logistic regression models, the likelihood of special education placement was adjusted for Kindergarten standardized math and reading scores, externalizing behaviors, interpersonal skills, and motor skills; schools' characteristics (urban location, poverty rate, and education climate); and age and gender. In predominately white schools, African Americans were more likely to be placed than whites. However, after adjusting for academic achievement and behavioral differences, placement rates in 1st through 5th grades did not differ by race. These results suggest that African-Americans' overrepresentation in elementary special education programs largely reflects greater prevalence of skill deficits among African American Kindergartners |
| Evaluating Prognosis and Decline in Hospice Dementia Patients: An Evaluation of the Effectiveness of the Functional Assessment Staging Instrument (FAST) |
| Poster Presentation 139 to be held in Sebastian Section J on Wednesday, Nov 11, 6:30 PM to 8:00 PM |
| Presenter(s): |
| William Cabin, Richard Stockton College, williamcabin@yahoo.com |
| Susan Havens-Lang, United Hospice of Rockland, slang@hospiceofrockland.org |
| Abstract: Clinical consensus does not exist regarding the most effective method for determining prognosis and measuring decline in hospice patients (Lang, 2009 forthcoming). Despite the lack of a national guideline, Medicare has increased requirements for hospices to substantiate and document decline in hospice dementia patients. this paper presents the results of an evaluation of one mid-sized hospice's use of one instrument, the Functional Assessment Staging Instrument (FAST). Implications for policy and practice will be discussed. |
| Ignoring the Internal Evaluator and Paying the Price: Agency Arrogance Meets Regulatory Compliance |
| Poster Presentation 139 to be held in Sebastian Section J on Wednesday, Nov 11, 6:30 PM to 8:00 PM |
| Presenter(s): |
| William Cabin, Richard Stockton College, williamcabin@yahoo.com |
| Abstract: There is extensive literature regarding internal evaluation methods (Rossi, 2008; Weiss, 2002, but limited literature regarding the impact of organizational context on internal evaluators and evaluations. This paper uses the case of one human service agency where the agency executives ignored multiple internal evaluations of a State-regulated substance abuse and mental health intervention program, resulting in a six month loss of Medicaid funding and near loss of its operating license. Details of the evaluations and contextual factors will be presented. |
| Involving 4H Young Adult Camp Staff in Program Evaluation |
| Poster Presentation 139 to be held in Sebastian Section J on Wednesday, Nov 11, 6:30 PM to 8:00 PM |
| Presenter(s): |
| Nancy Franz, Virginia Cooperative Extension, nfranz@vt.edu |
| Barry Garst, American Camps Association, bgarst@acacamps.org |
| Sarah Baughman, Virginia Tech, baughman@vt.edu |
| Chris Smith, Virginia Cooperative Extension, cksmith@vt.edu |
| Brian Peters, College of William and Mary, bapeters@wm.edu |
| Abstract: Extension educators have long known the value of direct experience in the learning process. Participatory evaluation extends this philosophy to the realm of program evaluation. This paper examines the value of involving front line 4-H camp staff, members of the camp community as practitioner researchers with university evaluators in studying the type and conditions of transformative learning in young adult camp staff. Best practices for conducting participatory evaluation in community education environments (including use of Myspace to gather data) and the benefits of this work are shared. |
| An Analysis of Empathy Training and Evaluation |
| Poster Presentation 139 to be held in Sebastian Section J on Wednesday, Nov 11, 6:30 PM to 8:00 PM |
| Presenter(s): |
| Tony C M Lam, University of Toronto, tlam@oise.utoronto.ca |
| Klodiana I Kolomitro, University of Toronto, klodiana.kolomitro@utoronto.ca |
| Flanny Alamparambil, Brother Andre Catholic High School, flannya@hotmail.com |
| Abstract: To address our question: 'To what extent can training improve empathy?' 29 evaluations of empathy training over the past 30 years were analyzed and their findings synthesized. These trainings were conducted in human service and social science disciplines. A variety of training methods were used with the most popular being instruction combined with exercises (62%). Regarding evaluations designs, 76% used a control group. All but two studies (93%) reported positive findings and mainly in regard to learning (90%).The most common method of measuring change was rating scales (55%), followed by open response items (31%).Some of the findings were moderated by gender, age, education level and time of measurement. However, validity and generalizability of these findings are not sufficiently high nor conceptualization of empathy across studies consistent enough to warrant a clear answer to our research question. The paper discusses methodological issues and presents suggestions for future research in empathy training. |
| Evaluation of a School Based Family Literacy Program: An Examination of Causation |
| Poster Presentation 139 to be held in Sebastian Section J on Wednesday, Nov 11, 6:30 PM to 8:00 PM |
| Presenter(s): |
| Zandra Gratz, Kean University, zgratz@kean.edu |
| Abstract: This poster describes the evaluation results of a school based Even Start Family Literacy project which served youngsters with demonstrated literacy needs and were living in poverty. Although random assignment was not possible, effort was made in the evaluation to garner data which could yield causative statements. In so doing, several quasi-experimental studies were completed. In particular, data available via published test norms were used to establish non-treatment expectations of growth over time. Also, test data of a non-participant group were accessed against which participant growth was compared. Lastly, teachers were asked to make judgments comparing participants to non participants with regard to achievement and other ancillary school measures. The extent to which these methods yielded confirmatory inferences is discussed as is their efficacy in supporting causative statements. |
| An Examination of Program Outcomes From Different Contexts to Uncover Process Weaknesses: A Comparison of Student and Faculty Perceptions of Technology Training |
| Poster Presentation 139 to be held in Sebastian Section J on Wednesday, Nov 11, 6:30 PM to 8:00 PM |
| Presenter(s): |
| Gale Mentzer, University of Toledo, gale.mentzer@utoledo.edu |
| Torey Hickman, University of Toledo, torey.hickman@utoledo.edu |
| Abstract: The US DOED Teacher Quality Enhancement grants have the required outcome that pre-service teachers gain proficiency in the use of technology as an instructional tool. Three years of evaluation of one project indicate that while pre-service teachers have mastered the use of technology tools, they have done little to integrate technology into their teaching. To determine why pre-service teachers were not integrating technology, an exploration of faculty perceptions of the role technology should and does play in their pre-service methods courses was conducted. This study included in-depth interviews and qualitative research analyses (faculty) as well as a student survey (rating scales and open ended responses) and examination of student portfolios. This study explored common themes in faculty perceptions and compared them with student responses. The result yielded interesting commonalities that indicated why pre-service teachers are competent in technology use themselves yet not confident in incorporating technology into their teaching. |
| The Role of Context in the Degree of Cooperative Participation in the Evaluation of a Comprehensive Project |
| Poster Presentation 139 to be held in Sebastian Section J on Wednesday, Nov 11, 6:30 PM to 8:00 PM |
| Presenter(s): |
| Gale Mentzer, University of Toledo, gale.mentzer@utoledo.edu |
| Abstract: This poster provides the results of a study that examined the relationship between project staff attitudes and beliefs about the role of evaluation in a multi-million dollar federal education grant and staff levels of cooperation in the implementation of the evaluation plan. The study used a mixed methods design wherein attitudes were collected through in-depth interviews and level of cooperation was measured using a rating scale based upon the type (formative or summative) and the frequency of evaluation implemented. Results showed a high, positive relationship between attitudes and cooperation and indicated that certain stereotypes were detrimental to full implementation of the evaluation plan. A scoring rubric for level of cooperation will be included in the display. |
| An Evaluation of Crop Producers Use and Desire for Land-Grant University Information |
| Poster Presentation 139 to be held in Sebastian Section J on Wednesday, Nov 11, 6:30 PM to 8:00 PM |
| Presenter(s): |
| Tanya Franke, Oklahoma State University, tanya.franke@okstate.edu |
| Kathleen D Kelsey, Oklahoma State University, kathleen.kelsey@okstate.edu |
| Tom Royer, Oklahoma State University, tom.royer@okstate.edu |
| Abstract: Cooperative Extension (CE) was traditionally the only source of research-based knowledge that producers had access to until the mid 20th century. Today, producers have many choices when seeking crop production information. To remain competitive, Extension educators must evaluate producers periodically via stakeholder engagement strategies to discover their preferred modes for receiving educational materials related to crop production. This poster will present the results of an evaluation that discovered where and how producers obtained crop production information. |
| Evaluation of the Effectiveness of Public Relations Media under the Happy Workplace Plan of Private Organizations by Discourse Analysis |
| Poster Presentation 139 to be held in Sebastian Section J on Wednesday, Nov 11, 6:30 PM to 8:00 PM |
| Presenter(s): |
| Suwimon Wongwanich, Research and Evaluation in Education, suwimon.w@chula.ac.th |
| Abstract: The Happy Workplace Plan of private organizations is an occupational plan set forth by the Office for the Thai Health Promotion Foundation. It has a performance time of 3 years for pushing toward the goal of good work life quality. This evaluation is an annual external evaluation with the following evaluate the efficiency of the primary media in the project and analyze the production forms, public relations conducive to the creation of a good image for the project. The objective of this project were evaluation of the efficiency of the primary media in the project and analyze the production forms, public relations conducive to the creation of a good image for the project, the language activities resulting in the creation of credibility, the relationship to an atmosphere of friendship between the media producer and the media consumer. The evaluation of the effectiveness of this public relations plan project implemented media discourse analysis techniques use information amounts from the observation of accessibility and perception of media of the target groups and the quality information from the interviews of those associated with the project while also using media discourse analysis techniques. In the evaluation of medias by using the method of discourse analysis by implementing the 12 issue framework for discourse analysis it was found that the medias randomly selected contained the forms of production, distribution and consumption that supported the creation of a good image for the project, language acitivities that created credibility, relationships to an atmosphere of friendship between the media producer and the media consumer, the use of language structure, form, words and idioms persuaded consumers to have the following ideas: the use of linguistic techniques that generated motivation for the recipients to have good attitudes toward the concept of the development of work life quality. |
| Toward Inclusion Education Quality: A Master's Degree Program Evaluation |
| Poster Presentation 139 to be held in Sebastian Section J on Wednesday, Nov 11, 6:30 PM to 8:00 PM |
| Presenter(s): |
| Olga Alegre, University of La Laguna, oalegre@telefonica.net |
| Luis Villar, University of Seville, mvillar@us.es |
| Abstract: Special education master's degrees are proliferating most probably in response to the requirement for all special teachers to be highly qualified. The aim of the study is to evaluate the 10-year Master's Degree 'Educating in Diversity' (MDED) at the University of La Laguna (ULL), and to examine the extent to which the development of diversity competencies in graduates is related to their perceptions of the overall quality of the postgraduate program. Two hundred and eight University students and 235 part-time faculty members evaluated the basic program indicators. Finally, MDED results gathered from 135 postgraduates and 707 stakeholders indicate high levels of purpose achievement and satisfaction with the program, the faculty, and the curricular content. The framework for improvement in which the MDED is viewed as compatible with national and regional evaluation and accrediting agencies is discussed. |
| Reducing Drinking Among Junior Enlisted Air Force Members in Five Communities: Early Findings of the Enforcing the Underage Drinking Laws (EUDL) Program's Influence on Self-Reported Drinking Behaviors |
| Poster Presentation 139 to be held in Sebastian Section J on Wednesday, Nov 11, 6:30 PM to 8:00 PM |
| Presenter(s): |
| Christopher Spera, ICF International, cspera@icfi.com |
| Keita Franklin, ICF International, kfranklin@icfi.com |
| Kazuaki Uekawa, ICF International, kuekawa@icfi.com |
| Ronald Szoc, ICF International, rzoc@icfi.com |
| John Kunz, ICF International, jkunz@icfi.com |
| Milton Cambridge, Air Force, milton.cambridge@pentagon.af.mil |
| Abstract: In the fall of 2006, the Office of Juvenile Justice and Delinquency Prevention (OJJDP) awarded discretionary grants to five communities in four states as part of the Enforcing Underage Drinking Laws (EUDL) initiative to design and implement a set of interventions across a three-year period using an environmental strategies approach to reduce drinking and associated alcohol-related misconducts among active duty Air Force members ages 18-25, with a focus on the underage population. Data from a large-scale, anonymous survey fielded in the spring of 2006 (i.e., pretest) and the spring of 2008 (i.e., posttest) indicated that the percentage of junior enlisted personnel at risk for an alcohol problem dropped 6.6% in the Air Force overall during the last two years, but dropped as much as 13.6% and 9.8%, respectively, in two Arizona demonstration communities that implemented the intervention. The potential role of the EUDL program, and the associated intervention activities, in reducing the percentage of junior enlisted Air Force members at risk for an alcohol problem in each of the demonstration communities is discussed. |
| Developing Criteria for Addressing Diversity in Evaluation of Science, Technology, Engineering, and Mathematics (STEM) Programs |
| Poster Presentation 139 to be held in Sebastian Section J on Wednesday, Nov 11, 6:30 PM to 8:00 PM |
| Presenter(s): |
| Wanda Casillas, Cornell University, wdc23@cornell.edu |
| William Trochim, Cornell University, wmt1@cornell.edu |
| Abstract: This project was designed to 1) establish criteria for conducting a culturally responsive evaluation of STEM programs and 2) to establish criteria for assessing cultural responsiveness in STEM program planning. A secondary objective was to evaluate the relationship between points 1 and 2. We recruited staff from programs conducting evaluations throughout New York. As well as evaluators nationally that identified a concern with diversity issues in evaluation through membership with American Evaluation Association topical interest groups. We employed a standard concept mapping methodology where participants were asked to generate statements about behaviors and attitudes toward diversity in evaluation and program planning. After participants sorted and rated the statements produced, we employed The Concept System software to produce statement clusters and created a taxonomy behaviors and attitudes identified across staff and evaluators. A comparison of evaluator and program staff ratings was also conducted to assess for differences between groups. |
| Defining Cultural Value: Using Merit Goods and Social Capital Concepts to Explain and Evaluate Support of the Arts in Maine |
| Poster Presentation 139 to be held in Sebastian Section J on Wednesday, Nov 11, 6:30 PM to 8:00 PM |
| Presenter(s): |
| Deborah Smith, University of Southern Maine, deborah.smith.2006@gmail.com |
| Abstract: This poster session will summarize initial findings from my doctoral dissertation on cultural policy: an analysis of 750 grants awarded to Maine nonprofit organizations (NPOs) by Federal, State and private arts funders. A current trend that affects both the arts and the NPOs that manage them is their role at the intersection of economic and community development; their impact on cultural policy is similarly interrelated. The policy question itself changes: not 'why does art deserve support?' but 'why should we support NPOs that produce/sustain art?' and 'what does 'successful' support look like?' Before we can evaluate the use of grant funds, however, we need to understand the rationales informing funders' decisions and expectations. The presentation will share results about two frameworks explored in the data and allow me to solicit feedback on preliminary ideas for the final chapters: developing appropriate assessment tools based on merit goods and social capital theories. |
| Analysis of the Vocational Rehabilitation Program Outcomes |
| Poster Presentation 139 to be held in Sebastian Section J on Wednesday, Nov 11, 6:30 PM to 8:00 PM |
| Presenter(s): |
| Alberto Migliore, University Massachusetts Boston, alberto.migliore@umb.edu |
| Abstract: During the past decade, the overall outcomes of the Vocational Rehabilitation program have not been growing while the employment gap between people with disabilities and the general population has been widening. The purpose of this paper is to show that a more in-depth analysis of Vocational Rehabilitation outcomes is critical for improving the employment outcomes of adults with disabilities. We computed standardized Z scores of state Vocational Rehabilitation agencies' outcomes and identified outcomes that were average or one standard deviation above or below the average. We found that above average outcomes were substantially greater than below average outcomes. In addition, we found that state Vocational Rehabilitation agencies that obtained above average outcomes on one variable could have obtained below average results on other equally important outcomes. We recommend that above average outcomes be used as benchmarks to steer the overall Vocational Rehabilitation outcomes upwards. |
| Embracing the Context of Pediatric Rehabilitation Settings: The Need for Family-Centered Evaluation Activities |
| Poster Presentation 139 to be held in Sebastian Section J on Wednesday, Nov 11, 6:30 PM to 8:00 PM |
| Presenter(s): |
| Katherine Moreau, University of Ottawa, kmoreau@cheo.on.ca |
| Abstract: As the current political and funding environment continues to stress the importance of accountability, performance measurement and evidence-based practice, paediatric rehabilitation settings are under increasing pressure to engage in a variety of evaluation activities. Simultaneously, these settings must also uphold the philosophy of family-centred care. This philosophy states that service providers have a responsibility to collaborate with families at all levels including program development, implementation and evaluation. However, a critical review of the literature reveals that the extent to which families are actively involved in evaluation activities and the degree to which the principles, premises and elements of family-centred service are upheld in these activities is relatively low. This presentation focuses on the findings of this literature review and in doing so justifies the need for pursuing research on family-centred evaluation activities and developing a family-centred evaluation approach that embraces the unique context of paediatric rehabilitation programs. |
| Impact of Financial Assessment in the Appraisal Process on the Commercialization of a Technology Transfer Program |
| Poster Presentation 139 to be held in Sebastian Section J on Wednesday, Nov 11, 6:30 PM to 8:00 PM |
| Presenter(s): |
| Yukio Kemmochi, Japan Science and Technology Agency, kemmochi@jst.go.jp |
| Abstract: Private enterprises that conduct technology transfer projects need to have technological skills as well as financial capabilities. Thus, the Contract Development Program, a technology transfer program that has been overseen by the Japan Science and Technology Agency (JST) for fifty years has been performing financial assessments in the appraisal process since 2001 in addition to providing technological judgement. The average development length of the projects in the program is about four years and the average time gap between the completion of the development and the projects' commercialization is about three years. As a result, we can now see the early results of the impact of financial assessment in the appraisal process on the commercialization of the project. In this presentation, we report the correlation between appraisal results (both technological and financial) and commercialization. |
| The Needle and the Damage Done: When Evaluation Samples Don't Reflect Populations and What Responsible Evaluators Should Do About It |
| Poster Presentation 139 to be held in Sebastian Section J on Wednesday, Nov 11, 6:30 PM to 8:00 PM |
| Presenter(s): |
| James Salzman, Ohio University, salzman@ohio.edu |
| Sharon Brown, Cleveland State University, s.a.brown54@csuohio.edu |
| Abstract: Evaluators and researchers use samples for the purpose of inferring their results and conclusions to the populations of interest. The choice of sampling procedures is one of the most important decisions that evaluators make in their proposals. When the samples chosen, though, do not reflect the population, evaluators will misinterpret or, worse, misjudge the value of a program. The authors analyze two recent external evaluations, one state (Westat/LPA, 2008) and one national (Gamse et al., 2008), of impacts on student achievement of a reading reform and identify two specific problems: 1) sometimes, theoretically sound sampling procedures still don't reflect population parameters; and 2) some evaluators allow their analytical techniques to lead their samples. They also provide recommendations to mitigate these concerns based on AEA's Evaluation Standards and accepted research procedures. |
| Effective Evaluation Methods for Indirect Education Interventions: Lessons From an Extension Nutrition Education Program |
| Poster Presentation 139 to be held in Sebastian Section J on Wednesday, Nov 11, 6:30 PM to 8:00 PM |
| Presenter(s): |
| Karen Franck, University of Tennessee at Knoxville, kfranck@utk.edu |
| Abstract: This paper examines the challenges, benefits, and strategies for effective evaluation of indirect education interventions with adult audiences. As adults face increasing time and money constraints, it is more difficult to attract and retain adults in traditional direct educational settings such as classes and group meetings. Indirect education methods are used frequently with nutrition education programs to help inform adult audiences. These methods are implemented at various locations such as health fairs, food commodity distribution sites, and community health centers and generally reach larger and more diverse audiences compared with direct education efforts. However, it is often difficult to assess the impact that these indirect methods have on actual behavior. This paper examines existing research on the impact of indirect education and reports on a study that used qualitative and quantitative methods to examine the impact of indirect methods employed by educators in a state Extension Nutrition Education Program for low-income adults. |
| An Assessment of Respiratory Protection Program for Agricultural Workers |
| Poster Presentation 139 to be held in Sebastian Section J on Wednesday, Nov 11, 6:30 PM to 8:00 PM |
| Presenter(s): |
| Prosper Doamekpor, Pennsylvania State University, prosperdoa@hotmail.com |
| Connie Baggett, Pennsylvania State University, bbc@psu.edu |
| Rama Radhakrishna, Pennsylvania State University, brr100@psu.edu |
| Abstract: A review of the agricultural safety and health injury/illness prevention research literature identified frequent flaws or limitations in research design and data analyses procedures. The flaws or limitations include no baseline measurements for intervention studies, same day pre- and post-tests, a lack of objective outcome measures, and inadequate data for process evaluation. To remedy flaws and limitations, the National Institute for Occupational Safety and Health (NIOSH) proposed a conceptual model to assess the effectiveness of intervention programs related to agricultural safety and health. Three phases are identified in this model: developmental, implementation, and effectiveness research. Each phase has five intervention research tasks, which include, needs assessment, developing partnerships, and selecting appropriate methodologies or research design for the intervention programs. The conceptual model and a logic model were employed to show how the evaluation of three research phases was carried in designing, implementing, and evaluating a respiratory safety program. The study focused on identifying and analyzing variables that could explain the variance in the perceptions of agricultural workers about preventing or alleviating certain behaviors and practices that could lead to serious health problems. |
| Focused Active Evaluation: Practical Model for Informed Decision Making |
| Poster Presentation 139 to be held in Sebastian Section J on Wednesday, Nov 11, 6:30 PM to 8:00 PM |
| Presenter(s): |
| Flordeliz Serpa, Non-Profit Evaluation and Resource Center Inc, nperci@nperci.org |
| Lizzette Rojas, Non-Profit Evaluation and Resource Center Inc, nperci@nperci.org |
| Abstract: The Focused Active Evaluation Model was developed to facilitate a practical and useful evaluation framework based on institutional context, immediate community, and extended community, as well as to improve an organization's decision-making process. The model presents the evaluator with a renewed role: as an extension of the control system and as a collaborator and facilitator. It can be easily integrated into the normal operations of any organization. It integrates innovative practices and utilizes new knowledge to contribute to social progress and better quality of life. |
| The Presence of Their Absence: The Collection of Sexual Minority Subject Variables in Evaluations of Abstinence-Only Education and HIV-Prevention Programs |
| Poster Presentation 139 to be held in Sebastian Section J on Wednesday, Nov 11, 6:30 PM to 8:00 PM |
| Presenter(s): |
| Sean Little, Independent Consultant, littles25@bellsouth.net |
| Abstract: This poster presentation utilizes a high contrast purposeful sampling strategy to select published peer-reviewed articles evaluating HIV prevention and abstinence only education programs. Using content analysis, this poster presentation examines whether and how sexual minority data are collected, constructed, analyzed, and reported. Evaluation articles published in 2005 are examined in an exploratory analysis and those published in 2008 are examined as part of a confirmatory analysis. Using a realist evaluation framework (what works for whom under what circumstances) this presentation focuses on the effect of program theory on which subject variables are collected, constructed, analyzed and reported. |
| America's Children's Museums: Supportive Education for Un- and under-served children |
| Poster Presentation 139 to be held in Sebastian Section J on Wednesday, Nov 11, 6:30 PM to 8:00 PM |
| Presenter(s): |
| Deborah Chapin, University at Albany - State University of New York, researchbydesign@gmail.com |
| Abstract: The purpose of this study was to explore nonformal education provided by children's museums to General Education Students, Minority Youth, and Children with Special Needs. The sample population consisted of 215 children's museum education directors who were members of The Children's Museum Association in the United States. A survey was designed and tested to gather the perceptions of these education directors on six constructs: 1) Curriculum Planning and Design, 2) Current Programming, 3) Pedagogy, 4) Reduction of Barriers, 5) Mode of Outreach, and 6) Effectiveness. Information was obtained from 80 education directors in 36 different states. A multi-phase analysis found three museum types with varying degrees, patterns, and styles of service planning, delivery, and effectiveness in reaching these student groups. Children's museums were most able to serve General Education students; however, most museums did not know how or did not attempt to reach un- or underserved children. |
| A Follow up Evaluation of New Horizons: A School-based Substance Use Prevention Program |
| Poster Presentation 139 to be held in Sebastian Section J on Wednesday, Nov 11, 6:30 PM to 8:00 PM |
| Presenter(s): |
| Ingrid Todd, The Center For Drug Free Living, itodd@cfdfl.com |
| Susan Nuckols, The Center For Drug Free Living, snuckols@cfdfl.com |
| Abstract: There have been four evaluations of the New Horizons Program. The purpose is to empirically validate the program based on its goals to determine if previous findings can be replicated. A causal comparative analysis was conducted using pre test post test data. The nine specific program objectives were: alcohol, marijuana, and other drug use; number of arrests; school attendance and grade point average; reducing risk factors and increasing protective factors; and increasing ATOD knowledge. The data for this study utilized the Florida Youth Substance Abuse Survey (FYSAS) and a database of information normally collected during the 2007-08 school year from 16 high schools in central Florida. Statistically significant improvement was found in all nine program objectives, by students who completed at least twelve of the sixteen group sessions in the New Horizons Curriculum. These findings were consistent with the three previous evaluations. |
| Evaluating Agricultural Universities Websites: A Wake-up Call for Extension Unit and Users Alike |
| Poster Presentation 139 to be held in Sebastian Section J on Wednesday, Nov 11, 6:30 PM to 8:00 PM |
| Presenter(s): |
| Surya Rathore, Govind Ballabh Pant University of Agriculture & Technology, suryarathore@gmail.com |
| Manmeet Kaur, Govind Ballabh Pant University of Agriculture & Technology, suryajodhpur@rediffmail.com |
| Abstract: One of the functions of agricultural universities in India is to extend their research and teaching competencies and information to the ultimate users such as students, faculty, job seekers, farmers, extension personnel etc. Website is the answer to cater to the needs of all odd clients of agricultural Universities in this era of Information and Communication Technology (ICT). To address the challenge of info-gut and info-garbage, evaluation of websites is the only answer. Discerning information consumers need to learn to apply some evaluation concepts critically to the sites they are visiting as well as website developers too can identify their customer needs through this evaluation study. To address the needs, a 5 point scale has been developed and the top rated websites were the ones having recent scientific information in the form of online publication of abstracts, research activities, news and views and online discussion and feedback in webpage development. |
| Measuring Healthy Decision-Making Outcomes Using The Decision Skills and Healthy Lifestyles Scales of The Health Rocks! Program |
| Poster Presentation 139 to be held in Sebastian Section J on Wednesday, Nov 11, 6:30 PM to 8:00 PM |
| Presenter(s): |
| Stewart Lee, Mississippi State University Extension Service, stewartl@ext.msstate.edu |
| Michael Newman, Mississippi State University Extension Service, mnewman@humansci.msstate.edu |
| Joseph Donaldson, University of Tennessee, jldonaldson@tennessee.edu |
| Abstract: Research shows that prevention programs that target youth between the ages of 8-12 may help increase life skills, thus lowering tobacco, alcohol and drug use. To measure these outcomes for the Health Rocks! program, two instruments from the University of Tennessee were selected: the healthy lifestyles scale and the decision-making scale. Both instruments were developed by expert panels as post-only measures. For this evaluation, the Cronbach's alpha estimate of reliability for the decision making scale (10 items) was .80 (N = 1373). The healthy lifestyles scale (6 items) had a Cronbach's alpha of .83 (N = 1373). The results clearly indicated an increase in decision making skills and healthy lifestyle skills with results for each items being positive and most being extremely positive (modal answer was highest response for 7 of 10 items on the decision making scale and 6 of 6 items on the healthy lifestyles scale). |
| Evaluators as Learners: Perspectives From Student Evaluators Serving Multiple Roles in the Evaluation Context |
| Poster Presentation 139 to be held in Sebastian Section J on Wednesday, Nov 11, 6:30 PM to 8:00 PM |
| Presenter(s): |
| Amanda Evert, Redlands Community College, everta@redlandscc.edu |
| Tricia Clapp, Oklahoma State University, tricia.clapp@okstate.edu |
| Kathleen D Kelsey, Oklahoma State University, kathleen.kelsey@okstate.edu |
| Abstract: The evaluation was the result of a graduate course project in program evaluation (AGED 6223). The student evaluators were also administrators and instructors at regional community colleges and were responsible for facilitating an online bachelorette degree completion program called Agriculture Pathways Partnership. Pathways was evaluated for student satisfaction with the online degree, administrative processes, and partnership functioning among collaborating campuses. The evaluators engaged in multiple roles as graduate students, internal evaluators, clients, and agents during the evaluation, enriching the process by having tacit knowledge of the program, its learners, and community culture. The results of the evaluation will be presented along with a discussion of the complexities of assuming multiple roles in the evaluation context. |
| Evaluating Performance in the Healthcare Settings Using the Values of Stakeholders |
| Poster Presentation 139 to be held in Sebastian Section J on Wednesday, Nov 11, 6:30 PM to 8:00 PM |
| Presenter(s): |
| Ezechukwu Awgu, Western Michigan University, eawgu@yahoo.com |
| Abstract: Stakeholders are groups or individuals who are affected by, and influence activities in the healthcare settings. The values of stakeholders consist of such things as their desires and goals. Healthcare costs have been on the rise in the past decade. This has resulted in increasing demand by the stakeholders for improved quality of services and accountability. The demands, therefore, have put pressure on the activities and organizational goals or success of healthcare settings which consists of such things as profitability, clinical quality, high professional standards, patient safety, and so forth. This presentation will address these questions. What are healthcare settings organizational goals? What are the essential values of stakeholders for evaluating performance in the healthcare settings? How can values of stakeholders be synthesized into merit and worth? Also, the study will discuss the numeric weight and sum, the qualitative synthesis method, and stakeholders mapping. |
| Degree or No degree, That is the Question: Evaluating the Effectiveness of Undergraduates Who Teach Undergraduates |
| Poster Presentation 139 to be held in Sebastian Section J on Wednesday, Nov 11, 6:30 PM to 8:00 PM |
| Presenter(s): |
| Corey Tincher, Southern Illinois University at Carbondale, ctincher@siu.edu |
| Abstract: One of the most important components in education is the ability level of the instructor. Often this attribute is associated with the documented qualifications of said teacher, usually expressed by an academic degree or certificate. In a large Midwestern university's psychology department, a program is employed in which undergraduate students formally instruct other undergraduates within the confines of an official class. Since the academic degree level between student and teacher is identical, there is a question as to the effectiveness of this instruction. An evaluation of this program was conducted to measure the actual outcomes of performance and self-concept variables of both instructors and students using a combination of quantitative and qualitative methods. |
| Rethinking Students' Educational Expectations in the Context of Evaluation |
| Poster Presentation 139 to be held in Sebastian Section J on Wednesday, Nov 11, 6:30 PM to 8:00 PM |
| Presenter(s): |
| Elise Arruda, Brown University, elise_arruda@brown.edu |
| Stephanie Feger, Brown University, stephanie_feger@brown.edu |
| Abstract: Research indicates that minority and non-minority students have similar educational aspirations but that minority students' tend to have lower educational expectations. That is, there is a gap in minority students' educational aspirations and expectations that does not exist in non-minority students. This poster will share results from a National Science Foundation funded after school technology program evaluation. While there was no significant change in students' use of technology, a significant change was found in students' educational expectations. Evaluators conducted analyses measuring whether the minority students participating in the after school program closed the gap between their education aspirations and expectations. Results suggested that prior to participation, students' aspirations and expectations were statistically significant; however, after participation, there was no longer statistical significance between students' aspirations and expectations. While counter-intuitive, the lack of significance at the post-test suggests that minority students' educational expectations positively increased to more closely match their aspirations. |
| Results From A Three-Year Study of Smaller Learning Communities in High Schools |
| Poster Presentation 139 to be held in Sebastian Section J on Wednesday, Nov 11, 6:30 PM to 8:00 PM |
| Presenter(s): |
| Shana Pribesh, Old Dominion University, spribesh@odu.edu |
| Linda Bol, Old Dominion University, lbol@odu.edu |
| Tiffany Hall, Old Dominion University, thall@odu.edu |
| Peter Baker, Old Dominion University, pbaker@odu.edu |
| David Blackburn, Old Dominion University, dblackburn@odu.edu |
| Abstract: Creating smaller schools/learning communities (SLCs) has been advocated for improving high school student engagement and graduation rates (NRC, 2002). And although the linkages of smaller learning communities to student achievement have been found to be promising (Felner, Ginter, and Primavera, 1982; NRC, 2002), the research connecting the SLC structure with student performance is tenuous due to methodological issues. This poster presents data from a three-year evaluation of smaller learning communities in four U.S. high schools. Using case studies methodology, we examine constructs such as school climate, student self-concept and academic press as well as student achievement, retention and sense of belonging. |
| Evaluating Supplemental Education Service Providers: A Methodological Review |
| Poster Presentation 139 to be held in Sebastian Section J on Wednesday, Nov 11, 6:30 PM to 8:00 PM |
| Presenter(s): |
| John Nunnery, Old Dominion University, jnunnery@odu.edu |
| Shana Pribesh, Old Dominion University, spribesh@odu.edu |
| Steve Ross, University of Memphis, smross@memphis.edu |
| Shanan Chappell, Old Dominion University, sfleming@odu.edu |
| Abstract: The No Child Left Behind Act of 2001 (NCLB) specifies that districts must provide supplemental educational services (SES) to students attending schools that are in the second year of school improvement. The mandate to provide SES services have given rise to praise (student and parent choice, greater accountability) and criticism (veiled vouchers, fragmenting Title I, bureaucratic overload). Although state education agencies are tasked with evaluating the effectiveness of SES providers, this has proven difficult due to selection and methodological issues as well as funding. In this poster, we review the methodological issues concerning the evaluation of SES providers. We also review the current status of SES evaluation in the United States. |
| Community Collaboration to Show Benefits of Youth Arts Opportunities |
| Poster Presentation 139 to be held in Sebastian Section J on Wednesday, Nov 11, 6:30 PM to 8:00 PM |
| Presenter(s): |
| Lora Warner, University of Wisconsin at Green Bay, warnerl@uwgb.edu |
| Abstract: Program staff, teachers, funders, and evaluators teamed up to demonstrate the impact of two school-based arts programs on children and youth. Part of a broader initiative by a community foundation to strengthen youth arts in a Wisconsin metro area, this evaluation seeks to show that the two arts programs 1) increased the school and civic engagement of youth and 2) increased young children's interest in and motivation to be involved in the arts. Both program evaluations were woven into the delivery of each program, employed a before-after design, and collected data via internet survey. Through this initiative, program staff will learn the skills needed to evaluate programs. Results will be used to educate the public about the value of arts education. An overview of the entire project will be offered, including a community needs assessment, asset mapping, and awareness-raising efforts augment this impact study. Findings will be available in summer, 2009. |
| Participatory Evaluation on an Educational Program: Challenges and Learnings |
| Poster Presentation 139 to be held in Sebastian Section J on Wednesday, Nov 11, 6:30 PM to 8:00 PM |
| Presenter(s): |
| Lorena dos Santos, Servióo Social da Industria, lorena.vilarins@sesi.org.br |
| Luiza Bazan, Servióo Social da Industria, luiza.bazan@sesir.org.br |
| Mario de Oliveira, Servióo Social da Industria, mrsilva@sesir.org.br |
| Miguel Fontes, John Snow Brazil, mfontes@johnsnow.com |
| Rodrigo Lara, John Snow Brazil, laro@johnsnow.com |
| Abstract: Objective: The Social Work of Brazilian Industry, the largest non-profit organization of the country, is implementing a cost-effectiveness and benefit evaluation on an educational program. The intention is to demonstrate how to measure the economic value of a social intervention. This kind of evaluation is quite rare in Brazil. Methods: A 6 step model of analysis has been built. It has been based on a highly participatory evaluation design. A logic model to determine the program theory, goals and indicators is the first step and the key to produce action, because it involves the project staff, the evaluation team and the decision-makers and engages them with the results. The model also regarded the context and decisions were taken about what would be the appropriate methodological rigor. Conclusions: A participatory evaluation which takes account of the institutional context helps to disseminate the findings, to improve decision-making process and to produce action. |
| The Influence of Referral Context on Progress in Treatment Test Results |
| Poster Presentation 139 to be held in Sebastian Section J on Wednesday, Nov 11, 6:30 PM to 8:00 PM |
| Presenter(s): |
| Sharon Ross, Founder's Trust, sross@founderstrust.org |
| Gibbs Kanyongo, Duquesne University, kanyongog@duq.edu |
| Abstract: The Parenting Skills Evaluation Scale (PES) is an instrument designed to measure within program progress for three parental skills development programs run by a treatment program provider for youth, adults and families in the Northeast. The parents participating in the program usually have a child involved in the juvenile justice system. Anecdotal evidence from the three program sites indicate differences in how parents are referred to the programs (either as a requirement or as an additional resource for help) - differences which impact the attitude of parents as they participate in the program. This study looked at the scores on the PES to determine if these differing referral contexts impact the test results and ultimately how progress in the program is measured. |
| Using Mixed Methods to Evaluate Mass Health Communication for Seniors and People With Disabilities in Three Languages |
| Poster Presentation 139 to be held in Sebastian Section J on Wednesday, Nov 11, 6:30 PM to 8:00 PM |
| Presenter(s): |
| McKenzie Oliver, University of California Berkeley, moliver@berkeley.edu |
| Carrie Graham, University of California Berkeley, clgraham@berkeley.edu |
| Elaine Kurtovich, University of California Berkeley, elainek@berkeley.edu |
| Linda Neuhauser, University of California Berkeley, lindan@berkeley.edu |
| Susan Ivey, University of California Berkeley, sivey@berkeley.edu |
| Abstract: Researchers at UC Berkeley used mixed-methods to evaluate a health communication intervention for seniors and people with disabilities (SPD) who are beneficiaries of California's Medicaid program, Medi-Cal. The intervention was a guidebook, called 'What are my Medi-Cal Choices?' that was created in four languages using participatory research. Qualitative evaluation included key informant interviews with professions who serve SPD and focus groups with beneficiaries. Using a randomized control trial design, a telephone survey was conducted with SPD beneficiaries to measure the guidebook's effect on beneficiary knowledge and understanding of their Medi-Cal choices. Both the qualitative and the telephone survey showed positive results. The guidebook increased both professional and beneficiary knowledge regarding their understanding of Medi-Cal choices. The telephone survey showed positive results in guidebook use, satisfaction, and knowledge gained. Mixed methods designs are a powerful way to evaluate health communication interventions in populations with limited literacy and limited English proficiency. |
| Assessing Outcomes Among Dual-System Welfare and Child Welfare Involved Families |
| Poster Presentation 139 to be held in Sebastian Section J on Wednesday, Nov 11, 6:30 PM to 8:00 PM |
| Presenter(s): |
| Sheridan Green, University of Denver, sheridan.green@du.edu |
| Julie Morales, University of Denver, julie.morales@du.edu |
| Robin Leake, University of Denver, robin.leake@du.edu |
| Abstract: A model of collaboration between the county-administered TANF (Temporary Assistance for Needy Families) system and child welfare to improve outcomes for children and families was funded by the U.S. Health and Human Services Administration, Children's Bureau in 2007. The model was implemented to help families involved in both systems simultaneously ('dual-system' families). The program model included collaborative case-planning; family assessment; interagency service delivery focusing on domestic violence, substance abuse and mental health screening and referral; job resource development; cross-system training; parent partner mentoring program; and development of data sharing technology. The evaluation study randomized program families into one of three intervention groups. Families were assessed and results were used for intervention and case planning, and to examine change over time. Initial findings suggest high levels of family stress, resource needs and moderate levels of perceived social support. Family functioning concerns include parent and child mental health, child education needs, and self-sufficiency issues. |
| Policy and Deliberation: Evaluating Health Policy Law Enforcement Roundtables |
| Poster Presentation 139 to be held in Sebastian Section J on Wednesday, Nov 11, 6:30 PM to 8:00 PM |
| Presenter(s): |
| Jeanette Treiber, University of California Davis, jtreiber@ucdavis.edu |
| Abstract: In 2008 the California Department of Public Health's Tobacco Control Program set out to conduct a series of roundtables with local law enforcement agencies in four different sites throughout California to explore local tobacco retail licensing implementation and enforcement challenges and solutions. In order to learn whether or not this format was going to be useful for advancing statewide health policy compliance and enforcement, the Public Health Department decided to evaluate the roundtable events. Rather than focusing on the results of the evaluation, which is still in progress (the events span over the period of more than one year), the purpose of this paper is to share evaluation challenges and preliminary findings to help reflect on the still emerging area of deliberation evaluation. The mixed methods approach used for this evaluation is informed by the literature on assessing policy change efforts and on deliberation. It has been quite formative in the roundtable events thus far. |
| A Collaborative Evaluation During Code Purple Chaos |
| Poster Presentation 139 to be held in Sebastian Section J on Wednesday, Nov 11, 6:30 PM to 8:00 PM |
| Presenter(s): |
| Merlande Petit-Bois, University of South Florida, mpetitbo@mail.usf.edu |
| Gwen Rogerson, University of South Florida, gwen.rogerson@lrmc.com |
| Ryan Lee, University of South Florida, ryan.lee@lrmc.com |
| Abstract: This poster focuses on two parts of our collaborative evaluation, television and the color purple. This television system is very unique. It performs normal television functions, and much more! Additionally, health education videos on numerous topics are available any time of the day and can be viewed by patients, family members, friends and/or employees. The color purple is actually code purple in this evaluation and takes place in a hospital. However, code purple can take place anytime, anywhere, any evaluation! Code purple means key stakeholders are extremely busy and need to be convinced of the importance of making time to collaborate and participate in the evaluation. What happens when you need to conduct your collaborative evaluation during a code purple? How do you get buy-in? This presentation will give new meaning to the word television and the color purple. |
| Teachers' Perceptions of How to Teach Children of Poverty |
| Poster Presentation 139 to be held in Sebastian Section J on Wednesday, Nov 11, 6:30 PM to 8:00 PM |
| Presenter(s): |
| Pei-Yi Ou Yang, University of South Carolina, teresa386672@yahoo.com.tw |
| Abstract: The purpose of this study is to learn how pre-service teachers view the knowledge, skills, and dispositions they need to have to teach students living in poverty, and how they perceive the effectiveness of the Teacher Education Program of teaching children of poverty on their practice teaching. To obtain answers to the researcher questions in the study, a mixed method approach that combines analyses of quantitative data from a questionnaire with qualitative data from interviews as well as document analysis will be used. Participants will be pre-service teachers enrolled in the Teacher Education Program of teaching children of poverty at Francis Marion University. Findings, implications, and limitations of the study will be discussed. |
| Evaluating the Effectiveness of Academic Advisement: Does Departmental Advisement Meet its Proposed Goals? |
| Poster Presentation 139 to be held in Sebastian Section J on Wednesday, Nov 11, 6:30 PM to 8:00 PM |
| Presenter(s): |
| Seth Berry, Southern Illinois University at Carbondale, sethb85@siu.edu |
| Alen Avdic, Southern Illinois University at Carbondale, alen@siu.edu |
| Josh Doerner, Southern Illinois University at Carbondale, jdoerner@siu.edu |
| Daniel Nguyen, Southern Illinois University at Carbondale, danpsy@siu.edu |
| Abstract: The current model of advisement that exists throughout a medium-sized Midwestern university is for students to visit their college advisers once a semester to plan and register for classes. However, a department has created a department-specific advisement program aimed at offering students a more detailed and more specialized advisement process. In addition to offering students a more fine-tuned advising process, this service also assists students in developing career goals and helping students prepare for graduate studies. Most importantly, this program is concerned with reaching a large volume of students. However, in order to accomplish this goal, the program needs increased resources. It is the goal of this evaluation, to demonstrate not only the effectiveness the program has in meeting program goals, but also that students that utilize the service are more satisfied with their coursework and perform better with the specialized advising. |
| An Assessment of the Merit of the Applied Research Consultants (ARC) Training Program From the Perspective of its Users |
| Poster Presentation 139 to be held in Sebastian Section J on Wednesday, Nov 11, 6:30 PM to 8:00 PM |
| Presenter(s): |
| Alen Avdic, Southern Illinois University at Carbondale, alen@siu.edu |
| Daniel Nguyen, Southern Illinois University at Carbondale, danpsy@siu.edu |
| Abstract: Applied Research Consultants (ARC) is a student-run consulting firm operating within the Applied Psychology graduate program at Southern Illinois University Carbondale. From the training standpoint, ARC has been evaluated previously; however, the attempts at assessing the effectiveness of the program did not incorporate ARC alumni. Previous evaluations looked at the current members and assessed the effectiveness of the program in isolation, that is, without any comparison criteria. To be able to determine the true value of this program, some standard of merit needs to be implemented. In other words, the true value of the ARC program is best established by comparing it to some standard of acceptable performance, which in this case will be ARC alumni's current employment positions. Based on the results of this evaluation, recommendations for improvement as well as potential differences in findings due to implementing standards of merit are discussed. |
| The Impact of Self-Determination and Self-Concept on the Literacy Skills and Transition Outcomes of Youth Who are Deaf and Hard of Hearing |
| Poster Presentation 139 to be held in Sebastian Section J on Wednesday, Nov 11, 6:30 PM to 8:00 PM |
| Presenter(s): |
| Amy Hebert, University of Arkansas at Little Rock, amhebert@ualr.edu |
| Abstract: The population of school-aged (3-17yrs) youth who are deaf and hard of hearing is an extremely heterogeneous group of about 968,000 individuals. This diverse group of youth has exhibited deficits in language acquisition, academic competence, literacy, and employment attainment. There is little evidence-based research related to transition outcomes with deaf and hard of hearing students and almost nil are studies related to levels of self-efficacy and literacy. This research analyzed how self-determination and self-concept constructs impact literacy and transition outcomes among deaf and hard of hearing students using the National Transition Longitudinal Study 2 (NTLS2) data. Findings from this study will enhance the knowledge regarding transition outcomes of students who are deaf and hard of hearing. Results will inform the development of evidence-based materials and resources, and guide future research practices analyzing levels of social efficacy and literacy among adolescents who are deaf and hard of hearing. |
| Implementation Evaluation of a 'Mediation' Approach in a Child Protection Caseworkers Team in Quebec City (Canada) |
| Poster Presentation 139 to be held in Sebastian Section J on Wednesday, Nov 11, 6:30 PM to 8:00 PM |
| Presenter(s): |
| Gilles Mireault, Quebec Youth Center, gilles_mireault@ssss.gouv.qc.ca |
| Serge Turgeon, Quebec Youth Center, serge.turgeon@ssss.gouv.qc.ca |
| Abstract: Mediation as a mean to resolve conflict is widely spread in areas such as political, commercial and marital dispute. Basically it involves a neutral third person trying to reconciliate antagonistic parties. If nobody agrees, the case is taken to court for judgement. The approach has also been experimented in child protection cases in the United states during the 90's with success. The Quebec Youth Centre, a child protection services agency in Canada, has experimented with a modified version of the mediation approach with its caseworkers. Even though the caseworkers can not pretend to be neutral, they must protect the child's best interests, they can use their unique position to influence the way the case is treated. Identifying each person needs, generating new solutions and focusing on consensual agreement among the persons involved, the caseworker get more collaboration and involvement in taking care of the child's protection needs. How and to which extent this can be implemented in a practitioners team is illustrated. |
| Using Qualitative Methods to Evaluate Outcomes: Focus Groups With Teen Parents |
| Poster Presentation 139 to be held in Sebastian Section J on Wednesday, Nov 11, 6:30 PM to 8:00 PM |
| Presenter(s): |
| Holli Slater, University of Texas at Arlington, holli.slater@mavs.uta.edu |
| Diane Mitschke, University of Texas at Arlington, dianemitschke@uta.edu |
| Abstract: Focus groups were conducted to qualitatively assess the effectiveness of a comprehensive, curriculum-directed model aimed at reducing repeat pregnancies and high school dropout among pregnant and parenting adolescents. A total of 27 pregnant and parenting adolescents participated in one of five focus groups and responded to questions regarding their experience in the program and their reactions to various components of the intervention model, including goal development, asset-building, individual and group counseling, and case management. A grounded theory approach was used to guide the study's development, and content analysis of verbatim transcriptions guided the subsequent unveiling of codes and themes. Analysis indicates that students had a positive perception of the program and felt privileged to have been able to participate. Participants felt that they benefited from working with a counselor in a one-on-one setting and felt that the program was instrumental in their academic success and in achieving individual goals. |
| One National Exam for Many Regional Contexts: The Case of Rio de Janeiro |
| Poster Presentation 139 to be held in Sebastian Section J on Wednesday, Nov 11, 6:30 PM to 8:00 PM |
| Presenter(s): |
| Ana Carolina Letichevsky, Cesgranrio Foundation, anacarolina@cesgranrio.org.br |
| Monica Barros, National School of Statistical Science, monica.barros@ibge.gov.br |
| Fatima Cunha, Cesgranrio Foundation, fatimacunha@cesgranrio.org.br |
| Thereza Penna-Firme, Cesgranrio Foundation, therezapf@uol.com.br |
| Abstract: The general objective of this paper is to explore the results of the evaluation of High School Students in Brazil that is done using an exam known by the acronym ENEM (National High School Exam). ENEM is currently used for quality evaluation of schools, university admissions and as an evaluation tool for entry-level jobs. In 2008, about 2,900,000 people took the exam, of which approximately 837,135 were high-school seniors. At application, participants fill out a complete questionnaire that describes their family and school environments. ENEM is an interdisciplinary exam, which intends to evaluate basic competencies in different areas. The objective of this article is twofold: to relate the environment indicators as revealed by the questionnaire with performance measures in the test through multivariate statistical models and to compare two different contexts that is the results in the state of Rio de Janeiro with the country indicators. |
| The South Carolina Cancer Alliance Process Evaluation: Visually Mapping the Planning and Implementation Phases |
| Poster Presentation 139 to be held in Sebastian Section J on Wednesday, Nov 11, 6:30 PM to 8:00 PM |
| Presenter(s): |
| Cathleen Jernigan, Bureau of Community Health and Chronic Disease, jernigcj@dhec.sc.gov |
| Virginie Daguise, Bureau of Community Health and Chronic Disease, daguisvg@dhec.sc.gov |
| Abstract: Visual mapping can be a useful tool for evaluators working with coalitions, not-for-profit organizations and community groups to assess functioning, structure and internal process, and programs and interventions at a glance. This poster has two purposes. First it will outline the process followed to collect data on the formation of the South Carolina Cancer Alliance (SCCA), transition from a planning phase to implementation phase and the development of the state Cancer Plan. Second, it will visually display the data collected in the form of two maps, the Planning Map and Implementation Map created in Microsoft Publisher. |
| The Iowa Tobacco Cessation Program Evaluation: Assessing Impacts of State-Funded Cessation Programs |
| Poster Presentation 139 to be held in Sebastian Section J on Wednesday, Nov 11, 6:30 PM to 8:00 PM |
| Presenter(s): |
| Disa Cornish, University of Northern Iowa, disa.cornish@uni.edu |
| Gene Lutz, University of Northern Iowa, gene.lutz@uni.edu |
| Abstract: Since 2002, the percentage of adult Iowans who are current smokers has decreased nearly 40%. Two state-funded tobacco cessation programs provide counseling and pharmacotherapy to Iowans who want to quit smoking. The goal of the multi-method evaluation of Iowa's tobacco cessation programming is to assess the implementation and impacts of cessation services in the state. The study design is quasi-experimental with multiple post-tests. Data sources include follow-up interviews with cessation participants at 3, 6, and 12 months following program entry; secret shopper calls to Quitline Iowa; chart review data from clinics implementing cessation programming. More than one year after the cessation services were implemented in the state, preliminary results suggest quit rates in the state of approximately 20% at 3, 6, and 12 months. The Iowa Tobacco Cessation Program Evaluation works to understand the outcomes of a public health program and the contextual issues that may influence those outcomes. |
| Evaluating Suicide Prevention Programs: What to Measure? |
| Poster Presentation 139 to be held in Sebastian Section J on Wednesday, Nov 11, 6:30 PM to 8:00 PM |
| Presenter(s): |
| Phil Rodgers, American Foundation for Suicide Prevention, prodgers@afsp.org |
| Abstract: If the purpose of suicide prevention programs is to prevent suicide, then the most valid measure of the effectiveness of these programs is suicide deaths. However, because of their relative infrequency, suicide deaths are not a reliable outcome for most suicide prevention programs. Therefore, program evaluators are forced to choose less valid measures of program effectiveness, but do these outcomes lead to reductions in suicide risk? This poster will address this 'measurement conundrum' by (1) providing a typology of outcomes for suicide prevention programs, (2) identifying the logical, theoretical, and empirical support-or lack of support-for outcomes commonly used in the evaluation of suicide prevention programs, (3) providing a checklist by which the relationship between program outputs, outcomes, and purpose may be assessed. |
| Lack of Capacity Coupled With Low Demand: Evaluation Issues in Russia |
| Poster Presentation 139 to be held in Sebastian Section J on Wednesday, Nov 11, 6:30 PM to 8:00 PM |
| Presenter(s): |
| Irina Sinelina, International Labour Organization, sinelina@ilo.org |
| Abstract: Problem statement: In a state which is dominated by "group" interests with a so called 'controlled democracy,' there is no genuine demand for objective, transparent, impartial assessment of worth or merit of policies, programs or activities. This diminishes opportunities for the development of a pool of professional world-class local evaluators. The formula 'Better evidence, better polices' does not apply here, because policies and programs seem to be determined on some other grounds of more or less voluntarist nature. What next: However, the current economic and financial crisis may actually create favorable conditions for professional evaluation and increase demand on such public interest driven tools as evaluation. The much needed, but painful modernization of policy making is yet to come and professional evaluation is one of the indispensable instruments of such modernization, provided the political will is there. |
| The ZingBall Experience: An Evaluation of a New Computer Game |
| Poster Presentation 139 to be held in Sebastian Section J on Wednesday, Nov 11, 6:30 PM to 8:00 PM |
| Presenter(s): |
| Julie Gloudemans, University of South Florida at Tampa, julie.gloudemans@gmail.com |
| Robert Fay, University of South Florida at Tampa, rfay@mail.usf.edu |
| Abstract: This poster will depict how the consumer checklist and participatory method was used to evaluate ZingBall. The evaluation was used to provide feedback to the developers of ZingBall, HyperFog Games, as they work to produce and distribute the best possible video game product to the online game-playing universe. In this evaluation, a consumer checklist was created and tested to ensure that every question was related to the purpose of the evaluation. It was examined with different games to determine defensible criteria in which to rate ZingBall. The game was then evaluated by participants (game players) to determine if it was enjoyable, interesting enough to purchase, and the appropriate price. The checklist also allows participants to provide recommendations for game improvement. |
| A Quantitative Evaluation of a Nutrition Education Program With Low-income Youth |
| Poster Presentation 139 to be held in Sebastian Section J on Wednesday, Nov 11, 6:30 PM to 8:00 PM |
| Presenter(s): |
| Olive Monalisa Kemirembe, Pennsylvania State University, ozk102@psu.edu |
| Rama Radhakrishna, Pennsylvania State University, brr100@psu.edu |
| Elise Gurgevich, Pennsylvania State University, eliseg@psu.edu |
| Abstract: The purpose of the study was to evaluate whether an afterschool nutrition education program using select lessons from the Up for the Challenge: Health, Fitness, and Nutrition curriculum can change knowledge, attitude and behavior about nutrition with low-income youth in an after-school setting. A quasi-experimental design consisting of pretest-posttest equivalent comparison control group was used. A total of 86 students (43 in treatment and 43 in control) participated in the study. Hands-on nutrition education lessons were taught to youth in the treatment group for two-three hours every other week over a four-week period. Results showed that youth who received nutrition education using lessons from Up for the Challenge curriculum significantly improved their 1) nutrition knowledge, 2) attitudes towards nutrition, fruits and vegetables, 3)eating healthy and making healthy food choices, and 4) nutrition behaviors than those who did not participate in the lessons. |
| It's Choice, Not Chance That Determines Your Destiny: How Do Evaluators Make Choices for Differing Organizational Contexts? |
| Poster Presentation 139 to be held in Sebastian Section J on Wednesday, Nov 11, 6:30 PM to 8:00 PM |
| Presenter(s): |
| Marisa Allen, Case Western Reserve University, marisa.allen@case.edu |
| Abstract: How do evaluators specifically deal with the context of organizations? This paper presents empirical research on the extent to which evaluators recognize readiness for organizational learning and how this recognition relates to their choices of evaluation design and recommendations for evaluation use. Results from a recent web-based survey of 314 evaluators are presented and the implications for evaluation practice are discussed. Quantitative analyses are used to explain differences in responses from a sample of AEA members. The paper will discuss how evaluator characteristics are related to their ability to perceive readiness for organizational learning and examine relationships between evaluator characteristics, readiness for organizational leaning and recommendations for evaluation design and use. It presents an integrated model that helps explain the relationship between evaluator and organizational characteristics and their impact on evaluation use. |
| Evaluation and Its Possible Use in Georgia |
| Poster Presentation 139 to be held in Sebastian Section J on Wednesday, Nov 11, 6:30 PM to 8:00 PM |
| Presenter(s): |
| Ketevan Chomakhidze, Georgian Evaluation Association, kchomakhidze@evaluation.org.ge |
| Abstract: To explore to what extend the Georgian evaluation sphere is ready to fulfill potential demand and requirements in the field of evaluation Association has conducted the survey. The survey had following objectives: 1) Identify the human resources and institutions in the field of evaluation; 2) Reveal the problems, prioritize directions and needs in the field of evaluation; 3) Develop the recommendations and comprehensive report about the research findings. Up to the 81 respondents, representatives of governmental, non-governmental, private and international organizations operating in Georgia have been inquired. Recommendations how to promote the establishment of the evaluation field in Georgia have been developed. |
| Evaluation of Professional Development in Schools: Response to Intervention (RtI) Training Among Coaches and School Psychologists |
| Poster Presentation 139 to be held in Sebastian Section J on Wednesday, Nov 11, 6:30 PM to 8:00 PM |
| Presenter(s): |
| Kevin Stockslager, University of South Florida, kstocksl@mail.usf.edu |
| Amanda March, University of South Florida, amandamarch8@hotmail.com |
| Abstract: This poster highlights an evaluation of professional development utilizing fundamentals of both the management-oriented and participant-oriented approaches. Professional development was offered to school psychologists and Response to Intervention (RtI) coaches to enhance their ability to utilize an RtI service delivery framework within a central Florida school district. Recent national policies, such as the No Child Left Behind Act (NCLB) of 2002 and the Individuals with Disabilities Education Act (IDEIA) of 2004, have mandated the use of an RtI framework of service delivery in order to enhance educational outcomes for all students. Since the RtI framework is relatively new to school educators, a significant amount of training is necessary for school staff to implement RtI practices. This poster illustrates the evaluation design, procedures employed, and outcomes used to formatively enhance the RtI-related professional development offered to school psychologists and coaches within this Florida school district. |
| Using Quadrant Charts as a Program Improvement Tool |
| Poster Presentation 139 to be held in Sebastian Section J on Wednesday, Nov 11, 6:30 PM to 8:00 PM |
| Presenter(s): |
| Paul Pope, Texas AgriLife Extension Service, ppope@aged.tamu.edu |
| Abstract: Quadrant analysis is a business tool for identifying drivers of customer satisfaction associated with a service or product. This poster focuses on adapting quadrant analysis for use with Extension educational programs, and how they can be used for program improvement purposes. Real examples will be highlighted. In quadrant analysis, satisfaction with various aspects of an Extension educational activity, including content and delivery, is used to identify: 1) drivers of overall satisfaction with the program and 2) areas of opportunity that, if improved upon, should increase overall satisfaction. Two attributes of individual elements are plotted: performance (mean score) and importance (correlation with overall satisfaction with an event). With all of the elements plotted, a quadrant chart with four distinct areas is created using the mean performance and importance scores. Each element falls into one of the four quadrants- each having its own meaning and prescription for action to improve future events. |
| EZAnalyze: A Simple, Affordable Analysis Tool for County Extension Agents |
| Poster Presentation 139 to be held in Sebastian Section J on Wednesday, Nov 11, 6:30 PM to 8:00 PM |
| Presenter(s): |
| Paul Pope, Texas AgriLife Extension Service, ppope@aged.tamu.edu |
| Abstract: This poster highlights EZAnalyze and how it has helped County Extension Agents be more efficient in tabulating and analyzing the results of their educational programs. With EZAnalyze, agents can now use Excel for both data entry and analysis. EZAnalyze is a free, add-in to Excel. It generates the most common, basic tabulations needed by agents including frequencies and descriptive statistics. Its point-and-click interface eliminates the need to define ranges and create formulas in Excel. EZAnalyze also allows for basic data manipulations such as difference variables, summary variables, and percentage change, and common inferential tests that most agents would need, such as chi-square, t-tests, and ANOVA. Along with its general capabilities, emphasis will be given to applications of this tool in Extension field work. How an agent can use EZAnalyze to process retrospective post evaluation data will be shown. |
| Measuring Technology Literacy Proficiency: How Evaluation Facilitated the Process in Maryland |
| Poster Presentation 139 to be held in Sebastian Section J on Wednesday, Nov 11, 6:30 PM to 8:00 PM |
| Presenter(s): |
| Shauna Clarke, ICF Macro, shauna.s.clarke@macrointernational.com |
| Edith Stevens, ICF Macro, edith.s.stevens@macrointernational.com |
| Abstract: Defining, measuring and reporting on the technology literacy proficiency of students, teachers and administrators are tasks that all states confront. It can be a daunting task to take the No Child Left Behind mandates on technology literacy and transform them into a logical process that produces tangible standards as well as data that not only address the requirements of the law but also meet the needs of all of the stakeholders involved. The collaborative approach used in Maryland which involves staff from local school districts; representatives from the State Department of Education; and Macro International, the third party evaluator, can serve as a practical and valuable model for other states that need to achieve objectives related to technology literacy. This poster session will present the dynamic role that the evaluator, Macro International has played in this process. |
| Tolerance of Ambiguity May Be Associated With Attitude Change in Medical Students |
| Poster Presentation 139 to be held in Sebastian Section J on Wednesday, Nov 11, 6:30 PM to 8:00 PM |
| Presenter(s): |
| Sharon Wayne, University of New Mexico, swayne@salud.unm.edu |
| Summers Kalishman, University of New Mexico, skalish@salud.unm.edu |
| Lisa Serna, University of New Mexico, lserna@salud.unm.edu |
| Roger Jerabek, University of New Mexico, rjerabek@salud.unm.edu |
| Abstract: Introduction: Research suggests student attitudes toward underserved populations become more negative during medical school. Providing care for the underserved can involve uncertainty or ambiguity. We investigated whether tolerance of ambiguity was associated with change in attitudes toward the underserved. Methods: An attitude survey with four subscales was administered at matriculation and graduation to six consecutive medical school classes (2003-2008) at the University of New Mexico. 255 students completed both surveys (59% response rate). Tolerance of ambiguity was measured at matriculation. Results: Student attitudes toward underserved populations declined over the four years of medical school; the professional responsibility subscale had the largest decrease, 11%. Linear regression models showed intolerance of ambiguity was significantly associated with a drop in score for all subscales after controlling for gender, age and ethnicity. Conclusion: Helping students deal with ambiguous situations in medical education or medical practice may moderate decreases in attitudes toward underserved populations. |
| Assessing the Long-term Impact of Meeting Nobel Prize Winners |
| Poster Presentation 139 to be held in Sebastian Section J on Wednesday, Nov 11, 6:30 PM to 8:00 PM |
| Presenter(s): |
| Erin Burr, University of Tennessee, erinburr1@gmail.com |
| Samuel Held, Oak Ridge Institute for Science and Education, sam.held@orau.org |
| Abstract: Results are presented from a survey of program alumni who attended a conference with fellow U. S. graduate students and other young researchers from around the world. The only speakers at this conference are Nobel Laureates, who have extensive contact with the students. Can a week with other US graduate students, international researchers, and the Laureates have a long-term impact on the participants? Graduate students have received funding to attend this Meeting since 2000 from federal and private funds. Meeting attendees complete a survey immediately after the meeting about the logistics and the anticipated impact. The follow-up survey measures contacts with other meeting alumni; documents personal, educational, and career impacts; degree obtainment; current career, and ties to federal funding agencies to verify anticipated impact. Contact with participants was maintained through the use of annual email messages and Facebook. With hundreds of respondents, we had representative populations from each meeting. |
| Evaluating a Program That Disseminates Research-based Education Materials |
| Poster Presentation 139 to be held in Sebastian Section J on Wednesday, Nov 11, 6:30 PM to 8:00 PM |
| Presenter(s): |
| Genevieve deAlmeida-Morris, National Institutes of Health, dealmeig@mail.nih.gov |
| Abstract: In September 2003, the 'NIDA Goes Back to School"(NGBTS) dissemination campaign was launched to promote the National Institute on Drug Abuse (NIDA), National Institutes of Health,as the source for science-based information on drug abuse and prevention for school-aged children. Dissemination and use of research results is a critical component of the Institute's mission--Executive Order 12862 directs federal agencies to provide significant services directly to the public. The materials in the NGBTS campaign are for students in grades K-12. They teach children about drug abuse consequences and encourage interest in biomedical science. The number of requests for the materials each year has been phenomenal; however a feasibility study for an impact evaluation identified school-specific barriers to access and use of the materials. Thus, it was necessary to examine the actual use of the materials and the system barriers to their use, before evaluating the national impact of the program. |
| Evaluation of an Urban Wildlife Education Program in High Schools Across the United States |
| Poster Presentation 139 to be held in Sebastian Section J on Wednesday, Nov 11, 6:30 PM to 8:00 PM |
| Presenter(s): |
| Jessica Tegt, Mississippi State University, jtegt@cfr.msstate.edu |
| Ben West, Mississippi State University, benw@cfr.msstate.edu |
| Abstract: Research indicates that High School aged adolescents are the least informed about urban wildlife issues. Numerous programs have been created as a supplement to existing science programs informing students about urban wildlife, but have fallen short in increasing knowledge or assessing student attitudes. We evaluated a different style of urban wildlife education program in classrooms across the eastern and mid-western United States where white-tailed deer are prevalent. The program titled, 'Living with White-tailed Deer' allows students to become stakeholder groups and conduct research regarding a hypothetical town overpopulated with deer. The students are able to conduct a mock-town meeting following the research to reach consensus on the management of the deer. Evaluation was administered to students using pre- and post-test attitude and knowledge surveys. We also considered demographic variables in the evaluation. Teachers were asked to evaluate the educational soundness of the program using the NAAEE Guidelines for Excellence as a model assessment. Preliminary results indicate that students and teachers alike responded favorably to the program in both teachability and increased knowledge. |
| Evaluating Youth Mentoring: A Randomized Control Trial Study of the Amachi Texas Program |
| Poster Presentation 139 to be held in Sebastian Section J on Wednesday, Nov 11, 6:30 PM to 8:00 PM |
| Presenter(s): |
| Aracelis Gray, ICF International, agray@icfi.com |
| Heather Clawson, ICF International, hclawson@icfi.com |
| Jing Sun, ICF International, jsun@icfi.com |
| Byron Johnson, Baylor University, byron_johnson@baylor.edu |
| Abstract: Children with a parent behind bars represent one of the fastest growing at-risk populations in the country. The Amachi Texas program aims to break the inter-generational cycle of crime and incarceration by matching children with incarcerated parents with a caring adult mentor. To evaluate the program impacts, ICF International implemented a three-year randomized control trial (RCT) study. Eligible children were randomly assigned to a treatment group - in which children receive a mentor - or to a control group - in which children are not granted access to a mentor for up to 18-months. Pretest and posttest surveys were administered to children, parents, and mentors to measure children's attitudes toward school, relationships with family and peers, prosocial behavior, and academic engagement. We will present the preliminary results and discuss the implications for mentoring programs and the youth development field in the context of implementation. |
| Cost-effectiveness of Supervised and Self-administered Treatments for Tuberculosis in HIV+ patients |
| Poster Presentation 139 to be held in Sebastian Section J on Wednesday, Nov 11, 6:30 PM to 8:00 PM |
| Presenter(s): |
| Pedro Paulo Chrispim, Laboratório de Avaliaįão de Situaįões Endémicas Regionais, chrispim@ensp.fiocruz.br |
| Luisa Oliveira, Universidade Federal Fluminense, luisa.oliveira@ensp.fiocruz.br |
| Adriana Mendoza, Nucleo de Assisténcia Farmacéutica, amenru@yahoo.com.br |
| Abstract: Eight Health Facilities in Rio de Janeiro municipality were visited looking for co-infected HIV/TB patients in a retrospective cost-effectiveness study on medical records ranging from 1999 to 2005. Of 635 confirmed cases, 532 were treated for TB infection under self-administered (SA) regimen and 103 under Supervised Treatment (ST) regimen. One hundred ninety-six were men (average age of 41 years) and 439 women (a.a. 40 years). The cure rate was found to be 62.2% for patients under SA and 69.9% for patients under ST. This difference in cure rates is not statistically significant. The abandon rate was found 22.3% and 12.8% for SA and ST regimens, respectively. Under Tuberculosis Control Program perspective, cost per cured case was US$269.22 among patients under ST and US$ 111.16 for ones under SA (US$1 = R$ 2.24). Incremental cost for ST regimen was estimated in US$ 2,195.05 per cured case. |
| The Preparing Parents for Success Project: Evaluation of a Secondary Prevention Program |
| Poster Presentation 139 to be held in Sebastian Section J on Wednesday, Nov 11, 6:30 PM to 8:00 PM |
| Presenter(s): |
| Diane Sasser, Louisiana State University, dsasser@agcenter.lsu.edu |
| Linda Robinson, Louisiana State University, lrobinson@agcenter.lsu.edu |
| Courtney Pitts, Louisiana State University, cpitts@agcenter.lsu.edu |
| Abstract: The Parents Preparing for $uccess Project is secondary prevention program for pregnant women or parents of infants participating in the Temporary Assistance to Needy Families program. We aim to equip our program participants with the necessary tools to become self-sufficient while transitioning from public assistance into the workforce. PP$P is offered through Cooperative Extension offices throughout Louisiana, serving approximately 900 families a year. The overall goal of PP$P is help prepare low income parents and their infants for success by increasing decision-making skills, positive coping skills, and positive parenting skills. PP$P is offered through Cooperative Extension offices throughout Louisiana, serving approximately 900 families a year. Program goals are evaluated quantitatively and qualitatively using pre- a post tests, brief class evaluations, and a follow-up interview three months after completion of the program. |
| Perceptions and Approaches to Innovation: A Needs Assessment of the Centers for Chemical Innovation Using Q-sort Methodology |
| Poster Presentation 139 to be held in Sebastian Section J on Wednesday, Nov 11, 6:30 PM to 8:00 PM |
| Presenter(s): |
| Ricardo Gomez, University of Massachusetts Amherst, rgomezye@educ.umass.edu |
| Abstract: As fellow of the "American Evaluation Association /Duquesne University Graduate Education Diversity Internship Program," I have been commissioned by the National Collegiate Inventors & Innovators Alliance (NCIIA) to implement a needs assessment of the Centers for Chemical Innovation (CCI). The purpose of this needs assessment is to serve as the basis for developing and implementing a training program to provide CCI faculty and students with the information and know-how needed for research to innovation (R2I) translation. Since the specific objective of the assessment is 'to explore attitudes, perceptions, and beliefs regarding research to innovation translation from the perspective of CCI key participants ['],' this study uses the Q-sort methodology as the primary method of inquiry. I have chosen this approach because Q-sort can provide a deeper understanding of the opinions, beliefs, perspectives, decision structures, frames or narratives of individuals on any topic that has a subjective component. |
| Modified Affinity Grouping and Collaborative Scaling as a Capacity Building Tool and a Means of Moving Beyond a 'Ceiling Effect' in a Basic Compliance Measure |
| Poster Presentation 139 to be held in Sebastian Section J on Wednesday, Nov 11, 6:30 PM to 8:00 PM |
| Presenter(s): |
| Sharon Brown, Cleveland State University, s.a.brown54@csuohio.edu |
| Abstract: Reading First Ohio (RFO) is a multisite school reform initiative with significant levels of local autonomy in program design and implementation. Although there are common elements at the compliance levels, the RFO evaluation and formative measures offered only minimal guidance for sites more deeply engaged. (Reading First, 2003) The affinity grouping/collaborative scale building process was completed with a group of change agents who serve in a technical assistance/accountability role in RFO, additionally it was conducted at individual sites with local staff. The process provided opportunity to quickly review efforts, develop common language, and create a common scale for programmatic progress. Further, it allowed the evaluators to gain a perspective on local priorities, strength and challenges in a non-threatening manner such that formative measures were ultimately more tailored and effective. This simple process allowed local context to shape formative efforts while summative measures remained appropriately removed. |
| Observing Secondary Mathematics and Science Teachers' Assessment Strategies: An Evaluation of Six Classroom Observation Protocols |
| Poster Presentation 139 to be held in Sebastian Section J on Wednesday, Nov 11, 6:30 PM to 8:00 PM |
| Presenter(s): |
| Jonathan Engelman, Western Michigan University, jonathan.a.engelman@wmich.edu |
| Amy Bentz, Western Michigan University, amy.e.bentz@wmich.edu |
| Abstract: This descriptive study evaluates six observation protocols that are commonly utilized to assess teacher performance. The researchers in the study are specifically interested in the extent to which the protocols attain to formative assessment within mathematics and science classrooms. Assessment for Learning (AfL) principles act as the framework through which each protocol is evaluated. The six protocols are: Inside the Classroom Observation and Analytic Protocol (Horizon Research, Inc.), Chicago Math &Science Initiative Classroom Observation Guide, STEMTEC Core Evaluation Classroom Observation Protocol, the Susuwele-Banda Classroom Observation Protocol, the McManus Classroom Observation Protocol, and the Wide-scale Interactive Development for Educators (WIDE) Classroom Observation Protocol. |
| External Evaluation of Gaining Early Awareness and Readiness for Undergraduate Programs(GEAR UP) North Carolina - The SERVE Center at University of North Carolina Greensboro |
| Poster Presentation 139 to be held in Sebastian Section J on Wednesday, Nov 11, 6:30 PM to 8:00 PM |
| Presenter(s): |
| Alexa Edwards, University of North Carolina at Greensboro, aedwards@serve.org |
| Finney Pamela, University of North Carolina at Greensboro, pfinney@serve.org |
| Tillery Chrissy, GEAR UP North Carolina, tillerycy@appstate.edu |
| Abstract: As the economy continues to take a toll on American families, the LEAs and SEAs are taking a deeper look into how to utilize academic outreach programs such as GEAR UP (Gaining Early Awareness and Readiness for Undergraduate Programs) as a vehicle of support for low income students and parents to help increase their awareness of postsecondary educational opportunities and the preparation needed to pursue postsecondary education. This poster presentation will share the external evaluation design of the GEAR UP North Carolina program and describe how other evaluators can begin to think about the most effective ways of evaluating this national program. In addition, ideas will be shared that can contribute to evaluation as a discipline and practice as well as for advancing the evaluation activities of nationally funded pre-college programming. |
| Evaluating Problem Solving via an Electronic Classroom Response System: Active Learning in a High-Enrollment, High-Impact, and Core Engineering Course |
| Poster Presentation 139 to be held in Sebastian Section J on Wednesday, Nov 11, 6:30 PM to 8:00 PM |
| Presenter(s): |
| Margaret Lubke, Utah State University, mlubke@ksar.usu.edu |
| Ning Fang, Utah State University, nfang@engineering.usu.edu |
| Abstract: Many university students perform poorly in, or even fail, Engineering Dynamics, because they lack solid problem-solving skills. This decreases student retention, a major nationwide concern in Science, Technology, Engineering, and Mathematics education. During the 2008-2009 academic year, this project is evaluating whether an Electronic Classroom Response System (Student Clickers), a radio-frequency, two-way communication system, used in conjunction with multiple cognitive learning levels questions improves students' active learning and problem-solving skills. Both quantitative and qualitative analyses, including data from classroom observations, focus groups, and student surveys will determine if there is a relationship between the effectiveness of clicker questions and students' problem solving. The Student Clickers will also be used to gather focus group and survey information. In addition, a mathematical decision-tree analysis will be conducted to derive a set of explicit if-then rules that relate cognitive and non-cognitive factors to student problem-solving skills. |
| Evaluating an Innovative Online Tool for Promoting Sexually Transmitted Infection Screening |
| Poster Presentation 139 to be held in Sebastian Section J on Wednesday, Nov 11, 6:30 PM to 8:00 PM |
| Presenter(s): |
| Sheetal Malhotra, Medical Institute for Sexual Health, sheetal.malhotra@gmail.com |
| Abstract: The STD Wizard, an innovative online sexually transmitted infection (STI) risk assessment tool, was evaluated in college students using an intervention/nonintervention, pre-test/post-test design. 468 students from 52 undergraduate classes in two universities completed an online pre-test and were randomly assigned to an STD Wizard intervention or a nonintervention group. Post-tests were administered to all students at two weeks and two months. Results: 292 (62%) females and 176 (38%) males participated in the study. 100% of the intervention group found the STD Wizard easy to use; most (81%) also found it useful. 45% (97/216) of the students who went through the STD Wizard received screening recommendations. 57% (55/97) of students who received recommendations had seen (21%) or intended to see (36%) a doctor in the near future. Conclusion: The STD Wizard evaluation suggests that it is an easy-to-use and effective online tool for the promotion of STI screening. |
| Collaboration Isn't For Wimps: Perspectives on the Benefits and Challenges of Collaborative/Participatory Evaluations |
| Poster Presentation 139 to be held in Sebastian Section J on Wednesday, Nov 11, 6:30 PM to 8:00 PM |
| Presenter(s): |
| Kathryn Rouse, Consumer Quality Team of Maryland, krouse@cqtmd.org |
| Joanne Creaney Meekins, Consumer Quality Team of Maryland, jmeekins@cqtmd.org |
| Diana Seybolt, University of Maryland, dseybolt@psych.umaryland.edu |
| Amanda Jones, University of Maryland, amjones@psych.umaryland.edu |
| Abstract: In 2007, consumers of mental health services and family members launched the Consumer Quality Team (CQT) of Maryland as a quality assurance initiative to help solve problems in the state's mental health system. CQT's program activities take place in a web of collaborative relationships with providers, consumers and state agencies. At CQT's request, the University of Maryland conducted an evaluation to identify strengths and weaknesses of program implementation. In this presentation, members of CQT and University of Maryland discuss how the experiences of both organizations led to the development of a collaborative evaluation of the program. Representatives from CQT will share their perspectives on the challenges and benefits of this type of relationship from their unique position as participants on both ends of this process. University of Maryland evaluators will discuss how their collaborative relationships with CQT staff and their evaluation methods evolved over time to produce useful results. |
| Linking Medicaid and State Birth Records to Participants in a California Family Planning Program: Unbiased Estimates of Probabilistic Linking Error |
| Poster Presentation 139 to be held in Sebastian Section J on Wednesday, Nov 11, 6:30 PM to 8:00 PM |
| Presenter(s): |
| Denis Hulett, University of California San Francisco, denis.hulett@cdph.ca.gov |
| Jaycee Karl, University of California San Francisco, jaycee.karl@cdph.ca.gov |
| Lewis Carrie, University of California San Francisco, carrie.lewis@cdph.ca.gov |
| Philip Darney, University of Central Florida, darneyp@obgyn.ucsf.ca.gov |
| Heike Thiel de Bocanegra, University of Central Florida, heike.thiel@cdph.ca.gov |
| Abstract: As part of the evaluation of California's publicly funded family planning program, Family PACT (Family Planning, Access, Care and Treatment), the number of births to mothers who are Family PACT clients are counted. To identify these births, Family PACT client enrollment records must be linked to California's Birth Statistical Master File (BSMF). To further determine which of these births are publicly funded, a link to Medi-Cal's Eligibility Data System (MEDS) is also required. Because these disparate data sources do not share a single reliable common record identifier, a probabilistic linking model was developed to link client records across files. The linking process used enrollment and birth records from calendar years 2000-2007, consisting of over 8 million Family PACT records, 500 million Medi-Cal records, and 4 million birth records. Over 40% of the Family PACT enrollees were linked to either the MEDS or BSMF using a set of variables common across files to create a probabilistic linkage score and link/no-link decision rules. Although social security numbers are present in each database, values are missing among a majority of the records. As such, the potential for matching error is substantial. Utilizing a unique application of logistic regression and stratified sampling to create a 'gold-standard' file of known results, unbiased estimates of error are calculated and their sources are isolated to optimize linkage processes. The linkage sensitivity ranged between 0.92 and 0.999, and specificity ranged from 0.94 to 0.98. The methodology allows investigators to determine the magnitude and direction of error that is appropriate for their needs. |
| Propensity Score Matching for the Evaluation of a Federal Teacher Incentive Fund Grant |
| Poster Presentation 139 to be held in Sebastian Section J on Wednesday, Nov 11, 6:30 PM to 8:00 PM |
| Presenter(s): |
| Holli Bayonas, University of North Carolina at Greensboro, hbayonas@serve.org |
| Eric Howard, University of North Carolina at Greensboro, ehoward@serve.org |
| Abstract: The Mission Possible (MP) program is a 5-year federally funded Teacher Incentive Fund grant program where 25% of Guilford County Schools (GCS) participate. Select teachers in hard to fill position are given a recruitment incentive and also a performance incentive when their students make gains in achievement (measured by SASŪ Value-Added). Experimental design was not possible within the county because schools that would be an ideal match within GCS were too few. Using propensity score matching (PSM) and publicly available data, comparison schools were selected across North Carolina. In the Year 2-5 evaluation reports, these comparison schools will be essential to see if trends witnessed in the MP schools are treatment effects or if these trends are shared by the comparison schools. This presentation will provide background about the evaluation, the evaluation plan, the PSM, and how the results will be visually displayed for easy review by school board members. |
| Adolescent Self Reports: An Evaluation of a Nonprofit Program Targeting Males Prone to Risky Sexual Behavior |
| Poster Presentation 139 to be held in Sebastian Section J on Wednesday, Nov 11, 6:30 PM to 8:00 PM |
| Presenter(s): |
| Lauren Fluegge, University of North Carolina at Greensboro, lbfluegg@uncg.edu |
| Abstract: Surveys are of critical importance to evaluation, but the reliability of those self-reports when adolescents are those reporting can be problematic. A nonprofit program designed to educate at-risk adolescent males on sexuality and the treatment of women was evaluated to assess the program's impact on the adolescents' perceptions and behaviors. The targeted population was adolescents who were especially prone to risky sexual behavior. They were enrolled on the recommendation of school personnel or enrolled by parents, and the program was offered both in school and community settings. Of specific interest in the evaluation were the adolescents' self-reports of sexual behavior and behavior toward women before the program and after completion. Reflective of previous literature on adolescent self-reporting behaviors, the reliability of the self-reports of sexual activity was called into question. A discussion of how to approach adolescent self-reports is included. |
| Rapid Ethnography as Community-Based Participatory Research: Research for Real World Settings |
| Poster Presentation 139 to be held in Sebastian Section J on Wednesday, Nov 11, 6:30 PM to 8:00 PM |
| Presenter(s): |
| Kathleen Ferreira, University of South Florida, kferreira@fmhi.usf.edu |
| Sharon Hodges, University of South Florida, hodges@fmhi.usf.edu |
| Tracy-Ann Gilbert-Smith, University of South Florida, tgsmith@fmhi.usf.edu |
| Abstract: Community-based participatory research is described as a collaboration between researchers and participants, in which community stakeholders are engaged as members of the research team. The utilization of this type of research model within the field of mental health has broadened over the last few years, particularly in relation to community change efforts. Rapid ethnographic methods are often used by a research team to collect a large amount of data within a short period of time. This poster presentation will describe the integration of community-based participatory research and rapid ethnographic methods in the study of the development of six established system-of-care communities examined as part of Case Studies of System Implementation, a five-year, federally-funded research project through the Research and Training Center for Children's Mental Health. Site selection, data collection and analysis, reporting of findings, and the key role community stakeholders played in the research process will be described. |
| The Electronic Scientific Portfolio Assistant (e-SPA): A Comprehensive Approach to Portfolio Analysis at the National Institute of Allergy and Infectious Diseases (NIAID) |
| Poster Presentation 139 to be held in Sebastian Section J on Wednesday, Nov 11, 6:30 PM to 8:00 PM |
| Presenter(s): |
| Brandie Taylor, National Institutes of Health, taylorbr@mail.nih.gov |
| Tarsha McCrae, National Institutes of Health, mccraet@mail.nih.gov |
| Kevin Wright, National Institutes of Health, wrightk@mail.nih.gov |
| Abstract: e-SPA is an innovative software tool designed to provide extramural program directors with the capability to analyze the performance of their research portfolios. The tool generates user-defined portfolios of NIAID research projects and links the projects to outcome indicators including funding, publications, citations, impact factors, inventions, and patents. e-SPA uses information available from multiple databases to enhance synthesis and analysis of relevant data, and provides data visualization capability through a dashboard and graphs. The development of e-SPA grew out of program needs to ensure accountability and transparency of information about program performance and results. This poster will highlight how e-SPA works to provide quantitative data for various metrics to assess research performance. Examples of ongoing program management, data sources, current limitations of the tool, lessons learned and next steps in the development of e-SPA will be shared as well as illustrations of portfolio summaries, key indicators, and cross-portfolio comparisons. |
| The Role of External Evaluation in Assisting An Urban Museum's Research on Scaffolding Strategies With Culturally Diverse Families |
| Poster Presentation 139 to be held in Sebastian Section J on Wednesday, Nov 11, 6:30 PM to 8:00 PM |
| Presenter(s): |
| Wendy Dickinson, Ringling College of Art and Design, wbdickinson@aol.com |
| Bruce Hall, University of South Florida, bwhall@tampabay.rr.com |
| Judith Lombana, Museum of Science and Industry, judithl@mosi.org |
| Doris Ash, University of California Santa Cruz, dorisbash@gmail.com |
| Abstract: The Museum of Science & Industry (MOSI) in Florida and the University of California Santa Cruz are partners in an NSF-sponsored study on successful scaffolding strategies for culturally diverse families engaged in museum-based science activities. Scaffolding is a process whereby learners, with strategic assistance, solve problems or achieve goals that would have been beyond their unassisted efforts (Wood, Bruner, & Ross, 1976). Two professional evaluators serve in an external capacity: responsibilities include observing and examining research activities in relation to project timelines, advising the PI's with respect to grant requirements, producing quarterly reports with recommendations, and assisting project personnel in communicating research results to appropriate audiences. This presentation explores the varying roles of external evaluators for complex, multi-leveled research involving interactions among culturally diverse families, museum educators, and science-based museum exhibitry. External evaluation as a means of enhancing project effectiveness is examined, and potential threats to evaluator objectivity contextualized. |
| Mental Model Diagramming for Needs Assessments: Lessons learned from information arictecture, user experience, and interaction design professionals |
| Poster Presentation 139 to be held in Sebastian Section J on Wednesday, Nov 11, 6:30 PM to 8:00 PM |
| Presenter(s): |
| Becky Melzer, Evaluation Edge LLC, becky@evaluationedge.com |
| James Melzer, SRA International Inc, jamesmelzer@gmail.com |
| Abstract: This poster will demonstrate how the the concept of mental model diagramming being used by information architects and user experience professionals in designing products such as web applications are similar and different from those diagrams or logic models used by evaluation specialists working in areas such as needs assessment. We hope to generate discussion at the poster event about how the fields can inform each other. |
| Implementing a Quality-Control Process for Evaluation in a Large Organization |
| Poster Presentation 139 to be held in Sebastian Section J on Wednesday, Nov 11, 6:30 PM to 8:00 PM |
| Presenter(s): |
| Kazuaki Uekawa, ICF International, kuekawa@icfi.com |
| Allan Porowski, ICF International, aporowski@icfi.com |
| Abstract: Large evaluation efforts with multiple analysts typically face challenges related to communications, duplication of effort, and working across multiple statistical analysis platforms. In response to these challenges, the authors of this presentation have implemented a corporate-wide quality control (QC) effort to ensure that standard data management, analytic, and reporting procedures are followed. The ultimate goal of our QC effort was to ensure that analysts were properly documenting their work, that the proper analytic procedures were being followed, and that all work was properly vetted internally before it is released to the public. Specific roles in the QC process were delineated to facilitate a coordinated set of actions by project managers, technical managers, data managers, and analysts. In this presentation, the study authors will describe the QC process that was developed, and will offer practical guidance on the application of QC procedures in a large organization. |
| Adaptive Evaluation Designs: Lessons Learned From Evaluating the Integration of Behavioral and Social Science Content Into a Medical School Curriculum |
| Poster Presentation 139 to be held in Sebastian Section J on Wednesday, Nov 11, 6:30 PM to 8:00 PM |
| Presenter(s): |
| Arthur Blank, Montefiore Medical Center, ablank@montefiore.org |
| William Burton, Albert Einstein College of Medicine, burton@aecom.yu.edu |
| Alice Fornari, Albert Einstein College of Medicine, afornari@aecom.yu.edu |
| Abstract: In 2005 the National Institutes of Health awarded 9 medical schools grants to incorporate health related content from the behavioral and social sciences into their medical school curricula. This evaluation sought to: (1) assess if incorporating behavioral and social science content into the curriculum changed medical students attitudes, skills, and knowledge, and (2) document the effort involved in incorporating the new curriculum content into the educational infrastructure. Initial results suggested the need to modify our research design without sacrificing but building upon our initial evaluation plan. We suggest that evaluators need to consider developing research designs and analytical strategies that can capture dynamic program change by adding-in a temporal manner-additional or replacement design and measurement elements. We use the term adaptive evaluation design to refer to both this idea and the designs and analytical strategies that need to be articulated and developed to address short-term and dynamic program shift. |
| A Historical Review of Evaluation Requirements in Federally Funded Post Secondary Education Projects From 1965 to 2009 |
| Poster Presentation 139 to be held in Sebastian Section J on Wednesday, Nov 11, 6:30 PM to 8:00 PM |
| Presenter(s): |
| Maxine Gilling, Western Michigan University, maxine.gilling@wmich.edu |
| Abstract: This study investigates evaluation requirements in federally funded post secondary education projects, from 1965 to 2009. While 'program evaluation has a long history as a practice it only became a recognized specialty in the United States in the 1960s' (Scriven, 1991, p. 285). Fitzpatrick, Sanders & Worthens (2004) state that, 'the one event that is most responsible for the emergence of contemporary program evaluation is the passage of the Elementary and Secondary Education Act (ESEA) of 1965' (p. 36). Moreover, ESEA also led to congressionally mandated evaluation requirements of Title I and Title III programs in 1965'. Since this time, procedures used to determine program effectiveness have undergone theoretical and technical changes. The focal points of this study will be illustrated in a historical matrix that maps the political and economic context, policy changes, and progression of evaluation requirements mandated by the Federal Government since 1965. |
| Overcoming Obstacles for Successful Evaluations |
| Poster Presentation 139 to be held in Sebastian Section J on Wednesday, Nov 11, 6:30 PM to 8:00 PM |
| Presenter(s): |
| Mary Ehmann, Pearson Corporation, mary.ehmann@pearson.com |
| Ann Vilcheck, Pearson Corporation, ann.vilcheck@pearson.com |
| Miriam Resendez, PRES Associates Inc, mresendez@presassociates.com |
| Mariam Azin, PRES Associates Inc, mazin@presassociates.com |
| Abstract: With continuing emphasis on evidence-based reforms in the United States, publishers are commissioning large-scale examinations of products. These evaluations offer many challenges, including limited budget, site recruitment, program fidelity, and timeline. A limited budget can affect such things as evaluation design, assessments, instrumentation, and travel. Successful site recruitment is important to ensure an adequate sample. Over the years, stakeholders have developed methods to make the recruitment process run more smoothly. Fidelity of program implementation is another potential obstacle. Teachers may be implementing new curriculum, and evaluators must document fidelity. Evaluation timeline is the final obstacle. Publisher's schedule and school contextual factors may disrupt material availability, implementation, data collection, and reporting. Armed with this knowledge, education stakeholders will be better prepared to perform a successful evaluation. |
| Measuring Local Economic Impact: Conducting an Input-Output Analysis for a State Health and Human Service Agency |
| Poster Presentation 139 to be held in Sebastian Section J on Wednesday, Nov 11, 6:30 PM to 8:00 PM |
| Presenter(s): |
| Natalie Towns, Georgia State University, ntowns@gsu.edu |
| Angela Synder, Georgia State University, angiesynder@gsu.edu |
| Bruce Seaman, Georgia State University, ecobas@langate.gsu.edu |
| John Matthews, Georgia State University, prcjwm@langate.gsu.edu |
| Abstract: During a state budget deficit, political leaders must make difficult decisions to cut programs serving low income children and families. Staff in a state health and human service agency decided that an evaluation was needed that would illustrate the agency's impact on the economic well-being of the local community, instead of an evaluation focused on measuring outcomes of clients served. University evaluators conducted an economic impact analysis for the agency using input-output modeling. The benefit of this type of economic evaluation is that it can be done relatively quickly using existing software; however, it also requires agencies to collect atypical financial data and program officials to adjust their evaluation framework. The unfamiliarity and perceived inapplicability of economic impact evaluations in health and human services exacerbated those challenges. The lesson learned was detailed accounting of program and personnel data, leadership buy-in, and an in-house champion are required to facilitate its success. |
| Does the Number of Years in Preschool Predict Social-emotional Outcomes? |
| Poster Presentation 139 to be held in Sebastian Section J on Wednesday, Nov 11, 6:30 PM to 8:00 PM |
| Presenter(s): |
| Julia Moore, Pennsylvania State University, jem470@psu.edu |
| Brittany L Rhoades, Pennsylvania State University, blr162@psu.edu |
| Celene Domitrovich, Pennsylvania State University, cxd130@psu.edu |
| Mark T Greenberg, Pennsylvania State University, mxg47@psu.edu |
| Abstract: Despite the emphasis on the benefit of preschool for children from low-income families, limited research has explored whether two years of preschool is better than one; the studies that do exist focus primarily on cognitive and academic outcomes. Yet, social-emotional competence is an important predictor of later adjustment and academic achievement. This study will compare children who received two (n = 157), one (n = 195), or no years (n = 116) of preschool on social-emotional outcomes in kindergarten. Preliminary analyses revealed significant differences on all of the measures when comparing children who received some preschool (one or two years) to those who received none. Significant differences were found for some, but not all of the outcomes when comparing one versus two years of preschool. Further analyses using hierarchical linear modeling will be conducted to account for nesting and to model changes over time. |
| Viewing Bennett's Hierarchy From a Different Lens: Implications for Extension Evaluation |
| Poster Presentation 139 to be held in Sebastian Section J on Wednesday, Nov 11, 6:30 PM to 8:00 PM |
| Presenter(s): |
| Rama Radhakrishna, Pennsylvania State University, brr100@psu.edu |
| Abstract: In the new millennium, assessing impact of programs, stakeholder demand for accountability, and continued reduction in government funding for all programs are a major concern. Examining Bennett's hierarchy from a different lens provides some insights to link the hierarchy with outcomes and costs associated in evaluating a program. As one moves up the hierarchy, collecting evidence to assess impact of programs becomes costly, time consuming, and requires more skill. Further, as one move s up the hierarchy the evidence of program impact becomes stronger. This needs to be communicated to field-based educators so that they can plan, early on, what is needed in terms of costs, time, and resources to evaluate a program. Early identification of costs associated with documenting short, intermediate, and long term outcomes will go a long way in assessing the costs and benefits associated with evaluation of a particular program. |
| Moving Towards Data-Informed Decision Making in a National Technical Assistance Support System |
| Poster Presentation 139 to be held in Sebastian Section J on Wednesday, Nov 11, 6:30 PM to 8:00 PM |
| Presenter(s): |
| Jason Katz, University of South Carolina, katzj@mailbox.sc.edu |
| Abraham Wandersman, University of South Carolina, wanderah@gwm.sc.edu |
| Ivonn Ellis-Wiggan, National Federation of Families for Children's Mental Health, ielliswiggan@ffcmh.org |
| Abstract: Although the benefits of technical assistance (TA) have been demonstrated in community-based prevention, health, and human service programs, more research is needed to better understand which aspects of TA are most effective. The present study evaluates the effectiveness of TA within the Center for Mental Health Services' 'Comprehensive Community Mental Health Services for Children and their Families' program in enhancing important outcomes including interagency collaboration, family involvement, and cultural competence. Community recipients of TA are currently providing feedback via surveys and semi-structured interviews about which aspects of TA are most helpful to them in their service provision and work with children and families. It is expected that the survey and interview data will inform and improve the work of technical assistance providers in the systems of care they work with. Although this evaluation is contextualized within child and adolescent mental health services, it also serves as a step towards more generally identifying best practices in technical assistance. |
| A Framework for Assessing Best Practices in Dropout Prevention |
| Poster Presentation 139 to be held in Sebastian Section J on Wednesday, Nov 11, 6:30 PM to 8:00 PM |
| Presenter(s): |
| Allan Porowski, ICF International, aporowski@icfi.com |
| Thomas Horwood, ICF International, thorwood@icfi.com |
| Abstract: In this presentation, study authors will describe the methods and results of a study funded by the State of Texas to identify Best Practices in Dropout Prevention. The Best Practices in Dropout Prevention study was a requirement of House Bill 2237, passed by the Texas Legislature in 2007. In addition to identifying the most effective dropout programs from the evidence, the study authors wanted to take one additional - and admittedly, ambitious - step: We wanted to not only determine what programs work, but also why they work, how they work, and in what situations they work. Results indicate that dropout prevention programs are reporting successes in various settings and with different populations. The evidence demonstrates that it is possible to achieve positive results using a core set of effective strategies, even among the highest risk populations. |
| Evaluating the Evaluator: A Theoretical Approach to Internally Evaluating an Assessment Center |
| Poster Presentation 139 to be held in Sebastian Section J on Wednesday, Nov 11, 6:30 PM to 8:00 PM |
| Presenter(s): |
| Alex Ramsey, Southern Illinois University at Carbondale, aramsey@siu.edu |
| Abstract: The purpose of this evaluation was to receive feedback from those involved with an assessment center at a Midwestern university. The program assesses the effectiveness of the university curriculum and identifies strengths and weaknesses of undergraduate students regarding core competencies necessary to perform well in academic and workplace settings. The primary focus of the project is to use Lee Cronbach's approach to guide the direction and methods of an internal, formative evaluation. Quantitative and qualitative feedback regarding the effectiveness of the assessment center was obtained from the supervisors overseeing the program, the interns who conduct the student ratings, and the undergraduate students being rated. The study will report the effectiveness of the assessment center in its current state and make a case for theoretically-grounded evaluations in the assessment of educational programs. |
| Eliciting Information From Young Children in the Evaluation Context |
| Poster Presentation 139 to be held in Sebastian Section J on Wednesday, Nov 11, 6:30 PM to 8:00 PM |
| Presenter(s): |
| Susan Menkes, Claremont Graduate University, susan.menkes@cgu.edu |
| Tiffany Berry, Claremont Graduate University, tiffany.berry@cgu.edu |
| Abstract: Youth participatory evaluation involves young people at all or various levels of evaluation (Sabo Flores, 2007). Because much of this research has targeted early to late adolescents, it is unclear how to engage young children substantively in evaluation. While including young children throughout the evaluation may not be feasible, they can provide valuable feedback related to the strengths, weaknesses, or suggested improvements. However, asking young children to verbalize perceived weaknesses requires using negatively phrased questions (e.g., 'What do you not like about the program?'). Surprisingly, there is little empirical support to suggest which type of negatively phrased question elicits the most descriptive information, especially for young children. Accordingly, the purpose of this poster is to discuss results examining question conditions that maximize young children's descriptive detail. Given the unchanging nature of the interview context, it is critical to understand how to obtain useful evaluative information using developmentally sensitive practices. |
| Empowering Youth to be Evaluators: Involving Young People in Evaluating a Museum-Based Youth Development Program |
| Poster Presentation 139 to be held in Sebastian Section J on Wednesday, Nov 11, 6:30 PM to 8:00 PM |
| Presenter(s): |
| Amy Grack Nelson, Science Museum of Minnesota, agnelson@smm.org |
| Abstract: Museums strive to improve the experience young people have in their institutions, but youth are rarely included as collaborators in evaluation. Hear how youth participatory evaluation was used in an out-of-school youth development and employment program at the Science Museum of Minnesota's Kitty Andersen Youth Science Center. Summative evaluation data was shared with youth through a series of evaluation workshops where they learned about evaluation, made their own meaning of the data, generated recommendations, and ultimately took more ownership and control of their program. As a result of the process, youth were empowered to lead their own evaluation of museum activities with visitors. The poster will share factors that led to the successful use of a participatory process and the resulting benefits to the program manager and youth. |
| Program and Evaluation Lifecycles: Implications for Educational Program Managers and Funding Agencies |
| Poster Presentation 139 to be held in Sebastian Section J on Wednesday, Nov 11, 6:30 PM to 8:00 PM |
| Presenter(s): |
| Claire Hebbard, Cornell University, cer17@cornell.edu |
| Abstract: This paper describes four major lifecycle phases in outreach programs, and their corresponding evaluation lifecycles; then presents results of lifecycle analyses from over fifty programs to date, and discusses the broad-based application of this approach. Evaluation is critical for linking educational programs to participant learning. But small programs may lack evaluation capacity and expertise, and often have limited resources for evaluation. Program managers must determine the need for evaluation, and identify if they are capable of conducting the evaluation themselves. New programs need rapid feedback, and are typically interested in process and participant satisfaction, but in later phases will be interested in program association with change. Mature programs want causal evidence, and the most mature programs will look at consistency of this evidence over various settings. These needs may conflict with the funders' need for causal evidence, no matter the level of development of the program. |
| Philanthropic Foundations' Perceptions of Evaluation in the Grantmaking Process |
| Poster Presentation 139 to be held in Sebastian Section J on Wednesday, Nov 11, 6:30 PM to 8:00 PM |
| Presenter(s): |
| Gabriel Grass, Florida State University, ggrass@foundation.fsu.edu |
| Abstract: Many national philanthropic foundations require an evaluation component to be included in every grant proposal submitted-regardless of the level of funding requested. The amount of attention and importance given to this evaluation component varies from foundation to foundation. This qualitative pilot study sought to gain a better understanding of the perspective foundation program officers hold in regard to the role evaluation plays in their grantmaking. Site observations of several grantmaking foundations' websites were conducted in conjunction with interviews of senior program officers. Preliminary findings suggest that a foundation's utilization of evaluation is a result of a desire to systematically increase grant recipients' capacity to manage a newly funded program as well as increase grant recipients' accountability in managing granted funds. |
| The Evaluative Learning Project |
| Poster Presentation 139 to be held in Sebastian Section J on Wednesday, Nov 11, 6:30 PM to 8:00 PM |
| Presenter(s): |
| Ananda Roberts, nFocus Software, amoffatt@nfocus.com |
| Abstract: In an effort to explore the notion of funders supplying technology to non-profits that track and evaluate data more efficiently for after-school programs in Maricopa County, Arizona, funders in Arizona and leaders at nFocus Software implemented the Evaluative Learning Project: After-school Provider Pilot. The goal is to use evaluation and technology as the cornerstone of organizational learning and capacity building to improve the impact of after-school programs for youth. KidTrax was identified as the tool for data collection. It offers improved data integrity and streamlined operations, that track progress, analyze trends, and forecast program performance. Several non-profit agencies serving both large and small after-school programs were chosen for the project, allowing the Pilot to demonstrate the difference in impact of evaluative technology on programs and services. Capacity building, technology, and evaluation data will improve the quality of after-school services provide in their communities. |
| Assessing Support and Perceived Competence for Using Science-based Approaches for Teen Pregnancy Programs Among Different Stakeholders |
| Poster Presentation 139 to be held in Sebastian Section J on Wednesday, Nov 11, 6:30 PM to 8:00 PM |
| Presenter(s): |
| Amy Mattison Faye, South Carolina Campaign to Prevent Teen Pregnancy, afaye@teenpregnancysc.org |
| Jennifer L Duffy, South Carolina Campaign to Prevent Teen Pregnancy, jduffy@teenpregnancysc.org |
| Mary Prince, South Carolina Campaign to Prevent Teen Pregnancy, mprince@teenpregnancysc.org |
| Abstract: Many grantors now require the use of science-based approaches (SBA) such as using data and health behavior theory to guide selection of target populations and intervention activities, selecting science-based or evidence-based programs, and conducting evaluation of implemented programs. Within organizations, the attitudes and perceptions of key staff and stakeholders may be barriers to using SBA.However, proactive training and technical assistance to appropriate staff/stakeholders can address problems early thereby reducing barriers and promoting early adoption of SBA. Through written and electronic surveys, this study assessed 1) attitudes towards using SBA and 2) perceived competencies to use SBA in three populations associated with program sustainability: program coordinators, direct service providers, and key stakeholders (board members, clergy, parents). Surveys were completed by 124 respondents in 14 organizations. In this session, the differences in attitudes and perceived competencies across the three populations will be described and the implications for training and TA discussed. |
| Evaluating the Quality of Science Teaching From Secondary School Students' Perspective |
| Poster Presentation 139 to be held in Sebastian Section J on Wednesday, Nov 11, 6:30 PM to 8:00 PM |
| Presenter(s): |
| Maritza Briceņo-Caballero, Universidad Autonoma de Yucatan, marimine99@hotmail.com |
| Georgina Piņa-Valladares, Universidad Autonoma de Yucatan, milinda_ceci@hotmail.com |
| Mariana L Santana-Cisneros, Universidad Autonoma de Yucatan, mary_ana0331@hotmail.com |
| Edith Cisneros-Cohernour, Universidad Autonoma de Yucatan, cchacon@uady.mx |
| Abstract: The purpose of this study was assessing from secondary school student perspectives, the strengths and limitations of math and science teaching in secondary schools of Southern Mexico. Data collection involved a survey of 5560 students at 17 secondary schools in Southern Mexico. Students identified the main problems and areas for improvement with science teaching. Students also made recommendations for improving their instructors' teaching |
| Using Appreciative Inquiry to Evaluate and Promote Partnerships for Tuberculosis Control and Prevention |
| Poster Presentation 139 to be held in Sebastian Section J on Wednesday, Nov 11, 6:30 PM to 8:00 PM |
| Presenter(s): |
| Vidya Venkataramanan, Centers for Disease Control and Prevention, hua3@cdc.gov |
| Kai Young, Centers for Disease Control and Prevention, deq0@cdc.gov |
| Abstract: The use of partnerships is a promising strategy for achieving program goals. This evaluation acknowledges partnerships among U.S. tuberculosis (TB) programs and seeks to understand how they use these partnerships, specifically with the American Lung Association (ALA), for their prevention and control activities. Study sites were selected based on the reporting of ALA partnerships in programs' progress reports. Control sites were matched based on similar epidemiological characteristics, but none reported ALA partnerships. Interviews with control groups help identify alternative strategies and can encourage the utilization of partnerships in this group. The goals of this project are achieved by using 30-minute interview questions developed using appreciative inquiry techniques for the following activities: managing TB in the homeless, education and training for workers, and TB control in low incidence states. Dissemination of findings highlighting best practices can encourage and strengthen the use of partnerships to improve program outcomes. |
| Opting for Evaluation Expediency in an Age of Budgetary Cost-Cutting |
| Poster Presentation 139 to be held in Sebastian Section J on Wednesday, Nov 11, 6:30 PM to 8:00 PM |
| Presenter(s): |
| Robin Kipke, Tobacco Control Evaluation Center, rakipke@ucdavis.edu |
| Abstract: California has been hit harder than most states with budget shortfalls. As a result, programs that rely on state funding have seen their own budgets shrink. This has left county tobacco control programs with roughly the same amount of policy and evaluation work to do but often with less money or personnel to complete it. They also experience a high turnover in staff. As the technical assistance center on evaluation for all California tobacco control projects, the Tobacco Control Evaluation Center has had to rethink how it helps client organizations build evaluation capacity, shifting from training on how to apply data collection methods to a more streamlined focus on how to adapt and utilize pre-designed generic tools. This session will describe the ways advocacy organizations and their technical assistance providers can utilize expedient evaluation strategies without sacrificing the value of the data to programs in an era of shrinking resources. |
| Understanding Transfer of Training From a Soft Skills Program |
| Poster Presentation 139 to be held in Sebastian Section J on Wednesday, Nov 11, 6:30 PM to 8:00 PM |
| Presenter(s): |
| Sandhya Rao Hermon, PriceWaterHouseCoopers LLP, sandhya.rao@us.pwc.com |
| Marisol Febles, PriceWaterHouseCoopers LLP, marisol.febles@us.pwc.com |
| Stephen Barone, PriceWaterHouseCoopers LLP, stephen.barone@us.pwc.com |
| Andrew Wolff, PriceWaterHouseCoopers LLP, andrew.wolff@us.pwc.com |
| Samantha Jones, PriceWaterHouseCoopers LLP, samantha.jones@us.pwc.com |
| Abstract: An important measure of the impact of training is change in job-related behaviours. This is Level 3 of the Kirkpatrick training evaluation model (1959). This study was designed to understand behaviour changes as a result of sending 40 employees of PricewaterhouseCoopers to a soft skills training program. The training included theory and practice on improving one's presentation skills. Baseline measures included learners' perceptions toward their presentation skills. Immediately after training, data on course satisfaction (Kirkpatrick's Level 1) were collected. A second follow-up -- currently in-process -- was conducted four months after training to understand transfer of learning (Kirkpatrick's Level 3). Learners were asked when they applied learning to the job, frequency of accessing materials, and perceptions of current presentation skills. Proposed analyses include pre- to post- comparisons of attitudes toward public presentation skills and perceived improvement in those skills. Findings will be used to improve training content and usability for future offerings. |
| A Process Evaluation for Assessing Self-reported Data Collection Procedures in Medically Underserved and Minority Populations Participating in the Patient Navigator Research Program |
| Poster Presentation 139 to be held in Sebastian Section J on Wednesday, Nov 11, 6:30 PM to 8:00 PM |
| Presenter(s): |
| Mariana Arevalo, Moffitt Cancer Center, mariana.arevalo@moffitt.org |
| Desiree Rivers, Moffitt Cancer Center, desiree.rivers@moffitt.org |
| Kristen Wells, Moffitt Cancer Center, kristen.wells@moffitt.org |
| Milagro Gonzalez, Moffitt Cancer Center, milagro.gonzalez@moffitt.org |
| Summer Fiddes, University of South Florida, tcb1416@aol.com |
| Richard Roetzheim, Moffitt Cancer Center, rroetzhe@health.usf.edu |
| Abstract: The Patient Navigator Research Program is a multi-site research program that aims to reduce cancer health disparities by eliminating barriers to health care in patients following an abnormality suspicious for cancer. Collection of self-reported data is conducted to assess satisfaction outcomes. In-person interviews were conducted (n=158), and a set of 17 instruments was administered to program participants in interviews lasting 60 to 90 minutes. A process evaluation was conducted to assess the procedures for collecting self-reported data, evaluate some of the challenges to the collection of these data in medically underserved and minority populations, examine the contextual factors that may contribute to the challenges, and list some of the lessons learned through the data collection process. The findings of this process evaluation will help improve data collection processes and outcomes in the program and will contribute to the current body of research in the field that addresses medically underserved populations issues. |
| Theory-driven Approach to a Family-based Agricultural Injury-Prevention Intervention: Testing the Theory of Planned Behavior |
| Poster Presentation 139 to be held in Sebastian Section J on Wednesday, Nov 11, 6:30 PM to 8:00 PM |
| Presenter(s): |
| Hamida Jinnah-Ghelani, University of Georgia, hamidajinnah@gmail.com |
| Zolinda Stoneman, University of Georgia, zo@ihdd.uga.edu |
| Abstract: In this paper, we use a theory driven approach to test the efficacy of a family-based farm safety intervention. We utilize the intervening mechanism evaluation to test the Theory of Planned Behavior (Azjen, 1985) as a framework for changing safety behaviors of individuals. Intervening variables include knowledge of family members on farm-safety. This model contains multiple determinants in a sequential order. Determinants include changes in their attitude towards farm-safety, the normative beliefs and their perceived behavioral control. These lead to change in intention, which is identified as the immediate determinant of performing specific behaviors. The outcome variable here is adoption of safety practices by individuals. The study uses a randomized control design with three groups - parent-led, staff-led and control. Results from structural equation modeling to examine the intervening process will be presented. Implications for agricultural research will be discussed. |
| Issues, challenges of a new masters program in evaluation in the current economy: a three year evaluation |
| Poster Presentation 139 to be held in Sebastian Section J on Wednesday, Nov 11, 6:30 PM to 8:00 PM |
| Presenter(s): |
| Deborah M Oh, California State University Los Angeles, doh2@calstatela.edu |
| Simeon Slobacek, California State University Los Angeles, sslovac@exchange.calstatela.edu |
| Abstract: California State University, Los Angeles (CSULA), created and implemented a master's program in evaluation. The faculty at CSULA documented the need for such a program in 2006 using student surveys, a review of existing master's programs, and interviews with potential employers. The goal of the program was to increase the capacity of educational leaders, faculty and local school districts in the Los Angeles area to conduct quality research and evaluation of various educational programs. Another goal of the program was to increase the participation of under-represented minority practitioners in the field of evaluation. The program was a response to crucial documented needs including a strong need for trained evaluators by colleges, local school districts and non-profit organizations. Issues and challenges in setting up a university program in evaluation are discussed. And to the extent the MA program accomplished its goals and objectives over a 3-year period are shared. |
| Evaluating a University's Parental Notification Program |
| Poster Presentation 139 to be held in Sebastian Section J on Wednesday, Nov 11, 6:30 PM to 8:00 PM |
| Presenter(s): |
| Susanne Kaesbauer, University of Tennessee at Knoxville, skaesbau@utk.edu |
| Abstract: The proposed poster reports on the results of an evaluation of a new alcohol policy at a large southeastern public university. The purpose of this evaluation was to assess the impact of the parental notification program on students and Student Judicial Affairs staff. The parental notification program requires the university to notify parents of students under the age of 21 who violated the alcohol and drug policy. The evaluation provided insight into the elements, design, and logic of the program and to what extent the program reduced student alcohol and drug consumption. Furthermore, the evaluation assessed the implementation of the notification process in the university's Office of Student Judicial Affairs. The evaluation offered insight into the effort and resources consumed by the notification process and the nature of the implementation problems encountered. |
| Evaluating After-school Programs: Does Program Site Make a Difference? |
| Poster Presentation 139 to be held in Sebastian Section J on Wednesday, Nov 11, 6:30 PM to 8:00 PM |
| Presenter(s): |
| Susan T Hibbard, Florida Gulf Coast University, shibbard@fgcu.edu |
| Abstract: This study compares evaluations of an afterschool program conducted in two locations: on-site at a school and off-site at a community center. The program's goals were to improve academic achievement, increase arts education, reduce risk behaviors of poor nutrition and obesity, involving parents, and improve student behavior. The program was held in two separate locations. Both program sites had the same objectives and requirements. As a result of the organizational context at each location, the evaluation was carried out differently at each site. Overall, a collaborative evaluation approach was used to include stakeholders from each site throughout the process. Cooperation of collaborative members varied among the two program sites resulting in the use of different strategies and approaches to increase program improvement. Data sources included interviews, survey questionnaires, and assessments. Strategies for increasing the use of formative evaluation results and lessons learned will be presented. |
| Learning Evaluation From Literature |
| Poster Presentation 139 to be held in Sebastian Section J on Wednesday, Nov 11, 6:30 PM to 8:00 PM |
| Presenter(s): |
| Kate Drew, Brigham Young University, kate.drew@comcast.net |
| David Williams, Brigham Young University, david_williams@byu.edu |
| Abstract: The discipline of evaluation is committed to ensuring adherence to various formal evaluation standards, which have evolved from social science and other scientific disciplines. But what might the humanities and the study of informal evaluations contribute to evaluation theory and practice? This poster summarizes a small team's efforts to analyze several literary works regarding how authors use their art to examine evaluation issues manifested in informal evaluation situations and to think about how their ideas might enhance the practice of formal evaluation. We found that through developing characters who conduct informal evaluations, most authors provide much fodder for evaluation theorists and practitioners to consider as they continue developing the discipline of formal evaluation. Using a framework for categorizing several traditional formal evaluation principles, this poster maps literary examples to evaluation principles and raises questions for evaluators to consider in light of issues explored in literature. |
| The Role of Faculty Academic and Social Support in the Achievement and Retention of Minority College Students |
| Poster Presentation 139 to be held in Sebastian Section J on Wednesday, Nov 11, 6:30 PM to 8:00 PM |
| Presenter(s): |
| Steven Lee, University of Kansas, swlee@ku.edu |
| Erik Fister, University of Kansas, efister@ku.edu |
| Justin P Allen, University of Kansas, jpallen@ku.edu |
| Abstract: According to the National Center for Education Statistics the percentage of minority students completing a bachelor's degree has increased 140% from 1990 to 2006. Despite higher graduation rates, their first year attrition rate of minority students still remains high. Models developed (Dennis, Phinney & Chuateco, 2005) attempting to explain retention and graduation rates in minority students have failed to take into account the role of the faculty in mentoring these students. In this study, we evaluated a mentoring program for minority students funded by the Fund for Improvement of Postsecondary Education (FIPSE) at a large Midwestern public university. Contrary to Dennis, Phinney & Chuateco's (2005) model, no social or academic support variables predicted cumulative grade point average. Interestingly, faculty academic and social support was predictive of satisfaction with the mentoring program. These results will be discussed in light of models and research on minority student college retention and mentoring programs. |
| Evaluation Inquiry and Higher Education Discourse: Situating Evaluation and Assessment Studies in Faculty Deliberations |
| Poster Presentation 139 to be held in Sebastian Section J on Wednesday, Nov 11, 6:30 PM to 8:00 PM |
| Presenter(s): |
| William Rickards, Alverno College, william.rickards@alverno.edu |
| Abstract: Like other practitioners, various faculty deliberations serve as a discourse of practice with structures and norms through which their work proceeds. Understanding this discourse provides a means for increasing faculty engagement bring evaluation studies into faculty conversations. A set of studies provides examples and issues of evaluation practice. |
| Helping Our Toddlers, Developing Out Children's Skills (HOT DOCS) Parent Training Program: Is it Ready for the World? |
| Poster Presentation 139 to be held in Sebastian Section J on Wednesday, Nov 11, 6:30 PM to 8:00 PM |
| Presenter(s): |
| JC Smith, University of South Florida, jcsmith6@mail.usf.edu |
| Abraham Salinas, University of South Florida, asalinas@health.usf.edu |
| Abstract: This poster session will focus on an evaluation using extant literature related to the Helping Our Toddlers Developing Our Children's Skills (HOT DOCS) program. This session will consider the quality of the literature related to the program and the its readiness for dissemination to community-based providers. We examined a peer-reviewed article, an article submitted for publication, a previous evaluation, and a thesis during the course of the present evaluation. Reviewers created separate rubrics to evaluate the program based on quality of the research and its readiness for dissemination. The reviewers independently scored each piece of literature and the materials necessary to provide the trainings. Consensus and detailed rationale were provided for all areas evaluated. Indication of the need for further quality research were identified in order to increase the likelihood that HOT DOCS will meet criteria to be designated an evidence based program by various agencies. |
| An Evaluation of the ShapeN' Up and SlimN' Down Program: Investigating the Effects of Parents and Physicians to Understand the Impact on Child Health Behavior |
| Poster Presentation 139 to be held in Sebastian Section J on Wednesday, Nov 11, 6:30 PM to 8:00 PM |
| Presenter(s): |
| Krista Collins, Claremont Graduate University, krista.collins@cgu.edu |
| Elizabeth Terris, Claremont Graduate University, elizabeth.terris@cgu.edu |
| Namrata Mahajan, Claremont Graduate University, namrata.mahajan@cgu.edu |
| Tarek Azzam, Claremont Graduate University, tarek.azzam@cgu.edu |
| Abstract: Preventative health programs typically evaluate changes in patient behavior. With child-centered services, there are several external factors that need to be considered to fully understand the program impact on child health. The ShapeN' Up and SlimN' Down program is a health education service offered to uninsured families in an effort to reduce the increasing rates of childhood obesity. This program is based on the idea that a healthy lifestyle is dependent on making smart behavior decisions, and that for children to be successful, parental involvement is imperative. Using the theory of developmental contextualism, this evaluation examines the contribution of parental involvement and physician practice on the overall program impact on child health behavior. We predict a positive correlation between these external factors and child behavior, such that the greatest positive changes in child health will be associated with positive changes in both parental behavior and physician care. |
| Using the Service Process Adherence to Needs and Strengths (SPANS) to Examine Quality of Treatment Planning in Out-of-Home Settings |
| Poster Presentation 139 to be held in Sebastian Section J on Wednesday, Nov 11, 6:30 PM to 8:00 PM |
| Presenter(s): |
| Keren S Vergon, University of South Florida, vergon@fmhi.usf.edu |
| Norin Dollard, University of South Florida, dollard@fmhi.usf.edu |
| Abstract: This study examined how well treatment plans for youth in out-of-home care in Florida reflected needs and strengths as measured by the Child and Adolescent Needs and Strengths (CANS) and how well service delivery reflected treatment plan recommendations as measured by the Service Process Adherence to Needs and Strengths (SPANS). The SPANS is a tool for reviewing case records to determine whether CANS' ratings that indicate moderate to high levels of needs or strengths are being addressed or utilized by the services and interventions that are provided to a child and family. A random sample of more than 350 records of youth in various levels of out-of-home care were reviewed. Treatment plans responded appropriately to most of the issues related to youths' mental health needs, although strengths were under-identified and under-utilized. SPANS scores suggest that several items of concern as indicated by CANS ratings were not addressed through service provision. |
| Embracing Context in Evaluation: Imperatives and Challenges |
| Poster Presentation 139 to be held in Sebastian Section J on Wednesday, Nov 11, 6:30 PM to 8:00 PM |
| Presenter(s): |
| Christine Emmons, Yale University, christine.emmons@yale.edu |
| Fay Brown, Yale University, fay.brown@yale.edu |
| Abstract: The landscape of school reform is structured by the historical, social, and particularly within the last several years, the political context in which this reform takes place. Policies at the federal, state and local level have a strong impact on the amount and type of reform and the kind of evaluation that is expected and valued. As such, perhaps for school reform more so than any other area, embracing context in evaluation is vital. In this paper, we identify critical areas in which a school reform program evaluation process integrates various contexts into the design and conduct of the evaluation and the reporting of results. We also discuss the challenges experienced in attempting such a comprehensive approach. The purpose of this paper is to identify and discuss imperatives and challenges to embracing context as an integral part of evaluation, and to stimulate discussion on the relative strengths and weaknesses of such an approach. |
| Challenges Involved in the Evaluation of Community-based Health Partnerships to Increase Awareness, Knowledge, and Promotion of Cancer Clinical Trials in the Community |
| Poster Presentation 139 to be held in Sebastian Section J on Wednesday, Nov 11, 6:30 PM to 8:00 PM |
| Presenter(s): |
| S Lisbeth Jarama, NOVA Research Company, ljarama@novaresearch.com |
| Bruce B Rytkonen, Tacoma Pierce County Health Department, brytkonen@tpchd.org |
| Brian K Gibbs, Harvard School of Public Health, bkgibbs@hsph.harvard.edu |
| Margo Michaels, Education Network to Advance Cancer Clinical Trial, margomichaels@enacct.org |
| Robert M Goodman, Indiana University, rmg@indiana.edu |
| Nakia C Brown, NOVA Research Company, ncbrown@novaresearch.com |
| Natasha Blakeney Wilson, Education Network to Advance Cancer Clinical Trials (ENACCT), natasha.blakeney@enacct.org |
| Abbey Britton, Decatur Memorial Hospital, abbeys@dmhhs.org |
| Liz Swords, Decatur Memorial Hospital, liz.swords@hotmail.com |
| Samantha Yeun, Tacoma Pierce County Health Department, syeun@tpchd.org |
| Naomi Bitow, Harvard School of Public Health, nbitow@hsph.harvard.edu |
| Abstract: Describe qualitative results of the evaluation of an innovative community-centered outreach and education program to enhance community awareness and knowledge about cancer clinical trials (CCT). Challenges in using an empowerment evaluation approach are also discussed. The program is based on a model of community engagement and education on CCTs. Three community-based health partnerships with diverse structures and backgrounds were selected to implement collaborative efforts among community leaders, health care providers, and cancer clinical trial staff with the ultimate goal of promoting CCT discussions and reduce access barriers to CCTs in the community. Qualitative data collected via interviews and focus groups was analyzed using a multi-site case study approach. Lessons learned and recommendations from evaluation findings can help refine community education about CCT, diffusion of CCT knowledge, and advocacy to facilitate local access to clinical trials. Challenges involved in using evaluation findings to improve local programs through self-evaluation and reflection are discussed. |
| Evaluating the Idea of Entrepreneurship: Testing the "Entrepreneurial" Practices of Responsive Evaluators |
| Poster Presentation 139 to be held in Sebastian Section J on Wednesday, Nov 11, 6:30 PM to 8:00 PM |
| Presenter(s): |
| Jing Che, University of Rochester, jing.che@warner.rochester.edu |
| Andrew Wall, University of Rochester, afwall@warner.rochester.edu |
| Nikhil Varerkar, University of Rochester, nimasiv@gmail.com |
| Abstract: Innovation and risk-taking are often cited to characterize entrepreneurial leaders, but few studies have examined evaluators' entrepreneurial practices. In the course of evaluating entrepreneurship initiatives at a northeastern university, evaluators of this study reflected upon their own 'entrepreneurial' experiences while taking a responsive evaluation approach, and explored possibilities of easing the tensions between meeting stakeholders' expectations and maintaining value pluralism. Informed by theories on academic capitalism and entrepreneurship traits, they problematized the notion of entrepreneurship, and triangulated different value perspectives of people affected by entrepreneurship initiatives, and focused on 'issue question' of gains and losses in the implementation rather than 'program objectives'. How to maneuver through political ramifications of entrepreneurship-initiated zeal and animosity at evaluation sites required their innovative and risk-taking practices, which also guided their design of a multi-case sequential mixed-method evaluation. The analysis was iterative. The study might have implications for entrepreneurial evaluation practices in value conflicts. |
| Context Cues and Reasoning Model Selection in Introductory Physics |
| Poster Presentation 139 to be held in Sebastian Section J on Wednesday, Nov 11, 6:30 PM to 8:00 PM |
| Presenter(s): |
| David Schuster, Western Michigan University, david.schuster@wmich.edu |
| Robert McCowen, Western Michigan University, robert.h.mccowen@wmich.edu |
| Abstract: Even with successful instruction, many students in introductory physics appear to hold on to misconceptions about fundamental concepts. We have repeatedly observed students attempting to use such misconceptions to solve problems, despite prior indications that the students do understand the correct model. We seek to further understand the criteria for which students select among mental models of phenomena for the purpose of problem-solving. In this study, students in an introductory physics course for elementary education majors were asked to solve problems regarding certain fundamental misconceptions; the treatment group received additional lead-in questions designed to cue the correct problem-solving model. |
| Educational Program Evaluation Capacity Building in India: A Descriptive Account |
| Poster Presentation 139 to be held in Sebastian Section J on Wednesday, Nov 11, 6:30 PM to 8:00 PM |
| Presenter(s): |
| J Bradley Cousins, University of Ottawa, bcousins@uottawa.ca |
| Jill Anne Chouinard, University of Ottawa, jchou042@uottawa.ca |
| Abstract: This poster presentation provides a descriptive account of a working visit to New Delhi considered as the first phase of an evaluation capacity building (ECB) initiative. We began training in evaluation methods and practice with faculty working with the National Council for Educational Research and Training (NCERT) as part of their mandate to evaluate educational quality initiatives associated with the Government of India's goals of attaining universal elementary education in the country. Described is a six-day training initiative that involved trainees working in groups to develop tentative evaluation plans for specific projects located in various states around the country. We provide an overview of the curriculum and a critique of the process based on training evaluation data and post-training reflections. Lessons learned and plans for ongoing ECB are described. |
| Project Organization and Management Using the FileMaker Pro Database Application |
| Poster Presentation 139 to be held in Sebastian Section J on Wednesday, Nov 11, 6:30 PM to 8:00 PM |
| Presenter(s): |
| Susan Saka, University of Hawaii, ssaka@hawaii.edu |
| Abstract: Technology, used properly, can enhance an operation, whether an evaluation or project/office management. The FileMaker Pro database application is used by multiple people on Macintosh and Windows systems to coordinate and manage many aspects of our evaluation office where six projects ranging in funding from fifteen thousand to a million dollars run simultaneously under two principal investigators in a university setting. It is also used for scheduling; storing data and information; generating memos, labels, and forms; tracking progress; analyzing data; and reporting for a project involving 52 schools and over 500 teachers and 13,000 students and to manage a library of hundreds of titles. An in-house IT specialist developed most of the databases, but non-techie student assistants and evaluators have also been able to create their own specialized databases. FileMaker Pro's ability to be used in conjunction with email, iCal, and FMtouch (mobile product) and cautions will also be discussed. |
| Evaluating HeathyU: A Model for Health Science Teaching in Underserved Schools |
| Poster Presentation 139 to be held in Sebastian Section J on Wednesday, Nov 11, 6:30 PM to 8:00 PM |
| Presenter(s): |
| Eunice Rodriguez, Stanford University, er23@stanford.edu |
| Randy Stafford, Stanford University, rstafford@stanford.edu |
| Nancy Morioka-Douglas, Stanford University, nmd@stanford.edu |
| Tien-Wen Wiedmann, Stanford University, twtw1@stanford.edu |
| Cammy Huang, Stanford University, cammy.huang@stanford.edu |
| Liana Gefter, Stanford University, lgefter@stanfordalumni.org |
| Alice Lee, Stanford University, alicelee2003@gmail.com |
| Abstract: To address the health threat posed by childhood obesity and lack of health education in underserved schools, we developed an educational intervention that focused on Nutrition, Exercise, Healthy Weight, and Sleep. This pilot evaluated the design and implementation of an innovative science-based health education program that could be integrated into regular curriculum and used by children, parents, and educators. In partnership with Kennedy Middle School in Redwood City, CA, we tested a service learning model and web-based model among 300 seventh grade students, with a purpose of assessing two different methods of teaching in changing knowledge of basic health science and/or healthy behavior. Phase I consisted of curriculum development of health science modules through web and classroom teaching with 'key teaching messages' that connected modules together. Phase II was implementation of the curriculum, engaging students with minimal teacher preparation in an attempt to be portable, accessible, reusable, and sustainable. |
| Building Local Evaluation Capacity in School Districts: The Development and Use of the Ohio Alternative Education Self-Assessment Instrument |
| Poster Presentation 139 to be held in Sebastian Section J on Wednesday, Nov 11, 6:30 PM to 8:00 PM |
| Presenter(s): |
| Charles Partridge, CHIP Evaluation Consulting Inc, cpartridge@chipeval.org |
| Abstract: Begun in academic year 1999-2000, Ohio's Alternative Education Challenge Grant supports approximately 500 public school districts in funding local alternative education programs for K-12 students at risk of expulsion and/or school drop-out (www.alted-mh.org/challengegrant/challengegrant.html). A main goal of the grant was to build locally sustainable programs over time. In turn, the grant's initial annual funding level of $20 million has declined annually to its current level of $13 million. A District Self-Assessment Instrument was developed to help build local capacity to implement a systematic quality improvement process, thought to be necessary for sustaining local programs. This session will illustrate the development and use of Ohio Alternative Education Self-Assessment Instrument. Validity evidence generated from empirical analyses of three consecutive years of data generally confirms the instrument's theoretical basis and structure, and suggests longitudinal growth in local evaluation capacity. |