SCHEDULE OVERVIEW ¨ WORKSHOP ¨ INDEX OF CONCURRENT SESSIONS ¨ KEYNOTES ¨ SESSIONS

Pre Institute Workshops

• PI1: (Baugh) Introduction to Evaluation - Beginner Edition

• P12: (Chapel) Introduction to Evaluation - Advanced Beginner Edition

Note that these pre-institute workshops are not included in standard Institute registration, and require an additional payment, but may be registered for on the same form.

PI1: Introduction to Evaluation - Beginner Edition

Level: Beginner

Handouts: Handouts1
 
Description: This course will provide an introduction to program evaluation and evaluation terminology for Institute participants with little/no prior background in program evaluation. The course will introduce the basic steps in a program evaluation, By the end of the session participants will understand the myriad purposes and uses for evaluation, the different types of evaluations, the definitions of basic components of a program, and the basic steps in a program evaluation, offering guidance on terminology, approaches, and options at each step. Several case studies will be used both as illustrations and as an opportunity for participants to apply the content of the course.

 

Audience: Attendees from any sector or setting who have little/no background in evaluation and need a basic working knowledge of evaluation to either conduct evaluations or use evaluation findings.

This workshop is for those brand new to evaluation who need an introduction to terminology and key steps in evaluation in order to participate more fully in the remainder of the Institute. Those with more evaluation experience, but who still desire an overview, should consider the companion workshop, “Evaluation Approaches and Challenges—An Overview Using the CDC Evaluation Framework” The cost of this workshop is not included in the Institute registration fee. It requires an additional payment, but may be registered for on the same form.

Ginneh Baugh is Director of Evaluation and Measurement at the United Way of Metropolitan Atlanta, where she is responsible for assisting the organization in identifying outcomes for major initiatives, defining grant evaluation and reporting expectations, and leading training and capacity building activities for grantees. Her nonprofit career has focused on program planning, development, and evaluation for health and human service organizations. She has been an independent consultant and previously worked as the Senior Evaluation Associate for United Way of Central Maryland, in Baltimore, MD. In Baltimore, she led the Outcomes Measurement Initiative, which included assisting programs to better monitor grants, and training agencies to improve evaluation processes. Ginneh holds a Master’s degree in Public Policy from The Johns Hopkins University, and a bachelor’s degree from The University of Georgia. In addition to working in the United Way system, Ginneh worked for U.S. Senator Max Cleland and the Annie E. Casey Foundation.

 

Offered:

• Sunday, June 22, 9:00 - 4:00 (1 hour break for lunch within)

PI2: Introduction to Evaluation - Advanced Beginner Edition
 

Level: (Advanced) Beginner

 

Handouts: Handouts 1  Handouts 2

Description: This workshop will provide an overview of program evaluation for Institute participants with some, but not extensive, prior background in program evaluation. The session will be organized around the Centers for Disease Control and Prevention’s (CDC) six-step Framework for Program Evaluation in Public Health as well as the four sets of evaluation standards from the Joint Commission on Evaluation Standards. The six steps constitute a comprehensive approach to evaluation. While its origins are in the public health sector, the Framework approach can guide any evaluation. The course will touch on all six steps, but particular emphasis will be put on the early steps, including identification and engagement of stakeholders, creation of logic models, and selecting/focusing evaluation questions. Several case studies will be used both as illustrations and as an opportunity for participants to apply the content of the course and work through some of the trade-offs and challenges inherent in program evaluation in public health and human services.

 

Audience: Attendees with some background in evaluation, but who desire an overview and an opportunity to examine challenges and approaches. Cases will be from public health but general enough to yield information applicable to any other setting or sector.

Note: This session is for those with some, but not extensive, experience in evaluation but who still desire an overview and a chance to examine approaches and challenges in program evaluation. Those with little/no evaluation experience should consider the companion workshop, “Evaluation 101: An Introduction for New Evaluation Practitioners”. The cost of this workshop is not included in the Institute registration fee. It requires an additional payment, but may be registered for on the same form.

Thomas Chapel, M.A., M.B.A., is a Senior Evaluation Scientist in the Office of Workforce and Career Development, at the Centers for Disease Control and Prevention. He serves as a central resource on strategic planning and program evaluation for CDC programs and their partners. Before joining CDC, Mr. Chapel was Vice-President of the Atlanta office of Macro International where he directed and managed projects in program evaluation, strategic planning, and evaluation design for public and non-profit organizations. He is a frequent presenter at national meetings, a frequent contributor to edited volumes and monographs on evaluation, and has facilitated or served on numerous expert panels on public health and evaluation topics. Mr. Chapel is active nationally and locally in the American Evaluation Association (AEA), currently as past-chair of the Membership Committee and convener of AEA’s Local Affiliate Collaborative. Mr. Chapel holds a BA degree from Johns Hopkins University and MA in public policy and MBA degrees from the University of Minnesota.

 

Offered:

• Sunday, June 22, 9:00 - 4:00 (1 hour break for lunch within)

SCHEDULE OVERVIEW ¨ WORKSHOP ¨ INDEX OF CONCURRENT SESSIONS ¨ KEYNOTES ¨ SESSIONS

Index of Current Sessions by Timeslot

SCHEDULE OVERVIEW ¨ WORKSHOP ¨ INDEX OF CONCURRENT SESSIONS ¨ KEYNOTES ¨ SESSIONS

Keynote Session Descriptions

• (Rog) Matching Methods to Context: Producing Actionable Evidence

• (Butterfoss) Building and Sustaining Community Coalitions and Partnerships

• (Poister) Challenges of Performance Measurement in an Age of Outcomes

Keynote 1:

(Rog) Matching Methods to Context: Producing Actionable Evidence

Description: There has been a resurgence of a debate among evaluators and policymakers over the last few years regarding the priority of the randomized study for causal inquiry. One of the needs revealed in the debate is for more productive dialogue on how best to match methods to particular program contexts to produce actionable evidence. This presentation will highlight several ways in which evaluators can make optimal method choices, including obtaining an understanding the complexity of the phenomenon and how it fits in the broader environment; using a range of design, data collection, and analytic strategies intended to improve the quality of the outcome-based evidence; and actively involving stakeholders and attending to social justice.

Debra Rog, Ph.D. is the current President-elect of the American Evaluation Association, an Associate Director at Westat, and a Vice President of the Rockville Institute. Prior to joining Westat in 2007, she served for over 15 years as the Director of the Washington Office for the Center for Evaluation and Program Improvement at Vanderbilt University. Her work has involved conducting a number of large-scale national evaluations, often multi-site studies using a range of methods. Dr. Rog has written extensively on program evaluation methods as well as homelessness, housing, and mental health. She was co-editor of the New Directions in Evaluation Volume on Informing Federal Policies on Evaluation Methodology: Building the Evidence Base for Method Choice in Government Sponsored Evaluations (Jossey Bass 2007), has been co-editor of the SAGE textbook series of Applied Social Research Methods since 1980, and is the co-editor of the upcoming second edition of the SAGE Handbook of Applied Social Research Methods.

Offered as Keynote Address:

• Monday, June 23, 8:30 - 9:15 AM

Keynote 2:

(Butterfoss) Building and Sustaining Community Coalitions and Partnerships

Handouts: Handout1

Description: Coalitions have the potential to involve multiple sectors of the community and to conduct multiple interventions that focus on systems and community change. The pooling of resources and the mobilization of talents and diverse approaches inherent in a successful coalition approach make it a logical strategy for health promotion and disease prevention. Before evaluating coalitions or their programs, however, one must understand how they work. This presentation will provide eight steps to building and maintaining durable coalitions based on research findings and practical experience with health and human service coalitions that have had an impact on the public’s health and social well-being.

Frances D. Butterfoss, Ph.D., is the EVMS Foundation Professor and Director of the Division of Community Health and Research in the Department of Pediatrics at Eastern Virginia Medical School. She is the founder and board member of the Consortium for Infant and Child Health (CINCH) and Project Immunize Virginia (PIV). She evaluated Virginia’s Healthy Start coalitions to prevent infant mortality, directed the National Coalition Training Institute, and was funded by the Robert Wood Johnson Foundation for community asthma and health insurance initiatives. Dr. Butterfoss is a researcher and consultant on the development and maintenance of effective coalitions for health promotion and disease prevention. She is on the editorial board of Family & Community Health, Deputy Editor of Health Promotion Practice, and her text, Coalitions and Partnerships in Community Health, was published by Jossey Bass in 2007. She is a past President of the Society for Public Health Education (SOPHE) and received its Mentor Award in 2002.

Offered as Keynote Address:

• Tuesday, June 24, 8:30 - 9:15 AM

Keynote 3

(Poister) Challenges of Performance Measurement in an Age of Outcomes

Description: The challenge of convincing agencies, managers, funders, and governing bodies etc. of the importance of monitoring performance on a systematic basis has largely been met at this point. Now the real challenge is to make good on the claims that have been made in promoting performance measurement by developing systems that really add value. Today's keynote from noted author Ted Poister will explore key challenges that face those who are trying to design and implement cost-effective measurement systems and identify strategies for strengthening these efforts.

Ted Poister, PhD has written extensively on results oriented management strategies, and his most recent book is Measuring Performance in Government and Nonprofit Organizations (Jossey-Bass 2003).  Dr. Poister is Professor of Public Administration at the Andrew Young School of Policy Studies at Georgia State University, where his teaching and research focus principally on strategic planning, performance management, public program analysis, and soliciting feedback on public services from customers and other stakeholders, as well as performance measurement.  In addition to helping numerous agencies in developing measurement systems, he has conducted training programs for various governmental units, and he regularly offers a two day course on performance measurement for the Evaluators’ Institute in San Francisco, Chicago, and Washington, D.C. Dr. Poister brings a multidisciplinary, multimethod perspective to measuring results.

Offered as Keynote Address:

• Wednesday, June 25, 8:30 - 9:15 AM

SCHEDULE OVERVIEW ¨ WORKSHOP ¨ INDEX OF CONCURRENT SESSIONS ¨ KEYNOTES ¨ SESSIONS

Concurrent Session Descriptions

Offering 1: Advanced Applications of Program Theory

Level: Intermediate

Handouts: Handouts1

Description: While simple logic models are an adequate way to gain clarity and initial understanding about a program, sound program theory can enhance understanding of the underlying logic of the program by providing a disciplined way to state and test assumptions about how program activities are expected to lead to program outcomes. Lecture, exercises, discussion, and peer-critique will help you to develop and use program theory as a basis for decisions about measurement and evaluation methods, to disentangle the success or failure of a program from the validity of its conceptual model, and to facilitate the participation and engagement of diverse stakeholder groups. 

You will learn:

• To employ program theory to understand the logic of a program,

• How program theory can improve evaluation accuracy and use,

• To use program theory as part of participatory evaluation practice.

Audience: Attendees with a basic background in evaluation and familiarity with logic models and program theory

Stewart I. Donaldson, Ph.D. is Professor and Chair of Psychology, Director of the Institute of Organizational and Program Evaluation Research, and Dean of the School of Behavioral and Organizational Sciences, Claremont Graduate University. He has conducted numerous evaluations, developed one of the largest university-based evaluation training programs, published numerous evaluation articles and chapters, and his recent books include Program Theory-Driven Evaluation Science: Strategies and Applications (2007), Applied Psychology: New Frontiers and Rewarding Careers (2006; with D. Berger & K. Pezdek), Evaluating Social Programs and Problems: Visions for the New Millennium (2003; with M. Scriven), Social Psychology and Policy/Program Evaluation (forthcoming; with M. Mark & B. Campbell), and What Counts as Credible Evidence in Evaluation and Evidence-Based Practice? (forthcoming; with C. Christie & M. Mark). He is co-founder of the Southern California Evaluation Association and is on the Editorial Boards of the American Journal of Evaluation and New Directions for Evaluation.

Offered:

• Tuesday, June 24, 9:25 – 12:45 (20 minute break within)

Offering 2: Evaluating Organizational Collaboration

Level: Intermediate

Handouts: Handouts1

Description: “Collaboration” is a misunderstood, under-empiricized and un-operationalized construct. Program and organizational stakeholders looking to do and be collaborative struggle to identify, practice and evaluate it with efficacy. This workshop aims to increase participants’ capacity to quantitatively and qualitatively examine the development of inter-organizational partnerships. Together, we will review, discuss, and try out specific tools for data collection, analysis and reporting, and we will identify ways to use the evaluation process to inform and improve collaborative ventures. You will practice using assessment techniques that are currently being used in the evaluation of PreK-16 educational reform initiatives and other grant-sponsored endeavors including the Safe School/Healthy Student initiative. 

Audience: Attendees with a basic understanding of organizational change theory/systems theory and familiarity with mixed methodological designs

Rebecca Gajda, Ph.D. has been a facilitator of various workshops and courses for adult learners for more than 10 years. She was a top-10 workshop presenter at Evaluation 2007, lauded for her hands-on, accessible, and immediately useful content. As Director of Research and Evaluation for a large-scale, grant-funded school improvement initiative, she is currently working collaboratively with organizational stakeholders to examine the nature, characteristics and effects of collaborative school structures on student and teacher empowerment and performance. Dr. Gajda received her Ph.D. from Colorado State University and is currently an assistant professor at the University of Massachusetts Amherst.

Offered (Two Rotations of the Same Content - Do not register for both):

• Tuesday, June 24, 9:25 – 12:45 (20 minute break within)

• Wednesday, June 25, 9:25 – 12:45 (20 minute break within)

Offering 3: Evaluating Programs for Vulnerable, Hard to Reach Populations (new!)

Level: Intermediate

Description: The purpose of this training session will be to discuss the state-of-the-art practice in evaluating programs serving hard-to-reach groups, such as individuals who are homeless, have HIV/AIDS, have substance abuse problems, are undocumented, among others.. Hard to reach groups generally are among the least empowered and most vulnerable members of our society. Traditional approaches to service delivery and evaluation generally do not work for these groups. This workshop will cover strategies for gaining knowledge on “why” a group is hard to reach, and strategies for designing and implementing evaluations that incorporate this knowledge. Specific areas to be covered include participant recruitment, retention and tracking, and gathering valid and reliable data.

Audience: Those with a basic understanding of evaluation or research methods.

Debra Rog, Ph.D. is the current President of the American Evaluation Association , an Associate Director at Westat , and a Vice President of the Rockville Institute . Prior to joining Westat in 2007, she served for over 15 years as the Director of the Washington Office for the Center for Evaluation and Program Improvement at Vanderbilt University. Her work has involved conducting a number of large-scale national evaluations, often multi-site studies using a range of methods. Dr. Rog has written extensively on program evaluation methods as well as homelessness, housing, and mental health. She was co-editor of the New Directions in Evaluation Volume on Informing Federal Policies on Evaluation Methodology: Building the Evidence Base for Method Choice in Government Sponsored Evaluations (Jossey Bass 2007), has been co-editor of the Sage textbook series of Applied Social Research Methods since 1980, and is the co-editor of the upcoming second edition of the SAGE Handbook of Applied Social Research Methods.

Offered:

• Monday, June 23, 9:25 – 12:45 (20 minute break within)

Offering 4: Strengthening Evaluation Through Cultural Relevance and Cultural Competence

Level: Intermediate

Handouts: Handouts1  Handouts2

Description: This skill-building session addresses the centrality of culture in evaluation. It is organized in two segments. The opening segment addresses the relevance of culture to all stages of the evaluation process, to the fundamental validity of our work as evaluators, and to ethical standards and guidelines of our profession. Presenters will use an FAQ format to raise questions and address common misconceptions that marginalize discussions of culture within the evaluation community (e.g., Is “culture” really just a code-word for “race”? How does culture apply to me as a white evaluator working within predominantly white populations? What is the “value added” of culture in evaluation? Why should I care?) The second segment extends cultural relevance to present strategies for building cultural competence through experience, education and self-awareness. Theoretical frameworks that situate culture in evaluation (e.g., Frierson, Hood & Hughes, 2002; Hall & Hood, 2005; Kirkhart, 2005) are presented as advance organizers for practice and application purposes. Presenters use case scenarios and participants’ own examples to integrate workshop content with participants’ field experience, interests, and concerns. They rely on various theoretical frameworks to guide the two segments in tangible and practical ways. Additional resources are provided to extend and reinforce participant learning.

Audience: Attendees working in any context with a basic background in evaluation.

Karen E. Kirkhart holds a Ph.D. in Social Work and Psychology from The University of Michigan and is currently Professor, School of Social Work, College of Human Ecology, Syracuse University. Rodney K. Hopson has undergraduate and graduate degrees in English Literature, Educational Evaluation, and Linguistics from the University of Virginia, and he is Associate Professor and Chair, Department of Educational Foundations and Leadership and faculty member in the Center for Interpretive and Qualitative Research at Duquesne University. Karen and Rodney have served in positions of leadership within the American Evaluation Association, and both are actively involved in education and scholarship on culture, diversity, and social justice in evaluation. Rodney serves as Project Director for the American Evaluation Association/Duquesne University Graduate Education Diversity Internship Program. Karen is a member of the AEA Diversity Committee task force charged with developing a public interest statement on the subject of cultural competence and evaluation.

Offered (Two Rotations of the Same Content - Do not register for both):

• Monday, June 23, 9:25 – 12:45 (20 minute break within)

• Tuesday, June 24, 9:25 – 12:45 (20 minute break within)

Offering 5: Methods for Analyzing Change Over Time

Level: Intermediate

Handouts: Handouts1 Handouts2

Description: We will focus on a variety of methods for analyzing change in outcomes over time, including the traditional fixed effects methods of pre/posttest ANCOVA and Repeated Measures ANOVA; the slopes-as-outcomes individual regression analysis approach; and multilevel modeling and random coefficients models. The purpose of the workshop is to explore the conceptual underpinnings of these different approaches to assessing change, and to compare the kinds of statistical information one is able to glean from these types of analyses when addressing questions of change. We will discuss what it means to measure change, how each method attacks that task, and how to determine which measure to use in a given situation since each method has its strengths and weaknesses with respect to its conceptual approach, parameter estimation, precision of estimates and handling missing data. Due to the nature of the topic, the majority of the workshop will involve presentations. Conceptual information, statistical output and graphs will be shared in a give-and-take format, where participants bring their own questions and concerns about analyzing change over time. Demonstration of how to set up longitudinal data for the different analytical methods will be included as well as interpreting statistical output.

Audience: Attendees with a good understanding of General Linear Models (including the ANOVA family and MRC) and some basic experience with longitudinal analysis.

Katherine McKnight, PhD., has taught statistics workshops at AEA's annual conferences for many years and began with the Institute in 2007. An outstanding facilitator, she is known for making difficult concepts accessible and interesting.

Offered:

• Wednesday, June 25, 9:25 – 12:45 (20 minute break within)

Offering 6: Accountability for Health Promotion Programs—Practical Strategies and Lessons Learned

Level: Beginner

Handouts: Handouts1

Description: Over the past decade or more, policy makers and others have called for greater accountability in the public sector. With ever-decreasing resources for public health, decision-makers want specific types of information to assess the “value” of continued investment in disease prevention and health promotion programs. For accountability purposes, how do we assess whether our public health programs are effective and result in progress toward program goals? This session will describe several strategies to assess program accountability including performance measurement, expert review and appraisal, and questions-oriented approaches. An emphasis will be given toward application and constructive use of these strategies for program improvement purposes. Practical examples demonstrating these approaches will be shared and potential, real-world challenges and lessons learned discussed.

Audience: Those working in public health contexts.

Michael Schooley is chief of the Applied Research and Evaluation Branch, Division for Heart Disease and Stroke Prevention with the Centers for Disease Control and Prevention. He has contributed to the development and implementation of numerous evaluation, applied research and surveillance projects, publications and presentations. Amy DeGroff is an evaluator working in the Division of Cancer Prevention and Control with the Centers for Disease Control and Prevention. Ms. DeGroff conducts qualitative research and evaluation studies and currently oversees a large scale multiple case study of a colorectal cancer screening program.

Offered (Two Rotations of the Same Content - Do not register for both):

• Monday, June 23, 2:30 – 4:00 PM

• Tuesday, June 24, 2:30 – 4:00 PM

Offering 7: Using Program Evaluation to Improve Nonprofit Outcomes

Description: Funders, communities, and service recipients themselves ask that nonprofits show results and demonstrate progress towards service goals. Yet, identifying, measuring, and reporting on service outcomes in ways that attend to the multiple stakeholders involved can be a challenging task. This workshop will help evaluators, program officers, and program administrators working in the nonprofit sector by providing an overview of the practice of outcomes assessment tailored to the nonprofit context. Attendees at the session will:

• Learn what/who drives program evaluation and outcomes assessment in nonprofit service providers

• Explore uses of performance data in the nonprofit sector

• Discuss challenges to measurement of social service program outcomes, and

• Develop strategies for making measurement decisions.

Handouts: Handouts1

Audience: Evaluators, program officers, and program administrators working in the nonprofit sector.

Kathryn Newcomer, Ph.D. is the Director of the doctoral program in Public Policy and Administration, and Associate Director of the Trachtenberg School of Public Policy and Public Administration, at the George Washington University where she teaches program evaluation, research design, and applied statistics. She is a sought-after trainer on the topics of performance measurement and program evaluation and has authored or co-authored numerous journal articles as well as five books, among them: The Handbook of Practical Program Evaluation (1994, 2004), and Meeting the Challenges of Performance-Oriented Government (2002). She was identified as one of the top 25 evaluation experts in the country in 2001 by the American Journal of Evaluation, is a Fellow of the National Academy of Public Administration, and currently serves on the Comptroller General’s Educators’ Advisory Panel. Dr. Newcomer earned her Ph.D. in political science from the University of Iowa.

Offered (Two Rotations of the Same Content - Do not register for both):

• Monday, June 23, 9:25 – 12:45 (20 minute break within)

• Tuesday, June 24, 9:25 – 12:45 (20 minute break within)

Offering 8: Theory-Driven Evaluation for Assessing and Improving Program Planning, Implementation, and Effectiveness

Level: Intermediate

Handouts: Handouts1

Description: Learn the theory-driven approach for assessing and improving program planning, implementation and effectiveness. You will explore the conceptual framework of program theory and its structure, which facilitates precise communication between evaluators and stakeholders regarding evaluation needs and approaches to address those needs. Mini-lectures, group exercises and case studies will be used to illustrate the use of program theory and theory-driven evaluation for program planning, initial implementation, mature implementation and outcomes. In addition, the participants will learn principles and strategies for using the theory-driven approach to deal with the following cutting edge issues: how to go beyond traditional methodology for designing a real world evaluation, how to achieve both internal and external validity in an evaluation, and how to use program theory for guiding the application of mixed methods in an evaluation.

Audience: Attendees with a basic background in logic models and/or program theory.

Huey Chen, PhD, is a senior evaluation scientist at the CDC. He was a professor at the School of Public Health at the University of Alabama at Birmingham until January 2008. Dr. Chen has contributed to the development of evaluation theory and methodology, especially in the areas of program theory, theory-driven evaluations, and evaluation taxonomy. His book Theory-Driven Evaluations has been recognized as one of the landmarks in program evaluation and his newest text, Practical Program Evaluation, offers an accessible approach to evaluation for those working in any context. In 1993 he received the AEA Paul F. Lazarsfeld Award for contributions to Evaluation Theory and in 1998 he received the CDC Senior Biomedical Research Service Award.

Offered (Two Rotations of the Same Content - Do not register for both):

• Tuesday, June 24, 9:25 – 12:45 (20 minute break within)

• Wednesday, June 25, 9:25 – 12:45 (20 minute break within)

Offering 9: Focus Group Research: Understanding, Designing and Implementing

Level: All

Description: As a qualitative research method, focus groups are an important tool to help researchers understand the motivators and determinants of a given behavior. This course provides a practical introduction to focus group research. At the completion of this course, participants will be able to 1) identify and discuss critical decisions in designing a focus group study, 2) understand how research or study questions influence decisions regarding segmentation, recruitment, and screening; and, 3) identify and discuss different types of analytical strategies and focus group reports.

Audience: Attendees working in any context who are new to focus group facilitation

Michelle Revels and Bonnie Bates are technical directors at ORC Macro specializing in focus group research and program evaluation. Ms. Revels attended Hampshire College in Amherst, MA and the Hubert H. Humphrey Institute of Public Affairs at the University of Minnesota. Ms. Bates, also a trained and experienced focus group moderator and meeting facilitator, received her bachelor’s and master’s degree in criminal justice from the University of Maryland.

Offered (Two Rotations of the Same Content - Do not register for both):

• Monday, June 23, 9:25 – 12:45 (20 minute break within)

• Wednesday, June 25, 9:25 – 12:45 (20 minute break within)

Offering 10: Rapid Ethnography in Evaluation

Level: Intermediate

Handouts: Handouts1

Description: Participants will learn how to plan, organize, and implement team-oriented time-constrained systematic qualitative methods whose results can stand alone or complement quantitative data collection and analysis in process and outcome evaluation work. Through specific evaluation project examples, included among the topics addressed in this course will be:

• Single and multiple case study designs

• Site selection criteria development and application

• Key informant / collaborator selection

• Systematic qualitative data collection strategies and associated team training/orientation

• Key concepts in the use of text-based database management software like N-6 and Atlas

The session will include case studies, discussion, and one participatory exercise designed to illustrate the difference between development of survey items and ethnographic interviewing topic guides.

Edward Liebow, PhD, is Associate Director of Battelle's Centers for Public Health Research and Evaluation. Liebow has conducted policy-related and evaluation research throughout the western US and in South Australia focusing on applying ethnographic research methods to understand the distinctive response of disadvantaged communities to potential environmental and public health hazards posed by development programs and policies. Dr. Liebow is affiliated with the University of Washington, where he teaches courses in ethnographic research methods, American Indian Studies, and comparative urban politics. He is also a visiting professor of applied anthropology and comparative economics at the Università Carlo Cattaneo in Castellanza, Varese (Italy). He is on the Executive Board and Treasurer of the American Anthropological Association.

Offered (Two Rotations of the Same Content - Do not register for both):

• Tuesday, June 24, 9:25 – 12:45 (20 minute break within)

• Wednesday, June 25, 9:25 – 12:45 (20 minute break within)

Offering 11: Technology for Responsive Evidence-Based Evaluation: Challenges and Solutions for the Data Management Needs of Multi-partner Initiatives (new!)

Level: Intermediate

Description: With recent advancements of data systems and their easy accessibility for information sharing over the Internet, the technology is now available to seamlessly integrate the monitoring of service delivery and related research interests. These imaginative technological solutions can be particularly useful for large multi-site, multi-partner community initiatives that require information that addresses questions such as: Which of our services are producing the most desired outcomes? Ideally, data systems can serve as a comprehensive, integrated, results-focused utility that is equally meaningful for evaluators, funders, and service delivery staff. This session will show participants a process with which to identify information needs and a data system strategy/utility that was invented to support the evaluation efforts of complex, multi-agency initiatives. The workshop will use both didactic and interactive approaches to first dissect three inter-related challenges - (1) deciding on information goals, (2) selection data systems, and (3) implementing the information goal in the data system - and then bring them all together to show strategies for successful implementations.

The workshop will be lead by a team from Mosaic Network, Inc., an evaluation solutions company focused on evaluation of community services and their impacts. Michael Bates, PhD, is the Director of Research and Evaluation for Mosaic and is the ead evaluator for First 5 Santa Barbara County and a contributing evaluator to numerous other human service and education projects. Michael Furlong, PhD, is the Chair of the Department of Counseling/Clinical/School Psychology Program at the University of California Santa Barbara, and one of Mosaic's co-founders and the major driving force behind Mosaic's research vision. Prashant Rajvaida, PhD, Mosaic's founder and President, is an internationally recognized computer science expert in emerging technologies.

Audience: The session is appropriate for anyone called upon to work with management and data collection systems as part of their evaluation work. Attendees should have a basic understanding of evaluation and its inherent data needs.

Offered (Two Rotations of the Same Content - Do not register for both):

• Monday, June 23, 9:25 – 12:45 (20 minute break within)

• Wednesday, June 25, 9:25 – 12:45 (20 minute break within)

Offering 12: RealWorld Evaluation I: Getting Started

Level: Intermediate

Handouts: Handouts1

Description: What do you do when asked to perform an evaluation on a program that is well underway? When your questions about baseline data and control groups are met with blank stares? When time and resources are few, yet clients expect “rigorous impact evaluation”? When there are political expectations and pressures to deal with? This workshop presents a seven-step approach that seeks to ensure the best quality evaluation under real-life constraints. Through presentations and discussion, with real-world examples drawn from international development evaluation, participants in this workshop will be introduced to the RealWorld Evaluation approach. The workshop focuses on developing country evaluation, but the techniques are applicable to evaluators working in any context with budget, time, data and political constraints. The fundamental framework for the session is an exploration of the seven steps of the RealWorld Evaluation approach, including ways to negotiate RealWorld solutions with clients. Finally, we’ll identify seven evaluation designs appropriate for RealWorld evaluations and discuss their appropriate context-dependent applications.

Audience: Evaluators working in any context with previous experience in many of the common evaluation approaches and methods.

Jim Rugh brings 44 years of experience in international development, including 28 years specializing in program evaluation. For 12 years he was the Coordinator of program Design, Monitoring and Evaluation for CARE, a large international NGO with programs in 70 countries. Under Jim’s leadership CARE adopted an evaluation policy, strategies and standards to promote learning for improved program quality and accountability for effectiveness. An active member of AEA for 22 years, he currently serves as the AEA representative to the IOCE (International Organization for Cooperation in Evaluation). Jim has facilitated evaluation workshops for many years at AEA’s annual conferences as well as many other countries including South Africa, Niger, Sri Lanka, Nepal, Ukraine, and elsewhere. He was co-author with Michael Bamberger and Linda Mabry of the book RealWorld Evaluation: Working Under Budget, Time, Data, and Political Constraints published by SAGE in 2006.

Offered:

• Monday, June 23, 9:25 – 12:45 (20 minute break within)

Offering 13: RealWorld Evaluation II: Advanced Applications

Level: Intermediate to Advanced

Handouts: Handouts1

Description: We’ll build on the lessons learned in RealWorld Evaluation I or your own expertise as a seasoned evaluator to focus on the application of a variety of methods to address constraints while being aware of threats to validity and reliability using the RealWorld Evaluation approach. Includes negotiating ToRs with clients.

Audience: Attendees who took RealWorld Evaluation I yesterday or at last year’s Institute, or who have read the RealWorld Evaluation book, or who have significant experience as practicing evaluators in varied contexts.

Jim Rugh brings 44 years of experience in international development, including 28 years specializing in program evaluation. For 12 years he was the Coordinator of program Design, Monitoring and Evaluation for CARE, a large international NGO with programs in 70 countries. Under Jim’s leadership CARE adopted an evaluation policy, strategies and standards to promote learning for improved program quality and accountability for effectiveness. An active member of AEA for 22 years, he currently serves as the AEA representative to the IOCE (International Organization for Cooperation in Evaluation). Jim has facilitated evaluation workshops for many years at AEA’s annual conferences as well as many other countries including South Africa, Niger, Sri Lanka, Nepal, Ukraine, and elsewhere. He was co-author with Michael Bamberger and Linda Mabry of the book RealWorld Evaluation: Working Under Budget, Time, Data, and Political Constraints published by SAGE in 2006.

Offered:

• Tuesday, June 24, 9:25 – 12:45 (20 minute break within)

Offering 14: Adapting Survey Research to the Technological and Cultural Shift in Telecommunications

Level: Intermediate

Description: This session will review the current state of telephone-based data collection, including the challenges of lower response rates, respondent call avoidance tools and reduced use of household landline telephones and the commensurate increased use of cell phones. The presenters will also demonstrate tested methods of increasing response rates through more effective use of caller ID and better targeting of household calling attempts. Finally, participants will determine how best to include cell phones as an integral component of their own future telephone-based survey research.

Audience: All attendees, working in any context, with an interest in telephone-based data collection

Naomi Freedner, MPH, is a Senior Research Manager at Macro International where she oversees all aspects of Macro’s Behavioral Risk Factor and Adult Tobacco Survey practices. Her tasks for these projects include survey development and evaluation, study and sample design, supervising interviewers and the data collection process, data analysis, report writing, and oral presentation of the survey findings. She and her team are continually seeking innovative ways to respond to declining response rates on RDD surveys. Randy Zuwallack, MS, coordinates and oversees statistical and sampling operations, specializing in the development of goal-oriented, cost-effective study designs for large scale research projects. Mr. ZuWallack frequently designs complex samples including dual-frame designs, particularly to reach rare populations, and has experience with stratification and sampling for small geographic areas. His current interests include the advantages and challenges of coupling cell phone surveys with traditional random digit dialing (RDD) surveys with landline equipped households.

Offered (Two Rotations of the Same Content - Do not register for both):

• Monday, June 23, 2:30 – 4:00 PM

• Tuesday, June 24, 2:30 – 4:00 PM

Offering 15: Employing Mixed-Methods in Evaluation

Level: Intermediate

Handouts: Handouts1

Description: This workshop will engage participants in a mixed methods way of thinking. Conceptual questions to be addressed include, “just what is being mixed in mixed methods evaluation designs?” and “what are varied legitimate purposes for mixing methods in applied research and evaluation?” Practical issues to be discussed include alternative dimensions of, and approaches to, mixed methods design, data analysis, and representation. In particular, we will examine ways of working with and respecting multiple inquiry traditions, forms of data, and norms regarding what count as credible results or warranted assertions in evaluation. The workshop will include activities related to mixed methods evaluation planning and data analysis.

Audience: Attendees familiar with multiple evaluation methods, working in any context.

Jennifer C. Greene received her doctorate in educational psychology from Stanford University in 1976 and has held academic appointments at the University of Rhode Island, Cornell University, and presently the University of Illinois, Urbana-Champaign. Her evaluation scholarship has broadly focused on probing the intersections of social science method with policy discourse and program decision-making, with the intent of making evaluation useful and socially responsible. Dr. Greene has concentrated specifically on advancing qualitative, mixed methods, and democratic approaches to evaluation. She has published widely in journals and books on program evaluation, has held leadership positions in AERA and AEA, and was recently co-editor-in-chief of New Directions for Evaluation.

Offered:

• Tuesday, June 24, 9:25 – 12:45 (20 minute break within)

Offering 16: Popping the Question: Developing Quality Survey Items (new!)

Level: Beginner

Handouts: Handouts1

Description: Developing reliable and valid surveys that gather useable, informative, data requires the selection of just the right questions. We will explore (a) when to use open- versus close-ended questions, including issues of feasibility of analysis; (b) the range of question types, such as yes/no, multiple choice, scales, ranking, short answer, and factors that influence selection; (c) question ordering and its impact on response; (d) careful wording to avoid common question development pitfalls; and (e) transferring pen and paper questions to online survey questions. Attendees will leave with a range of example survey questions, a guide to assist with the selection of question types, and a set of resources for further investigation of surveys and question wording.

Audience: Attendees working in any context.

Susan Kistler is the Executive Director of the American Evaluation Association. She has taught evaluation and methodology courses, including survey and instrument design, at the University of Minnesota and the Minnesota Evaluation Studies Institute, worked as a contracted workshop facilitator and evaluation capacity building coach for the Corporation for National Service through Aguirre International, and offered training on association business practices for the American Society of Association Executives.

Offered:

• Tuesday, June 24, 2:30 – 4:00 PM

Offering 17: Lenses, Filters and Frames: Calibrating and Cultivating Self As Responsive Instrument

Level: Beginner to Intermediate

Description: Who are you as you walk in the world? To what extent is who you think you be in alignment with who you actually be and are striving to become as a person and as an evaluator? Does your envisioned image of self radiate and would others agree? Which others? How do you know what you think you know about this—evidential cues, clues and signposts? Engaging and using the self as responsive instrument summons one’s capacities to move beyond unilateral self-awareness and personal declarations towards multilateral self-awareness and boundary-spanning communications as you work with colleagues, clients, and stakeholders. This workshop will give you the opportunity to mindfully develop your Self-as-Instrument Portfolio: i.e., discovering, calibrating and cultivating your lenses, filters and frames. More specifically, this would involve discerning and exploring:

• Lenses: The sensing portals through which one interfaces with the physical and social world--pathways for perceiving "data"

• Filters: one's sifting and winnowing processes and protocols given operational definitions of what is substance/"signal" versus noise and extraneous variation

• Frames: one's meaning-shaping/meaning-making infrastructure (structures and processes)

Audience: Attendees working in any context with a basic background in evaluation practice.

Hazel Symonette, Ph.D. is the Founder and Director of the Excellence Through Diversity Institute, a campus workforce learning community for faculty, staff, and administrators at the University of Wisconsin. She is an educator, researcher and intervention development and assessment specialist, as well as a Senior Policy and Program Development Specialist at the University. Dr. Symonette has offered training on the self as responsive instrument throughout the United States and in venues as far-flung as Niger, South Africa, New Zealand, and Australia. She has served as the Co-Chair of AEA’s Building Diversity Initiative and as an elected member of AEA’s Board of Directors. For ten years, she was a member of the External Evaluation Consultant Review Panel of the NSF-funded All Nations Alliance for Minority Participation in Science, Math, Engineering and Technology; for 3 years with Northwest Regional Education Lab. She serves as faculty for the Health Research and Education Trust's Cultural Competency Fellowship Program.

Offered:

• Wednesday, June 25, 9:25 – 12:45 (20 minute break within)

Offering 18: Logic Models as a Platform for Program Evaluation Planning, Implementation, and Use of Findings

Note: This workshop is full. Session 24 includes an exploration of logic models and may be an appropriate alternative.

Level: Beginner

Description: Practitioners use logic models to describe important components of a program; make visible a theory of change; and link activities to intended outcomes. For the purposes of evaluation practice, a well-constructed logic model provides a program-specific foundation for identifying evaluation questions; prioritizing data needs; and translating findings into recommendations for ongoing program improvement. Aimed directly at improving the utility of logic models and quality of evaluation practice in your setting, the workshop addresses 2 questions:

(1) What are the hallmarks of a well-constructed, scientifically-sound and useful logic model?

(2) How do we maximize the use of logic models for program evaluation planning, implementation and use of findings?

Workshop Objectives:

• Demystify and define the logic model as a starting point for everyday evaluation practice

• Identify the hallmarks of a well-constructed, scientifically-sound logic model

• Clarify the relevance of process and outcome evaluation to preparing and using program logic models

• Demonstrate the use of logic models to identify and prioritize evaluation questions and data needs

• Examine the use of logic models to identify opportunities/options for demonstrating accountability for scarce resources

• Demonstrate use of a logic model to guide preparation of findings/recommendations aimed at ongoing program improvement

•  Pinpoint additional resources for continued study/application

Audience: Attendees working in any context who are new to logic modeling.

Michelle Crozier Kegler, DrPH, MPH, is an Associate Professor in the Department of Behavioral Sciences and Health Education at the Rollins School of Public Health of Emory University. She is also Deputy Director of the Emory Prevention Research Center and Co-Director of its evaluation core. She has directed numerous evaluation projects primarily within the context of community partnerships and community-based chronic disease prevention. Dr. Kegler teaches evaluation at the Rollins School of Public Health and regularly conducts workshops on evaluation to a variety of audiences. Sally Honeycutt, MPH, CHES, joined the Emory Prevention Research Center in February 2007 as an Evaluation Specialist. Before coming to Emory, Sally was a member of the Surveillance and Evaluation Team for the Steps to a HealthierUS Program at CDC. She has served as a Maternal and Child Health Educator with the Peace Corps and has experience coordinating health promotion programs both domestically and internationally.

Offered (Two Rotations of the Same Content - Do not register for both):

• Monday, June 23, 9:25 – 12:45 (20 minute break within)

• Wednesday, June 25, 9:25 – 12:45 (20 minute break within)

Offering 19: Facing Evaluation Challenges in the Real World: A Case-Based Approach

Level: Beginner

Handouts: Handouts1

Description: This session will use case study analysis to highlight strategic, ethical, and methodological challenges that evaluation practitioners encounter in the real world and explore effective strategies for meeting those challenges. As a result of class discussion, case analysis and small group activities, participants will be able to:

• Discuss ethical, strategic and methodological challenges associated with evaluation practice

• Propose strategies for meeting those challenges

• Discuss proactive strategies for ensuring an effective and useful evaluation study.

Audience: Novice evaluators working in any context.

Iris Smith, PhD, MPH, holds a doctorate in Community Psychology from Georgia State University and a Master’s Degree in Public Health from Emory University. She is currently an Associate Professor and Director of the Career Master of Public Health Program (CMPH) at Emory University’s Rollins School of Public Health, where she also teaches a graduate level online course in Evaluation Research. Her current evaluation projects include the evaluation of the Atlanta Clinical Translational Science Institute, an interdisciplinary collaboration between Emory University, Georgia Institute of Technology and Morehouse School of Medicine. She is also Co- Director of the Evaluation Core for the Emory Prevention Research Center. Previously, Iris was the Director of National Evaluation Services for the American Cancer Society, and has also served as a Deputy Commissioner for the Georgia Department of Juvenile Justice.

Offered (Two Rotations of the Same Content - Do not register for both):

• Monday, June 23, 2:30 – 4:00 PM

• Tuesday, June 24, 2:30 – 4:00 PM

Offering 20: Introduction to Meta-analysis: Why, What, When, and an Introduction to How

Level: Intermediate

Handouts: Handouts1  Handouts2

Description: Meta-analysis is a detailed and organized approach to systematic review that makes explicit the domain of research covered, the nature and quality of the information extracted from that research, and the analytic techniques and results upon which interpretation is based. Meta-analysis has proven particularly useful in documenting effectiveness of social interventions where effects tend to be small and difficult to detect in single studies. Meta-analysis has also been used to compare the effectiveness of various treatment options, and to examine the impact of service delivery options on treatment effectiveness. After participating in this session participants will understand the logic and reasoning supporting meta-analysis, will understand and be able to estimate the quality of a meta-analysis, and will appreciate some of the judgments and challenges faced when conducting a meta-analysis.

Audience: Attendees would benefit from a basic understanding of statistics and evaluation

Jim Derzon is a Senior Evaluation Specialist in Battelle’s Centers for Public Health Research and Evaluation. Prior to joining Battelle, he was a Staff Associate at Econometrica, Inc. and past director of the Center for Knowledge Synthesis and Computer Simulation at the Pacific Institute for Research and Evaluation. His research has concentrated on developing and using meta-analytic techniques, methods, and thinking to improve prevention science, particularly in the policy relevant areas of predicting and preventing antisocial behavior, delinquency, violence, and substance use. In the course of his studies, he has compiled a 14,000 effect size database on the predictors of alcohol, tobacco, and marijuana use from longitudinal studies and, with his mentor and colleague Mark Lipsey, he built a similar database on predicting violence and antisocial behavior. He has also conducted meta-analytic studies on school-based interventions for antisocial behavior and violence and on mass-media interventions for substance use.

Offered (Two Rotations of the Same Content - Do not register for both):

• Monday, June 23, 2:30 – 4:00 PM

• Tuesday, June 24, 2:30 – 4:00 PM

Offering 21: Leading Concepts in Community Health Evaluation

Level: Intermediate

In this session participants will learn different evaluation methodologies for community assessments and be able to apply multiple approaches to the evaluation of community-based health promotion programs. Among other topics, the course will address:

• Dr. Goodman’s FORECAST method of evaluating complex community programs, including how to develop models for complex programs and then use them to develop markers, measures, and standards (meanings) for measuring program adequacy.

• Social ecological perspectives including assessing and determining adequacy of interventions from a social ecology perspective

• Community capacity including expanding our perspectives of community capacity within the construct of health promotion, identifying linkages that interconnect to strengthen community capacity, and developing an evaluation approach that assesses the development of community capacity

• Uncovering “deeper structural meanings” in community responses to evaluation

Audience: Attendees working with community health programs with experience conducting evaluations.

Robert M. Goodman, Ph.D., M.P.H., M.A., is a Professor and Dean of the School of Health, Physical Education and Recreation at Indiana University. Dr. Goodman has written extensively on issues concerning community health development, community capacity, community coalitions, evaluation methods, organizational development, and the institutionalization of health programs. He has been the principal investigator and evaluator on projects for CDC, The National Cancer Institute, The Centers for Substance Abuse Prevention, The Children’s Defense Fund, and several state health departments. In 2004, Dr. Goodman received the Distinguished Fellow Award from the Society for Public Health Education, the highest honor it bestows. Currently, Dr. Goodman is consulting on community-based public health practices and empowerment evaluation with the Diabetes Translation and Injury Prevention Branches at CDC. Also, he is leading an evaluation of community-based approaches to increasing interest in cancer clinical trials.

Offered:

• Monday, June 23, 9:25 – 12:45 (20 minute break within)

Offering 22: Qualitative Evaluation Approaches

Level: Beginner

Practical qualitative evaluation methods will be shared for community health programs. The workshop will be an overview of qualitative methods that include the use of logic models, social ecology principles, participatory-based strategies, and basic data analysis methods. The session is designed to foster creativity when planning, implementing and evaluating a community-based program. Simulation activities will be used to dramatize many of the concepts.

Audience: Attendees working in any context seeking an introduction to working with qualitative data for evaluation.

Robert M. Goodman, Ph.D., M.P.H., M.A., is a Professor and Dean of the School of Health, Physical Education and Recreation at Indiana University. Dr. Goodman has written extensively on issues concerning community health development, community capacity, community coalitions, evaluation methods, organizational development, and the institutionalization of health programs. He has been the principal investigator and evaluator on projects for CDC, The National Cancer Institute, The Centers for Substance Abuse Prevention, The Children’s Defense Fund, and several state health departments. In 2004, Dr. Goodman received the Distinguished Fellow Award from the Society for Public Health Education, the highest honor it bestows. Currently, Dr. Goodman is consulting on community-based public health practices and empowerment evaluation with the Diabetes Translation and Injury Prevention Branches at CDC. Also, he is leading an evaluation of community-based approaches to increasing interest in cancer clinical trials.

Offered:

• Tuesday, June 24, 2:30 – 4:00 PM

Offering 23: Evaluating Community Coalitions and Partnerships: Methods, Approaches, and Challenges (new!)

Level: (Advanced) Beginner

Handouts: Handouts1 Handouts2

A coalition must evaluate its infrastructure, function and processes; programs aimed to achieve the partnership’s goals; and changes in health/social status or the community. This session will complement the coalition plenary session and describe how to 1) develop a comprehensive evaluation strategy based on coalition theory; 2) select appropriate short, intermediate and long-term indicators to measure outcomes; 3) choose appropriate methods and tools; and 4) use evaluation results to provide accountability to stakeholders, and improve coalition function and program implementation 

Audience: Attendees working in community contexts with a general understanding of the distinction between quantitative and qualitative data collection methods and knowledge of evaluation terminology.

Frances D. Butterfoss, Ph.D., is the EVMS Foundation Professor and Director of the Division of Community Health and Research in the Department of Pediatrics at Eastern Virginia Medical School. She is the founder and board member of the Consortium for Infant and Child Health (CINCH) and Project Immunize Virginia (PIV). She evaluated Virginia’s Healthy Start coalitions to prevent infant mortality, directed the National Coalition Training Institute, and was funded by the Robert Wood Johnson Foundation for community asthma and health insurance initiatives. Dr. Butterfoss is a researcher and consultant on the development and maintenance of effective coalitions for health promotion and disease prevention. She is on the editorial board of Family & Community Health, Deputy Editor of Health Promotion Practice, and her text, Coalitions and Partnerships in Community Health, was published by Jossey Bass in 2007. She is a past President of the Society for Public Health Education (SOPHE) and received its Mentor Award in 2002.

Offered (Two Rotations of the Same Content - Do not register for both):

• Monday, June 23, 2:30 – 4:00 PM

• Tuesday, June 24, 2:30 – 4:00 PM

Offering 24: Every Picture Tells a Story: Flow Charts, Logic Models, LogFrames, Etc. What they are and when to use them (new!)

Level: Advanced Beginner

Handouts: Handouts1  Handouts2
 
Description: A host of visual aids are in use in planning and evaluation. This session will introduce you to some of the most popular ones—with an emphasis on flow charts, logic models, project network diagrams, and logframes. We’ll review the content and format of each one and the compare and contrast their uses so that you can better match specific tools to specific program needs. We’ll review simple ways to construct each type of tool and work through some simple cases both as illustrations and as a way for you to practice the principles presented in the session.

Audience: Assumes prior familiarity with evaluation terminology and some experience in constructing logic models.

Thomas Chapel, M.A., M.B.A., is a Senior Evaluation Scientist in the Office of Workforce and Career Development, at the Centers for Disease Control and Prevention. He serves as a central resource on strategic planning and program evaluation for CDC programs and their partners. Before joining CDC, Mr. Chapel was Vice-President of the Atlanta office of Macro International where he directed and managed projects in program evaluation, strategic planning, and evaluation design for public and non-profit organizations. He is a frequent presenter at national meetings, a frequent contributor to edited volumes and monographs on evaluation, and has facilitated or served on numerous expert panels on public health and evaluation topics. Mr. Chapel is active nationally and locally in the American Evaluation Association (AEA), currently as past-chair of the Membership Committee and convener of AEA’s Local Affiliate Collaborative. Mr. Chapel holds a BA degree from Johns Hopkins University and MA in public policy and MBA degrees from the University of Minnesota.

Offered (Two Rotations of the Same Content - Do not register for both):

• Monday, June 23, 9:25 – 12:45 (20 minute break within)

• Wednesday, June 25, 9:25 – 12:45 (20 minute break within)

Offering 25: Qualitative Interviewing: Asking the Right Questions in the Right Way (new!)

Level: Intermediate

Handouts: Handouts1
 
Description: Preparing a proper interview guide is only a first step to proper Q & A in a qualitative data collection episode. This session outlines key sections to include in an interview guide and offers suggestions for how to conduct a qualitative interview and/or focus group. The face-to-face interaction in this case is critical. The interviewer must balance attention to the questions designed for the interaction and the emergent topics in the interview. Core skills that focus attention on the audience for the study, the topics of the project, important lines of questioning and goals for ensuring quality interaction in this relationship improve the quality of data collection.

Audience: Researchers in any discipline with a basic knowledge of qualitative analysis who are interested in using conversational techniques in the form of interviews or focus groups

Raymond C. Maietta, PhD is president of ResearchTalk Inc., a qualitative research consulting company in Bohemia, New York. A sociologist from the State University of New York at Stony Brook, Ray's interests in the art of qualitative research methods motivated him to start ResearchTalk in 1996. ResearchTalk Inc. provides advice on all phases of qualitative analysis to university, government, not-for-profit and corporate researchers. Work with ResearchTalk clients using qualitative software informs recent publications: Systematic Procedures of Inquiry and Computer Data Analysis Software for Qualitative Research (with John Creswell in Handbook of Research Design and Social Measurement, Sage 2002) and State of the Art: Integrating Software with Qualitative Analysis (in Leslie Curry, Renee Shield and Terrie Wetle, (Eds.) Applying Qualitative and Mixed Methods in Aging and Public Health Research, American Public Health Association and the Gerontological Society of America 2006). More than 12 years of consultation with qualitative researchers informs the methods book Dr. Maietta is writing. Sort and Sift, Think and Shift will be completed in 2009.

Offered (Two Rotations of the Same Content - Do not register for both):

• Monday, June 23, 9:25 – 12:45 (20 minute break within)

• Tuesday, June 24, 9:25 – 12:45 (20 minute break within)

Offering 26: Analyzing Qualitative Data: Using Qualitative Data Analysis Software to Balance the Expected and Unexpected (new!)

Level: Intermediate

Handouts: Handouts1
 
Description: Ray Maietta's ‘Sort and Sift, Think and Shift’ qualitative method informs the content of this session. Qualitative evaluations are often defined by pre-determined goals and questions to pursue in analysis. However, issues emerge in initial document reviews that both confirm and challenge these goals. This session addresses ways to use qualitative data analysis software, specifically ATLAS.ti and MAXQDA, to facilitate serendipitous discovery and to balance new ideas with pre-existing questions for the study. We will discuss ways to ensure that software is always a tool that supports your exploration rather than it being a driver that defines where you are to go.

Audience: Researchers in any discipline who have collected qualitative data in the form of interviews, focus groups or fieldnotes

Raymond C. Maietta, PhD is president of ResearchTalk Inc., a qualitative research consulting company in Bohemia, New York. A sociologist from the State University of New York at Stony Brook, Ray's interests in the art of qualitative research methods motivated him to start ResearchTalk in 1996. ResearchTalk Inc. provides advice on all phases of qualitative analysis to university, government, not-for-profit and corporate researchers. Work with ResearchTalk clients using qualitative software informs recent publications: Systematic Procedures of Inquiry and Computer Data Analysis Software for Qualitative Research (with John Creswell in Handbook of Research Design and Social Measurement, Sage 2002) and State of the Art: Integrating Software with Qualitative Analysis (in Leslie Curry, Renee Shield and Terrie Wetle, (Eds.) Applying Qualitative and Mixed Methods in Aging and Public Health Research, American Public Health Association and the Gerontological Society of America 2006). More than 12 years of consultation with qualitative researchers informs the methods book Dr. Maietta is writing. Sort and Sift, Think and Shift will be completed in 2009.

Offered (Two Rotations of the Same Content - Do not register for both):

• Monday, June 23, 2:30 – 4:00 PM

• Tuesday, June 24, 2:30 – 4:00 PM

Offering 27: Using the Guiding Principles to Improve Your Evaluation Practice

Level: Beginner to Intermediate

Handouts: Handouts1

Description: The Guiding Principles for Evaluators focus on five areas of evaluation practice: systematic inquiry, competence, integrity and honesty, respect for people, and responsibilities for general and public welfare. The Principles guide the professional practice of evaluators, and inform evaluation clients and the general public about the principles they can expect to be upheld by professional practitioners. This session will share ways to use the Principles to improve the ways in which you plan for and conduct evaluations and work with stakeholders and clients. After a brief presentation that introduces the Principles, participants will work together in small groups to discuss the Principles as they relate to a topical case study. Through case explorations, lecture and small and large group discussions, you will gain a deeper understanding of the practical applications of the Principles. The workshop will also introduce resources—print, web-based and collegial networks—that evaluators can consult to handle professional dilemmas that arise in their practice. You will receive copies of the workshop presentation, the case study, the Principles in full and abbreviated brochure format, and a list of resources for more information and consultation.

Audience: Evaluators and commissioners of evaluation working in any context

Leslie Goodyear, PhD is a Research Scientist at Education Development Center, where she conducts research and evaluation for a large and small scale initiatives. As a program evaluator and researcher, Dr. Goodyear has worked with programs focused on HIV/AIDS Prevention; Out-of-School Time; Youth Engagement and Youth Media; Educational Research; and Science Education. Leslie is the recent past Chair of the AEA Ethics Committee and a new AEA Board member. She earned her M.S. and Ph.D. in Human Service Studies, with focus on Program Evaluation, from Cornell University.

Offered (Two Rotations of the Same Content - Do not register for both):

• Monday, June 23, 2:30 – 4:00 PM

• Tuesday, June 24, 2:30 – 4:00 PM

Offering 28: Portfolio Planning and Evaluation: Integrating Practices from the New Product Development and Evaluation Fields

Level: Intermediate

Handouts: Handouts1

Description: Many organizations offer a “portfolio” of discrete programs, products, and services. The evaluation field has, in recent years, begun to think about how to apply best evaluation practices to the assessment of a portfolio. Such an approach can yield the kind of information organizations need to help them achieve their goals and meet their missions. While this discussion is new to evaluators, fortunately, there are practices from other fields that, when combined with evaluation practices, offer great promise for helping organizations build effective planning and evaluation strategies for their portfolios. This session will show how insights from new product development and from evaluation can be integrated for portfolio planning and evaluation purposes. Based on evidence from both fields illustrating their relevance in the portfolio strategy process, this session will help participants learn more about what methods to use, what types of data the methods would produce, and how to use the data within an organizational context. It will be based on a case study from one organization, and lessons learned, after a year of implementing a new portfolio strategy. This session is intended as an opportunity for all participants – and the presenter – to learn from one another. 

Audience: Those with at least moderate experience in evaluation but new and/or early into their experience related to portfolio planning and evaluation strategy

Jennifer W. Martineau, Ph.D. is the director of the Design & Evaluation Center at the Center for Creative Leadership. Jennifer has extensive experience designing, implementing, and disseminating evaluation findings from leadership development programs for a wide variety of organizations and contexts. She is responsible for the instructional design and evaluation of much of the Center’s portfolio of leadership development programs and services so that CCL is offering the most relevant, high impact leadership development portfolio to address our clients’ and the market’s needs. In this capacity, she has led and participated in efforts such as creating processes and systems to support more effective and efficient development and launch of CCL programs, reviewing the Center’s entire (known and unknown) portfolio to identify strengths and gap areas, and establishing a new integrated portfolio strategy that will enable CCL to make increasingly better decisions about what portfolio development investments to make.

Offered (Two Rotations of the Same Content - Do not register for both):

• Monday, June 23, 2:30 – 4:00 PM

• Tuesday, June 24, 2:30 – 4:00 PM

Offering 29: Conducting and Using Success Stories for Capacity Building

Level: Intermediate

Handouts: Handouts1

In order to build program capacity a programs “success” must be told at many levels. In addition, impacts of prevention programs may not be able to be demonstrated for several years therefore communicating success during the various life stages of a program is important for long term sustainability. The presenters will use their experience with 13 national oral health grantees to demonstrate how to use success stories to build both program and evaluation capacity. The session will be a practical and hands on session enabling attendees to begin writing their own success stories. This is an expanded version of last year’s session of the same title with more time for practicing practical applications for use in your own practice. Attendees will receive the newly developed workbook: Impact and Value: Telling Your Program’s Story for use during the class and to take home for reference.

Audience: Attendees working in any context with a working knowledge of both evaluation and qualitative inquiry

René Lavinghouze is with the Division of Oral Health at CDC where she leads a multi-site, cluster evaluation designed to assess infrastructure development. Rene has over 15 years experience with CDC and in the private sector. She is Co-chair of AEA’s TIG for Cluster, Multi-site/level evaluations and serves on the communications team for the local evaluation affiliate, AaEA. Ann Price, PhD, is president of Community Evaluation Solutions, Inc and has over 20 years experience in both treatment and prevention. She has conducted evaluations in many areas including intimate partner violence, mental health, substance abuse, tobacco prevention and oral health. Prior to CES, Dr. Price was a Senior Data Analyst at ORC Macro on a multi-site national child mental health evaluation.

Offered (Two Rotations of the Same Content - Do not register for both):

• Monday, June 23, 9:25 – 12:45 (20 minute break within)

• Wednesday, June 25, 9:25 – 12:45 (20 minute break within)

Offering 30: Sampling 101: Basics of Probability and Purposeful Sampling 

Level: Beginner

Handouts: Handouts1

Description: Choosing and implementing an appropriate sampling strategy can affect the validity, credibility and cost of an evaluation. Some studies require sophisticated probability sampling methods to produce accurate estimates of the characteristics of the populations served or of the size of the effects of the program or policy on the target population. Other studies may appropriately use purposeful samples to support theory development or to do detailed case analysis. In this workshop, you will be exposed to alternative sampling strategies that are frequently used in evaluation and social research. The instructor will address the 14 questions from his book Practical Sampling (Sage, 1990) that should be answered prior to sample design, as a part of sample design, and prior to analysis of the data. You will become acquainted with ways to plan and implement sampling strategies that meet the needs of an evaluation. Examples will be used to illustrate the designs and issues that arise in implementation and you will have the opportunity to raise specific sampling issues that encountered in your own work.

Audience: Attendees who are new to sampling and working in any context

Gary T. Henry holds the Duncan MacRae ’09 and Rebecca Kyle MacRae Professorship of Public Policy in the Department of Public Policy and directs the Carolina Institute for Public Policy at the University of North Carolina (UNC) at Chapel Hill. Also, he holds the appointment as Senior Statistician at the Frank Porter Graham Institute for Child Development at UNC-Chapel Hill. He previously served as the Director of Evaluation and Learning Services for the David and Lucile Packard Foundation. Henry has evaluated a variety of public policies and programs and is the author of Practical Sampling (Sage 1990), Graphing Data (Sage 1995) and the co-author of Evaluation: An Integrated Framework for Understanding, Guiding, and Improving Policies and Programs (Jossey-Bass 2000). He received the Evaluation of the Year Award from the American Evaluation Association in 1998 for his work with the Georgia’s Council for School Performance and the Joseph S. Wholey Distinguished Scholarship Award in 2001 from the American Society for Public Administration and the Center for Accountability and Performance. 

Offered (Two Rotations of the Same Content - Do not register for both):

• Tuesday, June 24, 9:25 – 12:45 (20 minute break within)

• Wednesday, June 25, 9:25 – 12:45 (20 minute break within)

Offering 31: What Counts as Credible Evidence in Contemporary Evaluation Practice: Moving Beyond the Debates

Level: Beginner to Intermediate

Handouts: Handouts1

This workshop is designed to explore one of the most fundamental issues facing evaluators today, and the 4th step in CDC's Framework for Program Evaluation, what counts as credible evidence in contemporary evaluation practice? Many thorny debates about what counts as evidence have occurred in recent years, but few have sorted out the issues in a way that directly informs contemporary evaluation and evidence-based practice. Participants will come away from this workshop with an understanding of the philosophical, theoretical, methodological, political, and ethical dimensions of gathering credible evidence and will apply these dimensions to fundamental evaluation choices we encounter in applied settings.

Audience: Attendees should have a basic background in evaluation

Stewart I. Donaldson, Ph.D. is Professor and Chair of Psychology, Director of the Institute of Organizational and Program Evaluation Research, and Dean of the School of Behavioral and Organizational Sciences, Claremont Graduate University. He has conducted numerous evaluations, developed one of the largest university-based evaluation training programs, published numerous evaluation articles and chapters, and his recent books include Program Theory-Driven Evaluation Science: Strategies and Applications (2007), Applied Psychology: New Frontiers and Rewarding Careers (2006; with D. Berger & K. Pezdek), Evaluating Social Programs and Problems: Visions for the New Millennium (2003; with M. Scriven), Social Psychology and Policy/Program Evaluation (forthcoming; with M. Mark & B. Campbell), and What Counts as Credible Evidence in Evaluation and Evidence-Based Practice? (forthcoming; with C. Christie & M. Mark). He is co-founder of the Southern California Evaluation Association and is on the Editorial Boards of the American Journal of Evaluation and New Directions for Evaluation.

Offered (Two Rotations of the Same Content - Do not register for both):

• Monday, June 23, 2:30 – 4:00 PM

• Tuesday, June 24, 2:30 – 4:00 PM

Offering 32: Improving Survey Quality: Assessing and Increasing Survey Reliability and Validity

Level: Intermediate

Handouts: Handouts1  Handouts2

Description: Develop higher-quality surveys! This workshop is designed to teach participants how to improve survey quality, thus increasing their utility of and confidence in the data they collect. We will look at surveys to elicit factual information as well as ones that ask about subjective and abstract concepts. Through the use of hands-on activities, mini-lectures, and demonstrations participants will understand what is meant by reliability and validity with respect to surveys and will learn ways to improve each during the survey design phase for both types of surveys. Next, using a case example and SPSS we will explore ways to use pilot test responses to assess the reliability of subjective / abstract survey constructs by conducting confirmatory factor analysis and calculating Cronbach’s alpha. We will work together to understand what our findings tell us as well as what they don’t tell us, and consider other ways to assess survey quality. Last we will explore the types of validity associated with surveys and ways to assess the various forms of validity again using our case example. You will receive a workbook and SPSS screenshots to help you remember how to perform many of the computations we will perform. Participants will be surprised by how easy it is to improve survey quality through a few easy to implement steps!

Audience: Attendees working in any context with a basic background in survey development and an understanding of factor analysis.

Amy A. Germuth, PhD earned her PhD in Evaluation, Measurement and Psychology from UNC Chapel Hill and a certificate in survey methodology via a joint program through UNC-CH, UMD, Westat and RTI. She is a founding partner of Compass Consulting Group, LLC, a private evaluation consulting firm that conducts evaluations at the local, state, and national levels. . As part of Compass she has evaluated numerous initiatives, including health prevention and outreach programs, Math Science Partnerships, K-12 science outreach programs, and workforce development initiatives, and has worked with a variety of organizations including the US Education Department, Department of Health and Human Services, Westat, Georgia Tech., Virginia Tech., University of North Carolina, the New York State Education Department, multiple NC Childhood Education Partnerships, and Hawaii’s Kamehameha Schools. As part of her evaluation work she has developed and guided large-scale survey initiatives including the NC and SC Teacher Working Conditions survey. Dr. Germuth is the current chair of AEA’s Independent Consulting TIG and teaches evaluation and instrument development as part of Duke University’s Certificate Program in Non-Profit Management.

Offered (Two Rotations of the Same Content - Do not register for both):

• Tuesday, June 24, 9:25 – 12:45 (20 minute break within)

• Wednesday, June 25, 9:25 – 12:45 (20 minute break within)

Offering 33: Taking it Global: Tips for International Evaluation (new!)

Level: Beginner

Description: This session offers practical considerations for those interested in or preparing to work in evaluation overseas. It is organized in three segments. The opening segment provides an overview of the organizational context for international evaluations, highlighting key entities focused on strengthening, sharing, and supporting evaluation theory and practice around the world. The next segment focuses on the stakeholder environment, with the role of donors, host governments, local evaluation associations, and civil society contrasted with that of major players in U.S.-based evaluations. The discussion on how to reconcile the different needs and expectations of these stakeholders sets the stage for the final segment, which presents a case study illustrating common challenges encountered in the field. Participants will work in small groups to consider such issues as credibility, evaluation capacity building, cultural competence, and local ethical standards. Throughout the session, participants' own evaluation experience in the domestic arena will serve as a catalyst for discussion and concrete guidance.

Audience: Attendees with a basic background in evaluation who are currently working in, or considering working in, international contexts

Donna Podems, M.P.H., PhD. is a senior evaluation facilitator for Macro International. She has practical evaluation experience in the United States as well as numerous countries in Africa, Asia, and Latin America. An evaluation generalist, she has experience in evaluating a wide range of projects, including gender, women’s empowerment, HIV/AIDS, and youth interventions, among others. Her doctorate in interdisciplinary studies, focused on Program Evaluation and Organizational Development, is from the Union Institute and University and her Masters degree in Public Administration and BA in Political Science are from The American University. Helen Coelho, M.P.H. is a senior research associate at Macro International providing support to the Monitoring and Evaluation (M&E) Team of CDC’s Global AIDS Program for field strategic information activities. In collaboration with headquarters agency staff and their field and country counterparts, she provides remote and in-country technical assistance designed to increase M&E capacity for HIV/AIDS programs in developing countries. Ms. Coelho holds a Bachelor’s degree in Biology from Oglethorpe University and a Master’s degree in Public Health from Emory University.

Offered:

• Tuesday, June 24, 2:30 – 4:00 PM

Offering 34: Measurement for Evaluators: Key Issues in Reliability, Validity, and Instrument Design

Level: Intermediate

The session will include informative discussion on the some key issues facing evaluators in the area of measurement. We will focus on the importance of testing instruments for reliability and validity prior to data collection, define and explain the various types of reliability and validity testing, and discuss implications of not testing instruments prior to data collection. The audience will also learn about current methods being used to assess reliability and validity. We’ll also touch on other issues in measurement that are critical for today's practicing evaluators, such as questionnaire design and using technology in data collection. The presenter will bring dual perspectives from experiences in education and public health to this discussion, and will facilitate interactive activities to engage the audience in the presentation.

Audience: Evaluators working in any context with a basic knowledge of instrument development.

Frances McCarty, Ph.D. is a Biostatistician in the Behavioral Sciences and Health Education Department in the Rollins School of Public Health. She currently serves as the statistician on several grants that focus on interventions related to HIV prevention, nutrition and exercise, and HIV medication adherence. She teaches research methods and statistics courses. Prior to her position at Emory, she was an assistant project director for a five-year grant at Georgia State University that focused on Head Start program quality and child outcomes. She holds a B.S. in Health/ Physical Education from Bridgewater College and a M.Ed. in exercise physiology from the University of Virginia. She earned her Ph.D. in measurement and statistics from Georgia State University (Educational Policy Studies Department). Her research interests include the application of item response theory to instrument development and topics related to hierarchical linear modeling.

Offered (Two Rotations of the Same Content - Do not register for both):

• Monday, June 23, 9:25 – 12:45 (20 minute break within)

• Wednesday, June 25, 9:25 – 12:45 (20 minute break within)

Offering 35: Introduction to Statistics for Evaluation

Level: Beginner

Handouts: Handouts1

Description: Brush up on your statistics! We will focus primarily on descriptive statistics: measures of central tendency (e.g., the mean) and measures of dispersion (e.g., standard deviation), defining what these terms mean, when and how to use them, and how to present them to evaluation stakeholders. From there, we will move to a basic discussion of inferential statistics, the difference between a statistic and a parameter, and the use of population parameters. The workshop will involve demonstrations by the presenter using SPSS software as well as hands-on exercises for you to calculate some of the descriptive statistics we cover. This workshop is not designed to teach you how to use SPSS or any other statistical software package, but rather to introduce you to the process of statistical analysis and interpreting statistical output. We will focus on interpreting statistical indices (e.g., means and standard deviations) as well as graphical output, such as histograms and stem-and-leaf plots. The understanding you gain regarding the concepts presented in this course, coupled with the chance to interpret statistical output, should ready you to employ the most basic descriptive statistics and interpret them in your evaluation work.

Audience: Attendees who have never taken a statistics course before, or who feel they need a refresher workshop on BASIC statistics. Those who attend this workshop need no prior background in statistics

Katherine McKnight, PhD, has taught statistics workshops at AEA's annual conferences for many years. An outstanding facilitator, she is known for making difficult concepts accessible and interesting.

Offered:

• Tuesday, June 24, 9:25 – 12:45 (20 minute break within)

Offering 36: Ensuring Evaluation Use

Level: Beginner

Handouts: Handouts1  Handouts2  Handouts3

Description: Many evaluators and program staff are concerned with designing evaluations that are intended to inform more immediate decision-making and promote organizational change. A goal, then, is to offer the most useful information to answer the evaluation questions given the program context and the resources available to conduct the investigation. This session will focus on developing participants’ understanding of, and methods for increasing, evaluation use. We will begin by providing a theoretical framework for understanding and promoting evaluation use. Employing interactive and small group exercises, we will examine strategies and techniques for increasing the use of both the evaluation process and findings. Upon completion of the session, you will: understand use as a primary purpose of evaluation, understand the difference between evaluation process and evaluation findings use, become familiar with a framework for promoting use, and identify strategies for promoting and increasing evaluation use.

Audience: Those who are new to evaluation, including stakeholders in the evaluation design process, who are working in any context.

Christina A. Christie is an Associate Professor, Director of the Masters of Arts Program in Psychology and Evaluation, and Associate Director of the Institute of Organizational and Program Evaluation Research at Claremont Graduate University. Her research, which has been supported by several funders including the Haynes Foundation, focuses on investigating the relationship between evaluation theory and practice and issues related to the development of descriptive theories of evaluation. She has also received funding from a variety of sources, including the National Science Foundation, the U.S. Department of Education, and the Hewlett-Carnegie Foundation to evaluate social, education, and health behavior programs targeting high-risk and underrepresented populations. Christie co-founded the Southern California Evaluation Association, a local affiliate of the American Evaluation Association, and is the former Chair of the Theories of Evaluation Division of the American Evaluation Association. In 2004, Christie received the American Evaluation Association’s Marcia Guttentag Early Career Achievement Award.

Offered (Two Rotations of the Same Content - Do not register for both):

• Tuesday, June 24, 9:25 – 12:45 (20 minute break within)

• Wednesday, June 25, 9:25 – 12:45 (20 minute break within)

Offering 37: Systems Thinking for Public Health

Level: Beginner to Intermediate

Handouts: Handouts1

Description: The persistent challenges that show up most in public health often share common characteristics. They defy quick fixes, they build and dissipate slowly over time, they are rarely “owned” by any one player in the system but affect all, and their components are tightly coupled. The analytical approaches of systems thinking, which was invented in the 1950s at MIT, was created to address such “systems” problems. Systems thinking uses diagramming and simulation modeling to help groups of people improve their understanding of how to improve the performance of a range of social/physical systems such as a city, an ecosystem, an industry, or, in this context, a population facing a threat to their health. In this interactive session, participants will learn about a systems thinking perspective on public health, learn about causal mapping techniques, learn about insights generated by a computer-based “management flight simulator” created with the Division of Diabetes Translation at CDC, and have the opportunity to reflect on applying systems thinking to the public health challenge they face.

Audience: Attendees working in public health with a basic understanding of the many actors in and on public health settings.

Drew Jones is a project director with Sustainability Institute. His work focuses on applying systems thinking approaches to a wide range of societal concerns, particularly in the public health and environmental areas. Working with CDC’s Division of Diabetes Translation, he has led the development of the Diabetes System Model, which is shaping diabetes strategies at the national and state level. A paper on the work was recently published in the March 2006 issue of AJPH. Mr. Jones received a B.A. in Engineering Sciences and Environmental Studies from Dartmouth College and earned a M.S. from M.I.T., where he studied System Dynamics simulation modeling under John Sterman and Peter Senge.

Offered:

• Monday, June 23, 9:25 – 12:45 (20 minute break within)

Offering 38: System Dynamics Modeling: A Case Study from Diabetes (new!)

Level: Beginner to Advanced

Handouts: Handouts1

Description: This interactive session will introduce participants to the role that computer simulation can play in helping diverse stakeholders plan their public health strategies and engage new partners. Using exercises and "real time" running of a system dynamics model, this session will explore the "Diabetes System Model" created over the past five years through the Division of Diabetes Translation at CDC, and used with multiple states. This session will build on the concepts presented in the morning session, "Systems Thinking for Public Health" but attendance in that session, while encouraged, is not a prerequisite.

Audience: Attendees working in public health with a basic understanding of the many actors in and on public health settings.

Drew Jones is a project director with Sustainability Institute. His work focuses on applying systems thinking approaches to a wide range of societal concerns, particularly in the public health and environmental areas. Working with CDC’s Division of Diabetes Translation, he has led the development of the Diabetes System Model, which is shaping diabetes strategies at the national and state level. A paper on the work was recently published in the March 2006 issue of AJPH. Mr. Jones received a B.A. in Engineering Sciences and Environmental Studies from Dartmouth College and earned a M.S. from M.I.T., where he studied System Dynamics simulation modeling under John Sterman and Peter Senge.

Offered:

• Monday, June 23, 2:30 – 4:00 PM

Offering 39: Enhanced Group Facilitation: Techniques and Process

Level: All

Description: This popular and well-received workshop will familiarize participants with a variety of group facilitation techniques as well as the management of the facilitation process. You will learn how to choose a facilitation technique based on goals and objectives, anticipated outcome, type and number of participants, and logistics. Two to three facilitation techniques for generating ideas and focusing thoughts, including item writing and nominal group technique, will be explored in greater detail. We will also cover variations on these techniques and how they may be used for your facilitation purposes. Finally, participants will learn more about the different roles and responsibilities they may have in group facilitation (there are more than you think!), and how these roles intersect with the tasks inherent in planning and managing a group facilitation experience. Job aides and reference lists will be provided.

Audience: Attendees working in any context who work with, or expect to be working with, client groups of any size.

Jennifer Dewey, PhD, is a Technical Director with the research and evaluation professional services firm of Macro International Inc. Jennifer oversees a program of ongoing training and technical assistance to local evaluation teams for the national evaluation of a federally-funded children’s mental health services program under the Substance Abuse and Mental Health Services Administration (SAMHSA). Prior positions include Director of Internal Evaluation at Learning Point Associates, Senior Consultant at Andersen, and post-doctoral scholar at the Center for Prevention Research at the University of Kentucky. Jennifer holds a doctorate in Applied Experimental Psychology with a specialization in program evaluation. Her knowledge and skills encompass project management, proposal development, methodological and statistical design, qualitative and quantitative analysis, needs assessment, survey development, telephone and in-person interviews, and group facilitation. Jennifer is a 2007 and 2008 Malcolm Baldrige National Quality Award Examiner. She has published in the Journal of Primary Prevention, American Journal of Evaluation, Advances in Developing Human Resources, and has made over 40 professional conference presentations.

Offered (Two Rotations of the Same Content - Do not register for both):

• Monday, June 23, 9:25 – 12:45 (20 minute break within)

• Wednesday, June 25, 9:25 – 12:45 (20 minute break within)

Offering 40: Evaluating Culturally-tailored Health Communications

Level: Intermediate

Handouts: Handouts1

Description: This session provides answers to three basic questions: Why integrate culture in communications? What are some ways culture is operationalized in health communication? How might culturally tailored messages be developed and evaluated? It employs the definition of cultural tailoring first advanced by Pasick et al in 1996 as the recognition and inclusion of specific cultural characteristics to improve the relevance and utility of health communications. Tailoring is a procedure selected from the social marketing toolkit, and founded on the concept that any program or educational materials will be more effective when audience characteristics are taken into account in their development and dissemination. There is increasing attention being paid in the health promotion literature to the concept of culturally-relevant or culturally-appropriate programs or messages. Many of the strategies for evaluating such communications focus on fairly explicit linguistic or formatting metrics; is the language used accessible to the population, the spokespeople ethnically-appropriate, culturally-relevant constraining and facilitating factors included? This session will explore the integration and application of cultural schemas and models, concepts derived from cognitive anthropology, and the mental modeling approach, a process derived from risk communication theory, to develop and apply additional evaluative metrics for communications. The session describes how these concepts can be employed in the evaluation of culturally tailored health communication campaigns in community settings.

Audience: Attendees with a basic background in evaluation and working in the public health arena.

David Driscoll, PhD, MPH, is an anthropologist with extensive experience evaluating tailored health promotion and risk communications in with diverse audiences including efforts to promote cancer screening, immunization, risk perception, and environmental health. Michelle Jones-Bell, MA, is a Public Health Research Analyst at RTI International who experience includes moderating focus groups, facilitating group discussion, and conducting in-depth interviews with various audiences. She conducts exploratory, concept testing, and message testing for evaluation and social marketing campaigns.

Offered (Two Rotations of the Same Content - Do not register for both):

• Monday, June 23, 2:30 – 4:00 PM

• Tuesday, June 24, 2:30 – 4:00 PM

Offering 41: Evaluating the Efficacy of Health Communications and Public Health Branding

Level: Beginner to Intermediate

Handouts: Handouts1

Description: This workshop provides a decision-oriented framework for determining what to measure in evaluating health communication and social marketing campaigns. Within this framework we offer a more in-depth training in efficacy and effectiveness research design options. We describe key design features and benefits of efficacy versus effectiveness research and help participants to identify the complimentary strengths and limitations of various evaluation methods for different types of questions and audiences.
The session concludes with training about methods to evaluate public health branding. We review recent advances in public health branding and offer theory-based, practical approaches to evaluate the contribution of brand equity to changes in health behavior. The session includes hands-on practice with methods to develop measures tailored to brand image and communication focus. We discuss effective ways to provide feedback to tailor and refine health communication efforts.

Audience: Attendees with a basic background in evaluation and working in the public health arena.

W. Douglas Evans, Ph.D., is Vice President of RTI’s Public Health and Environment Division. He specializes in evaluation of social marketing campaigns and the translation of commercial marketing strategies, such as branding, to public health. He recently edited a volume on public health branding, to be published in 2008 by the Oxford University Press. Jennifer Uhrig, PhD, MHA is a Senior Health Communication Scientist at RTI whose experience includes testing and evaluation of print and Web-based interventions in the area of HIV prevention and informed health-decision-making. James Hersey, Ph.D., director of RTI’s Health Psychology Program, has worked extensively in evaluation of broadcast media and social marketing campaigns in immunization, obesity prevention, and tobacco control.

More information on the team’s work may be found at: http://www.rti.org/experts.cfm?objectid=119713C3-4480-44D8-99300B120F72BC6D

Offered (Two Rotations of the Same Content - Do not register for both):

• Tuesday, June 24, 9:25 – 12:45 (20 minute break within)

• Wednesday, June 25, 9:25 – 12:45 (20 minute break within)

Offering 42: Qualitative Evaluation in the Real World

Level: Intermediate

Description: Qualitative evaluation allows program planners and evaluators to apply qualitative methods to the questions and concerns of those in the world of practice. Participants will learn under what circumstances qualitative methods are appropriate in the real world, how to balance the limitations of field data collection, ethical considerations that must be taken into account and how to focus and present the evaluation findings to meet stated and unstated program priorities. Case studies, both domestic and international, will be presented to add context to the step by step process that will be discussed.

Audience: Attendees with a basic background in qualitative research and working in any context.

Cynthia Klein, PhD, a research and evaluation project director for Constella Group, currently manages the evaluation of CDC’s Choose Respect Initiative and the National Youth Violence Prevention Resource Center projects. Her areas of expertise include qualitative and quantitative research methodologies, health communication evaluations, field data collection, and research planning. She holds a PhD in Social Psychology from the University of Florida. Pat Shifflett, RN, MS, provides consulting for program planning and development projects for The Cloudburst Group. Ms. Shifflett has extensive experience in designing, implementing and evaluating community-based interventions. In addition, she has expertise in organization and change management, using data for decision-making and public health planning. A public health nurse for over 20 years, Ms. Shifflett, also, has a graduate degree in Organization and Management from Capella University.

Offered (Two Rotations of the Same Content - Do not register for both):

• Monday, June 23, 2:30 – 4:00 PM

• Tuesday, June 24, 2:30 – 4:00 PM

Offering 43: Using the Balanced Scorecard (BSC) in Public and Nonprofit Organizations: An introduction to the methodology, benefits and application of the BSC (new!)

Level: All

Description: The balanced scorecard is a strategic planning and management system that aligns activities to the vision and strategy of the organization, monitors the organization’s performance against strategic goals and improves internal and external communications. Drs. Robert Kaplan (Harvard Business School) and David Norton developed the BSC in the early 1990's as a performance measurement framework that added strategic non-financial performance measures to traditional financial metrics to give managers and executives a more 'balanced' view of organizational performance. This interactive workshop will introduce participants to the Balanced Scorecard methodology and provide the opportunity to engage in exercises to learn how the Balanced Scorecard can help you understand, improve and measure the effectiveness of your programs and communicate results to your stakeholders.

Patsy Nemchik provides business consulting to public and private organizations, primarily focusing in the areas of strategic planning (including the Balanced Scorecard), enterprise performance management and program development. She has held the position of Director of Management Engineering and Director of Process and Systems Improvement. Mrs. Nemchik received her Bachelor of Science degree from the Georgia Institute of Technology and is a certified facilitator for process improvement and planning. Ryan Beach helps organizations in the private and public sectors to develop strategic plans and implement strategic management systems using the Balanced Scorecard methodology. Her primary areas of experience include: strategic planning, performance management, program development and project management. Ms. Beach received her Bachelor of Science from Virginia Tech.

Offered:

• Monday, June 23, 9:25 – 12:45 (20 minute break within)

Offering 44: Evaluation Strategies for Communicating and Reporting

Level: All

Description: Communicating evaluation processes and results is one of the most critical aspects of evaluation practice. This is especially true when learning and evaluation use is an expected goal of the evaluation. Yet, evaluators continually experience frustration with hours spent on writing reports that are seldom read, or shared with others. While final reports will continue to be an expectation of many evaluation contracts, there are other ways in which evaluators can communicate and report on the progress and findings from an evaluation. Using a variety of strategies for communicating and reporting can increase learning from the evaluation’s findings, stakeholders’ understanding of evaluation processes, the evaluation’s credibility, and action on the evaluation’s recommendations. In this session, you will: explore reasons for communicating and reporting throughout an evaluation’s life cycle, consider how stakeholders’ characteristics and information needs influence the choice of communicating approaches, and learn about more than 15 strategies for communicating and reporting evaluation processes and findings.

Audience: Attendees working in any context with a basic understanding of evaluation processes as well as the relevant stakeholders to an evaluation.

Rosalie T. Torres, Ph.D. is president of Torres Consulting Group, a research, evaluation and management consulting firm specializing in the feedback-based development of programs and organizations since 1992. She earned her Ph.D. in research and evaluation from the University of Illinois and formerly was the Director of Research, Evaluation, and Organizational Learning at the Developmental Studies Center, an educational non-profit. Over the past 28 years, she has conducted more than 60 evaluations in education and nonprofit organizations, and has worked extensively with U.S. schools and districts in both internal and external evaluator roles. Drawing on this evaluation practice, she has authored/co-authored numerous books and articles including, Evaluation Strategies for Communicating and Reporting (Torres, Preskill, & Piontek, 2005), and Evaluative Inquiry for Learning in Organizations (Preskill & Torres, 1999). She has served on the AEA Board, taught graduate research and evaluation courses, and is a sought after workshop facilitator on various topics related to evaluation practice.

Offered (Two Rotations of the Same Content - Do not register for both):

• Monday, June 23, 9:25 – 12:45 (20 minute break within)

• Tuesday, June 24, 9:25 – 12:45 (20 minute break within)

Offering 45: Performance Measurement for Public & Nonprofit Organizations: Strategies for Improving the Quality of Performance Data New!

Level: Beginner to Intermediate

One of the central challenges in designing performance measurement systems lies in defining operational indicators that generate data on performance that are truly useful for policymakers, managers, and other audiences. This workshop will assist participants in identifying appropriate data sources, specifying good potential indicators, and then assessing their usefulness on a systematic basis before selecting which indicators to include in a measurement system. Participants will become familiar with a set of methodological and managerial criteria - from reliability to goal displacement - that often plague performance measures, and will learn how to anticipate problems regarding data quality as well as strategies for overcoming such problems. In addition to presentation of concepts and principles by the instructor, the course will also provide opportunities for participants to apply them to real world examples.

Audience: Attendees should already understand the basics of evaluation but may need an introduction to the concept of performance measurement.

Ted Poister, PhD has written extensively on results oriented management strategies, and his most recent book is Measuring Performance in Government and Nonprofit Organizations (Jossey-Bass 2003).  Dr. Poister is Professor of Public Administration at the Andrew Young School of Policy Studies at Georgia State University, where his teaching and research focus principally on strategic planning, performance management, public program analysis, and soliciting feedback on public services from customers and other stakeholders, as well as performance measurement.  In addition to helping numerous agencies in developing measurement systems, he has conducted training programs for various governmental units, and he regularly offers a two day course on performance measurement for the Evaluators’ Institute in San Francisco, Chicago, and Washington, D.C. Dr. Poister brings a multidisciplinary, multimethod perspective to measuring results.

Offered:

• Wednesday, June 25, 9:25 – 12:45 (20 minute break within)

Offering 46: Gender Issues in Global Evaluation

Level: Intermediate

Description: A brief review of the various gender and feminist approaches and their dissimilar histories, contexts, and critiques will set the stage for participant discussion and practical application. Participants will engage in facilitator led discussions regarding the different, appropriate, and practical applications of gender and feminist evaluation, and how these approaches have the potential to enhance various evaluation contexts throughout the ‘developed’ and ‘developing’ world. Participants will demystify feminist and gender evaluation by examining case studies from Africa and Asia in various fields including HIV/AIDS, human rights, public private partnerships, and environment. Through small group work participants will apply elements of both approaches resulting in an advanced understanding of how feminist and gender approaches can enhance evaluation and the evaluator in any context.

Audience: Attendees with a basic understanding of evaluation and working in all contexts although examples are drawn from international development.

Donna Podems, M.P.H., PhD. is a senior evaluation facilitator for Macro International. She has conducted trainings, developed M&E systems, and implemented evaluations for government agencies and grassroots and nongovernmental organizations. She has practical evaluation experience in the United States, Botswana, Cameroon, Ghana, Kenya, Madagascar, Namibia, Republic of South Africa, Somaliland, Swaziland, Uganda, Zimbabwe, Bosnia, Guatemala, Panama, Peru, Belize, Nicaragua, India, Indonesia, Pakistan, Sri Lanka, Thailand and Vietnam. An evaluation generalist, she has experience in researching, evaluating and/or developing M&E systems for a wide range of projects. Topics have included gender, women’s empowerment, HIV/AIDS, youth interventions, natural resource management, education, capacity building, human rights, public private partnerships, and community needs. Her doctorate in interdisciplinary studies, focused on Program Evaluation and Organizational Development, is from the Union Institute and University and her Masters degree in Public Administration and BA in Political Science are from The American University.

Offered:

• Monday, June 23, 2:30 – 4:00 PM

Offering 47: Management, Improvement, and Accountability: The Intersection of Program Planning, Evaluation, and Performance Measurement

Level: All

Handouts: Handouts1

Description: Confused about the differences and similarities in performance measurement and evaluation? You are not alone. They are highly related fields that share a significant amount of overlapping foundations, principles, logic, and techniques. Most importantly, these strategies are both management tools for organizations seeking to plan more effectively, implement their programs more consistently, and maximize results. These strategies help focus managers', employees', and stakeholders' attention on common priorities and the processes to meet them. This session will show how utilization-focused evaluation approaches such as CDC's Framework for Program Evaluation in Public Health help ensure that evaluation results influence program action. The session will emphasize planning and evaluation as an iterative cycle. We will explore how developing a strong foundation of an evaluation plan-stakeholder engagement and program description/logic modeling-help focus measurement and evaluation strategies on common priorities and identify areas for strategic action.

Audience: Those new to the field of evaluation and working in any context or those with some experience in the field of evaluation but with interest in how to extend their work to include performance measurement and planning.

David Cotton, PhD, MPH, is a Senior Vice President in the Applied Research Division at Macro International in Atlanta where he directs a portfolio of evaluation work for federal and non-federal clients. He and his staff provide training and technical assistance to community-based organizations and health departments in applied monitoring and evaluation, including the development of performance measurement systems.

Offered (Two Rotations of the Same Content - Do not register for both):

• Monday, June 23, 2:30 – 4:00 PM

• Tuesday, June 24, 2:30 – 4:00 PM

Offering 48: Case Study Methods for Evaluators

Level: Beginner

Handouts: Handouts1

Description: Case Study Methods allow evaluators to approach program assessment from a powerful and flexible design palette. While often heavily steeped in the use of qualitative methods, case studies also may include the use of quantitative data. The approach is particularly rich for tinting and shading the effects of programs as well as investigating important program questions in depth.

This interactive, three-hour session will provide participants with an overview and examples of case study research methods as they apply to evaluation settings. Through the development and expansion of sample case studies, by the end of the session participants will:

• comprehend the role of case study methods within the context of other evaluation approaches;

• be able to describe the elements of case study research and identify the major strengths and weaknesses of case study methods;

• understand the sequential, operational guidelines for implementing case study research;

• review techniques for establishing the validity and reliability of case study data

• strengthen data gathering and analysis skills through use of techniques common to case study research

Audience: Attendees working in any context

Dr. Rita O’Sullivan is Associate Professor of Evaluation and Assessment at the University of North Carolina at Chapel Hill where she teaches graduate courses in Educational Program Evaluation, Case Study Methods, Research Design, Measurement, and Statistics. She is also Executive Director of Evaluation, Assessment, and Policy Connections (EvAP), a unit she founded within the UNC School of Education that conducts local, state, and national evaluations. Dr. O’Sullivan has specialized in developing collaborative evaluation techniques that enhance evaluation capacity and utilization among educators and public service providers. She is senior author of Programs for At-Risk Student: A Guide to Evaluation (Corwin Press, 1993) and wrote Practicing Evaluation: A Collaborative Approach (Sage) in 2004.

Offered (Two Rotations of the Same Content - Do not register for both):

• Monday, June 23, 9:25 – 12:45 (20 minute break within)

• Tuesday, June 24, 9:25 – 12:45 (20 minute break within)

Offering 49: Exploring Effect Size and Measures of Association

Level: Intermediate

Handouts: Handout1

Description: Answer the call to report effect size and association measures as part of your evaluation results. This workshop will improve your capacity to understand and apply a range of measures including: standardized measures of effect sizes proposed by Cohen, Glass, and Hedges; Eta-squared; Omega-squared; the Intraclass correlation coefficient; and Cramer’s V. Through mini-lecture and demonstration you will improve your understanding of the theoretical foundation and computational procedures for each measure. The session will include: definitions of and procedures for computing a range of effect size and association measures, a presentation that examines the relationships among the common measures, and description of computation of selected confidence intervals for effect sizes and association measures. You will receive SPSS and SAS software program codes for performing many of the computations related to the measures and common confidence intervals.

Audience: Attendees with an understanding of basic statistics through regression and working in any context

Jack Barnette, PhD has served as a faculty member at Penn State University, University of Virginia, University of Memphis, University of Alabama at Tuscaloosa, University of Iowa, and is now Senior Associate Dean for Academic Affairs and Professor of Biostatistics at the University of Alabama at Birmingham. He has served as an APHA Statistics Council Member and Section representative to the APHA Action Board. Presently, he is chairing the ASPH biostatistics competency workgroup and is co-chair of the ASPH Biostatistics/Epidemiology Section. He has more than 30 years experience in teaching, advising students, and applying research, evaluation, and statistical methods to a wide variety of educational and public health projects. He has conducted evaluations of projects funded by CDC, HRSA, SAMHSA, NHLBI, and NIOSH. He serves on three of the ASPH/CDC Preparedness Exemplar Groups: Education and Evaluation Methods, Certificate Programs, and University-based Student Preparedness. He has been conducting research on the use of effect sizes and measures of association for the past eight years and he has presented pre-sessions on this topic at the last four AEA annual meetings. He holds the PhD in Educational Research and Development from Ohio State (1972).

Offered (Two Rotations of the Same Content - Do not register for both):

• Monday, June 23, 9:25 – 12:45 (20 minute break within)

• Tuesday, June 24, 9:25 – 12:45 (20 minute break within)

Offering 50: Public Health Evaluation: Getting to the Right Questions

Level: Advanced beginner to Intermediate

Handouts: Handouts1  Handouts2

Description: In 1999, the Centers for Disease Control and Prevention published the provide public health professionals with a common evaluation frame of reference. Beyond this basic framework, however, there are nuances and complexities to planning and implementing evaluations in public health settings. An important skill of the public health evaluator is to work with stakeholders who may have an enormous range of potential evaluation questions to arrive at a focused set of evaluation questions that are most likely to provide useful, actionable results for public health. The workshop will employ real public health examples, a role play demonstration and small group discussion to examine how to explore and then narrow the scope of possible evaluation questions to "get to the right questions" for a variety of evaluation contexts. The class will focus on strategies to work with stakeholders to identify what types of evidence will have credibility while taking into consideration issues such as politics, accountability, and rotating personnel.

Audience: Attendees working in any context and familiar with evaluation planning and implementation frameworks, such as the CDC Evaluation Framework.

Mary V Davis is Director of Evaluation Services at the North Carolina Institute for Public Health and Adjunct Faculty in the University of North Carolina School of Public Health where she teaches several advanced evaluation courses. Diane Dunet is Team Lead of the Evaluation and Program Effectiveness Team, Applied Research and Evaluation Branch in CDC's Division of Heart Disease and Stroke Prevention, where she conducts and supervises public health evaluations.

Offered (Two Rotations of the Same Content - Do not register for both):

• Tuesday, June 24, 9:25 – 12:45 (20 minute break within)

• Wednesday, June 25, 9:25 – 12:45 (20 minute break within)

Offering 51: Introduction to Qualitative Data Analysis

Level: Beginner

Handouts: Handouts1

Description: Qualitative methods, such as interviews, qualitative responses to surveys, and focus groups, can provide a wealth of information about specific points of view, cultural contexts, and the meaning behind health beliefs and behaviors. The key to accessing this information is learning how to analyze qualitative data. This session will provide step by step lessons in basic qualitative analysis. A group activity will allow participants to practice coding qualitative data, triangulating findings between multiple analysts, and developing conclusions based upon the findings.

Audience: Attendees working in any context with little or no experience undertaking qualitative data analysis.

Beverly A. Warden, PhD, MPH, is the Director of Clinical Research and Experimental Studies at Constella Group. She is currently involved in overseeing process, outcome and impact evaluation projects at Constella. She has planned and managed variety of evaluation projects involving collection and analysis of both qualitative and quantitative data. She holds a PhD in Biomedical Sciences from Northeastern and an MPH from the University of North Carolina at Chapel Hill. Jennifer T. Lyden, MPH is a Survey Methodologist and field studies manager at Constella group.   She has worked on numerous qualitative public health studies.  She holds an MPH from the University of North Carolina at Chapel Hill.

Offered (Two Rotations of the Same Content - Do not register for both):

• Monday, June 23, 2:30 – 4:00 PM

• Tuesday, June 24, 2:30 – 4:00 PM

Offering 52: An Introduction to Economic Evaluation

Level: Beginner

Handouts: Handouts1

Economic evaluation refers to applied analytic methods used to identify, measure, value, and compare the costs and consequences of prevention and treatment strategies. This course provides an overview of these methods, including cost analysis, cost-effectiveness analysis (CEA), and cost-benefit analysis (CBA) with an opportunity for hands-on application of each. You will leave understanding when and how to apply each method appropriately in a range of evaluation contexts.

Audience: Attendees with a basic background in statistical methods and understanding of evaluation.

Phaedra S. Corso, Ph.D. is currently an Associate Professor in the Department of Health Administration, Biostatistics, and Epidemiology at the University of Georgia College of Public Health. Previously, she served as the senior health economist in the National Center for Injury Prevention and Control at CDC where she worked for over a decade in the areas of economic evaluation and decision analysis, publishing numerous articles on the cost-effectiveness of prevention interventions and co-editing a book on prevention effectiveness methods in public health. She holds a Master’s degree in public finance from the University of Georgia and a Ph.D. in health policy and decision sciences from Harvard University.

Offered (Two Rotations of the Same Content - Do not register for both):

• Monday, June 23, 2:30 – 4:00 PM

• Tuesday, June 24, 2:30 – 4:00 PM

Offering 53: Six Steps to a Good Evaluation Plan New!

Level: Beginner

Handouts: Handouts1  Handouts2

A good evaluation requires a detailed evaluation plan. A strong evaluation plan should:

• Lay out a cohesive approach to conducting the evaluation and using the results

• Guide the evaluation by explaining what the activities are, when the activities should occur, how they will be accomplished, and who is responsible for completing them

• Document the process for all stakeholders

• Ensure implementation of the evaluation

This course will walk participants through a step-by-step approach to developing and writing an evaluation plan which will guide the evaluation 's implementation and assure the standards of utility, feasibility, propriety and accuracy are met. 

Audience: Attendees working in any context.

Maureen Wilce and Kai Young, MPH work in the Division of TB Elimination (DTBE), CDC, where, among other duties, they lead the TB Evaluation Working Group, which produced “A Guide to Developing a TB Program Evaluation Plan” on which this session is based.  Maureen is team lead of the Program Evaluation Team. Prior to joining CDC, Maureen worked as an evaluator for a private consulting firm and for other government agencies.  She is a skilled evaluator with experience in survey instrument design, data collection and analysis, site visiting, and database manipulation.  Kai is a program evaluation specialist.  She currently leads the development and implementation of the National Tuberculosis Indicators Project, a monitoring system for tracking TB program progress toward objectives and prioritizing evaluation efforts. She is also  past- president of the Atlanta-area Evaluation Association. 

Offered (Two Rotations of the Same Content - Do not register for both):

• Monday, June 23, 2:30 – 4:00 PM

• Tuesday, June 24, 2:30 – 4:00 PM

SCHEDULE OVERVIEW ¨ WORKSHOP ¨ INDEX OF CONCURRENT SESSIONS ¨ KEYNOTES ¨ SESSIONS