| Session Title: The Theory and Practice of Evaluation in The Research to Sustainability Continuum |
| Multipaper Session 784 to be held in Palos Verdes A on Friday, Nov 4, 4:30 PM to 6:00 PM |
| Sponsored by the AEA Conference Committee |
| Chair(s): |
| Leslie Goodyear, National Science Foundation, lgoodyea@nsf.gov |
| Discussant(s): |
| Gabriel M Della-Piana, Independent Consultant, dellapiana@aol.com |
| Abstract: Moving from basic educational research to the adoption and sustainability of evidence-based practices is a significant issue for educators, researchers, and funders. Evaluation is an integral part of this process. In this session, the research to sustainability continuum will be presented and discussed from multiple perspectives. The panels' viewpoints merge around the values inherent in the development and adoption of evidence-based practices in both formal and informal educational settings. The "stages" of (1) basic educational research; (2) development of educational practices; (3) replication and field testing; (4) scaling; and (5) adoption/institutionalization/sustainability provide a common foundation for discussing the theory and practice of evaluation in the research to sustainability continuum. |
| The Role of Evaluation in the Basic Research to Practice Continuum and an Example |
| Linda Thurston, National Science Foundation, lthursto@nsf.gov |
| Evaluation is a critical tool in the continuum that reaches from basic educational research to the adoption and sustainability of evidence-based practices in educational settings. At each juncture of the continuum, evaluation provides the evidence upon which to make decisions to continue forward toward the next stage or to return to the previous stage with an aim of improvement. This presentation will describe the basic stages of the continuum that are common to many evaluations and will use an example of the evaluation of a program for low-income women, Survival Skills for Women, from basic research to state-wide adoption during the years of the welfare reform movement. The program, developed by researchers 30 years ago, is still being implemented. The basic evaluation questions for making decisions at each stage of the continuum for this educational program will be described. |
| Examining the Value Assumptions Underlying the Research to Practice Continuum: Implications for Critical Tensions in Evaluation Practice |
| Connie Kubo Della-Piana, National Science Foundation, cdellapi@nsf.gov |
| Evaluation has roots in the confluence, competing demands, and tensions among expert practical knowledge, constraints of the social sciences, and the value underpinnings of the humanities. This paper examines the value assumptions underlying the interdependence of research, evaluation, and practice in developing program theory and setting standards of evidence. The paper argues that the inherent "pushes and pulls" of this interdependent relationship can lead to clarity, ambiguity, and paradoxes in evaluation practice. Implications are drawn for responding to several key critical tensions relevant to value assumptions in evaluation practice: fidelity to treatment and expert practitioner knowledge in adapting to context; choice of critical competitor or rival treatment comparison groups and the more common practice of "no treatment" or "extant groups", without observation of implementation in each case; and representation of programs as simple, complicated, or complex in relation to the organizational capacity for program development, implementation, and evaluation. |
| Utilizing the Research to Sustainability Continuum for Evaluation Capacity Building |
| Jan Middendorf, Kansas State University, jmiddend@ksu.edu |
| Providing quality evaluations for educational programs and projects requires evaluators to utilize basic paradigms and models to communicate the role of evaluation in decision-making at various stages in the journey from research to sustainability. This presentation will demonstrate using the continuum to facilitate important evaluation discussions in educational evaluation conducted by the Office of Educational Innovation and Evaluation at Kansas State University. Researchers must provide basic findings that can be translated into successful practices. Educators must utilize research-based practices, programs and products to assure success with learners. Administrators must make decisions about adopting and sustaining programs and practices. The research to sustainability model provides an important tool for evaluators to use to describe the relevance and value of evaluation at each step in the continuum to clients and stakeholders. This educative role of the evaluator promotes the capacity of evaluation users to collaborate in decisions about the evaluation process. |
| Evaluation of Discrete Steps in the Process of Institutionalizing Educational Practices: Are We Asking the Right Questions? |
| Leslie Goodyear, National Science Foundation, lgoodyea@nsf.gov |
| Underlying this continuum are assumptions that educational programs are progressive, that research informs that progress toward scale up and sustainability, and that implementation fidelity is required in order to move through the continuum and evaluate the effects of the program over these stages. This presentation will address the question: What happens when these assumptions do not mirror reality? It will focus on a program evaluation that is confronted with a program that has little if any implementation fidelity yet is scaling rapidly and could reach sustainability in many sites soon. The presenter will discuss: What questions should be asked in such cases? What challenges to the research to sustainability continuum are surfaced by such exceptions to the rule? |