|
Session Title: ACT I: Exploring the Complexities of Evaluating Infrastructure Development in Cluster, Multi-site, or Multi-level Initiatives
|
|
Panel Session 406 to be held in California D on Thursday, Nov 3, 2:50 PM to 4:20 PM
|
|
Sponsored by the Cluster, Multi-site and Multi-level Evaluation TIG
|
| Chair(s): |
| Rene Lavinghouze, Centers for Disease Control and Prevention, shl3@cdc.gov
|
| Discussant(s):
|
| Rene Lavinghouze, Centers for Disease Control and Prevention, shl3@cdc.gov
|
| Abstract:
ACT I: As evaluators and researchers of public health systems and policies we know that systems, environmental, and policy interventions are dynamic, complex, and unpredictable. One task is, through evaluation, to understand the stages of growth and development of a given initiative. Then and only then can optimal alignment of focus, people and resources for achievement of program goals be actualized. The task is complex and daunting because the line from infrastructure elements to distal outcomes is circuitous at best and does not lend itself to simple evaluation solutions. The complexity of the evaluation is further increased when programs involve multiple-sites and/or multiple-levels of implementation. A mixture of quantitative, qualitative, and out-of-the-box methods are required. This panel will discuss the complexities of evaluating infrastructure and its link to outcomes across sites and levels of program implementation as well as the relevance of evaluating infrastructure to the overall logic model.
|
|
The Difficulties and Complexities of Evaluating "Inputs:" The Forgotten Box of the Logic Model
|
| Rene Lavinghouze, Centers for Disease Control and Prevention, shl3@cdc.gov
|
|
Systems, environmental, and policy interventions are dynamic, complex, and unpredictable. Many initiatives in multi-site or multi-level evaluations have similar outcome measures but often dissimilar or unpredictable inputs. As the foundation to achieve outcomes, the "inputs" or infrastructure of an initiative are often overlooked in the evaluation plan or logic model. The challenge is to consider how to evaluate the effect of typical inputs such as partnerships, leveraging of resources, or leadership roles as direct or indirect links to the achievement of outcomes. Omitting these aspects in the evaluation plan, initiatives can fail to realize the importance of core infrastructure elements for intervention development and sustainability. This is true for all types of multi-site, multi-level initiatives. Mixed methods and "out-of-the-box" methods are often required to evaluate the complex nature of infrastructure elements. This presentation will focus on the complexities of evaluating infrastructure elements and making necessary links between inputs and outcomes.
|
|
|
Evaluating the Infrastructure of Success
|
| Martha Engstrom, Centers for Disease Control and Prevention, mengstrom@cdc.gov
|
| Rene Lavinghouze, Centers for Disease Control and Prevention, shl3@cdc.gov
|
| Ray Maietta, ResearchTalk, ray@researchtalk.com
|
|
Public health programs often have well defined outcome measures such as the Key Outcome Indicators for tobacco control and prevention programs (www.cdc.gov/tobacco). However, these multi-site initiatives often lack the ability to delineate the exact nature of the inputs required to achieve those outcomes and program sustainability. This is particularly complex when considering multi-site/multi-level programs where contexts for infrastructure and implementation differ. Using the success case method, we examined successful tobacco control programs to identify the core infrastructure elements that were required to achieve program outcomes and sustainability. This presentation will focus on the evaluation methods employed and the need to carefully evaluate the inputs section of the logic model for program success and sustainability.
| |
|
Moving Quality to the Community Level: The Challenges and Possibilities of Taking Improvement Efforts Outside the Walls of Individual Health Care Organizations
|
| Dennis Scanlon, Pennsylvania State University, dpscanlon@psu.edu
|
| Jeff Alexander, University of Michigan, jalexand@umich.edu
|
| Jeff Beich, Pennsylvania State University, jjb235@psu.edu
|
| Laura Bodenschatz, Pennsylvania State University, ljb191@psu.edu
|
| Jillian Harvey, Pennsylvania State University, jbh221@psu.edu
|
|
Many health care leaders are promoting improving health care at the community level (AHRQ; Berwick; Lavizzo-Mourey; IOM). The motivation for this approach is the belief that combining the efforts and resources of health plans, purchasers, providers, public and private agencies, and consumers into a unified initiative will result in better and more sustainable outcomes than the independent activities of the participants (Beich/Scanlon). Using data from the Aligning Forces for Quality (AF4Q) initiative, this presentation will focus on using qualitative methods to evaluate how communities are approaching infrastructure development and to ultimately develop a framework for community quality improvement infrastructure. The presentation will also discuss the process and challenges associated with using large amounts of qualitative data (interviews and secondary documents) collected across multiple sites, stakeholders, and time periods.
| |
|
Infrastructure Development for the Continued Provision of Evaluation Technical Assistance Through the Establishment of a National Network
|
| Karen Debrot, Centers for Disease Control and Prevention, kdebrot@cdc.gov
|
|
Infrastructure development is important for organizations that want to survive, especially given potential future funding limitations. Having a strong infrastructure is akin to a tree having a healthy root system, allowing it to withstand droughts and heavy storms alike. Identifying areas for infrastructure development and devoting resources to this development will likely pay long-term dividends for organizations that can plan ahead. The Office on Smoking and Health (OSH), Centers for Disease Control and Prevention (CDC), has begun to explore expansion of its evaluation technical assistance infrastructure through the development of a national level evaluation network. This strategy will be reviewed and discussed in the context of ensuring that CDC's current level of evaluation technical assistance can continue to be provided to states and local tobacco control programs in the future.
| |