|
Session Title: ACT II: Exploring the Complexities of Evaluating Infrastructure Development in Cluster, Multi-site, or Multi-level Initiatives
|
|
Panel Session 456 to be held in California D on Thursday, Nov 3, 4:30 PM to 6:00 PM
|
|
Sponsored by the Cluster, Multi-site and Multi-level Evaluation TIG
|
| Chair(s): |
| Rene Lavinghouze, Centers for Disease Control and Prevention, shl3@cdc.gov
|
| Discussant(s):
|
| Rene Lavinghouze, Centers for Disease Control and Prevention, shl3@cdc.gov
|
| Abstract:
ACT II: As evaluators and researchers of public health systems and policies we know that systems, environmental, and policy interventions are dynamic, complex, and unpredictable. One task is, through evaluation, to understand the stages of growth and development of a given initiative. Then and only then can optimal alignment of focus, people and resources for achievement of program goals be actualized. The task is complex and daunting because the line from infrastructure elements to distal outcomes is circuitous at best and does not lend itself to simple evaluation solutions. The complexity of the evaluation is further increased when programs involve multiple-sites and/or multiple-levels of implementation. A mixture of quantitative, qualitative, and out-of-the-box methods are required. This panel will discuss the complexities of evaluating infrastructure and its link to outcomes across sites and levels of program implementation as well as the relevance of evaluating infrastructure to the overall logic model.
|
|
Assessing Emergency Communication Infrastructure
|
| Keri Lubell, Centers for Disease Control and Prevention, kgl0@cdc.gov
|
|
During a public health emergency, CDC's Emergency Communication System (ECS) integrates relevant communication from across CDC and ensures coherent risk communication information reaches the public, affected communities, and partners. Principles of emergency risk communication indicate that messages need to be timely, scientifically accurate, and consistent to decrease emergency-related morbidity and mortality. Carrying messages to the intended audiences requires substantial and properly functioning communication infrastructure: connections between CDC and local and state public information officers, clinicians, and other community based organizations through which risk information can be shared; trust in CDC as a source; and transparency about what is not yet known. Before the impact of CDC's emergency messaging can be assessed, we first need to know the extent to which we have effectively built the communication infrastructure that supports message dissemination during emergency events. We will present qualitative data from a systematic investigation of CDC's emergency communication infrastructure.
|
|
|
Using Qualitative Research to Understand Public Health Infrastructure
|
| Ray Maietta, ResearchTalk, ray@researchtalk.com
|
| Rene Lavinghouze, Centers for Disease Control and Prevention, shl3@cdc.gov
|
| Judith Ottoson, Independent Consultant, jottoson@comcastnet
|
|
We have identified 5 essential elements of public health infrastructure. These elements are tangible, visible and accessible for study: 1. A Strategic Plan, 2. Effective Leadership, 3. Active Partnerships, 4. Managed Resources, and 5. Engaged Data. As we learn more about these elements we witness less tangible, less visible factors that fuel the system that builds infrastructure. Evolving strategic understanding and tactical action of key actors within any system shapes immediate and long-term action and potential of a public health infrastructure.
Qualitative research methods are necessary to understand the interplay of the tangible essential elements and less visible driving forces of such strategic understanding and tactical action. Maietta's Sort and Sift, Think and Shift method has been successfully applied to examine holistic and evolving case stories of state programs in oral health and smoking and health. These studies inform the story of understanding evolving public health infrastructure.
| |
|
Infrastructure: It's More Interesting Than You Think!
|
| Judith Ottoson, Independent Consultant, jottoson@comcastnet
|
| Rene Lavinghouze, Centers for Disease Control and Prevention, shl3@cdc.gov
|
| Ray Maietta, ResearchTalk, ray@researchtalk.com
|
|
The overall purpose of this case study was to understand whether and how the infrastructure of state oral health programs impacts progress towards oral health outcomes. Using Yin's approach to case study design, data were collected through meetings and site visits with state oral health personnel and their partners in four states. A dual analytical approach examined data within and across states. The Ecological Model of Oral Health Infrastructure emerged from the data and was used to identify five "essential elements" of oral health infrastructure including the state plan, partnerships, leadership, resources, and engaged data. "Strategic thinking" and "tactical action" are needed to support these elements. Infrastructure enables states and their partners to engage and create oral health opportunities, to offer a maximum response, and to track and sustain results. The case study offers general recommendations about infrastructure support, as well as, strategy-specific recommendations tied to the Ecological Model.
| |
|
He Said / She Said / They Said / It Said: Authenticity and Meaning in Qualitative Data Analysis
|
| Patrick Koeppl, Deloitte Consulting, pkoeppl@deloitte.com
|
|
Mixed methods are often the best way to conduct comprehensive evaluation of complex systems and situations. In-depth interviews, focus groups, dyads, triads, participant observation, archival research, literature and policy reviews and analyses, and many other methods each have a place and role in the development of understanding. Determining the validity and reliability of qualitative data collected via mixed methods both poses challenges to and provides opportunities for authentic understanding of complex systems and phenomenon. Triangulization, intersubjectivity, and introspection all contribute to study validity and subject/source reliability, but only go so far. This presentation describes the path to authentic understanding of public health infrastructure via a mixed-methods approach. Data stability, reproducibility, and accuracy are considered, as are the pros and cons of face, criterion, construct, and predictive validity as they relate to the application of qualitative data collection and analysis techniques and the quest for "useful" evaluation.
| |