|
Session Title: The Need for Mixed Methods in Advocacy Evaluation
|
|
Panel Session 494 to be held in Panzacola Section F1 on Friday, Nov 13, 10:55 AM to 11:40 AM
|
|
Sponsored by the Advocacy and Policy Change TIG
|
| Chair(s): |
| Brian C Quinn, Robert Wood Johnson Foundation, bquinn@rwjf.org
|
| Abstract:
Foundations are increasingly supporting advocacy to expand health insurance coverage and achieve other important social goals. As resources devoted to such efforts grow, the need to evaluate them is also growing. No single evaluation methodology can capture the complexity of the advocacy process, assess advocacy outcomes, and describe the contextual factors that influence policy. To evaluate a foundation-sponsored 12-state program funding consumer advocacy networks in 12 states to support expanding health insurance coverage, Mathematica Policy Research, Inc. (MPR) designed an evaluation using capacity assessment surveys, focus groups, and site visits with advocates; structured interviews with policymakers; and a social network analysis (SNA) of consumer advocacy coalitions to describe and measure advocacy capacity, advocacy strategies, and policy changes within and across states and over time. In this panel, MPR will show why multiple methods are critical for advocacy evaluation and discuss the challenges of applying SNA to evaluate advocacy coalitions.
|
|
A Multi-Method Approach to Evaluating Consumer Advocacy to Expand Health Insurance Coverage
|
| Debra A Strong, Mathematica Policy Research Inc, dstrong@mathematica-mpr.com
|
|
As programs supporting advocacy to achieve social goals grow, the need to evaluate them is also growing. Funders need to ensure their dollars are invested wisely. They also want to use evaluation to maximize opportunities for grantees to succeed in specific advocacy efforts. To meet these needs, advocacy evaluations must provide real-time feedback, emphasize interim outcomes, use meaningful measures, and above all, be flexible. No single evaluation methodology can meet these ambitious goals. The first paper in this panel will describe how and why Mathematica Policy Research designed a multi-method evaluation of a 12-state program to support consumer advocacy coalitions, why specific methods were chosen, and how results have been developed and provided to the foundation, advocates, and other stakeholders. It will discuss whether and to what extent evaluation results provided by different methods appear to be useful to the foundation, to advocates, and to technical assistance providers.
|
|
|
Applying Social Network Analysis to Understand Advocacy Coalitions
|
| Todd C Honeycutt, Mathematica Policy Research Inc, thoneycutt@mathematica-mpr.com
|
|
As part of an evaluation of a consumer advocacy project, Mathematica Policy Research (MPR) is using social network analysis (SNA) to assess the structure and operation of 12 advocacy leadership coalitions. SNA is an important evaluation tool that for understanding and quantifying relationships among organizations, showing how networks change over time, and considering whether relationships are associated with program outcomes. The insights that SNA brings to advocacy networks can be valuable only if results are adapted and interpreted to meet their needs. This paper will explain how MPR is applying SNA using data from coalition meeting records and a survey of coalition members to understand how organizations work together, share values, and make contact with policy makers. We will discuss the challenges to using SNA to provide formative feedback, and the adaptations made to keep results meaningful for advocates.
| |