|
Session Title: Strategy as Evaluand: Quality and Utilization Issues in Evaluating Strategy
|
|
Panel Session 690 to be held in Texas E on Friday, Nov 12, 4:30 PM to 6:00 PM
|
|
Sponsored by the Evaluation Use TIG
, the Organizational Learning and Evaluation Capacity Building TIG, and the Non-profit and Foundations Evaluation TIG
|
| Chair(s): |
| Michael Quinn Patton, Utilization-Focused Evaluation, mqpatton@prodigy.net
|
| Abstract:
Strategy as a new unit of analysis for evaluation involves a different purpose and use: strategic use. Traditionally, evaluation has focused on projects and programs. Organizational development makes the organization the unit of analysis for assessing effectiveness, usually focused on mission fulfillment. Management, in contrast, often focuses on strategy as the defining determinant of effectiveness. Herbert Simon, one of the preeminent management and organizational theorists, posited that the series of decisions which determines behavior over some stretch of time may be called a strategy. Distinguished management scholar Henry Mintzberg in his recent book Tracking Strategies defines strategy as "pattern consistency in behavior over time" (2007: 1). Philanthropy, government, and non-profits, greatly influenced by business management trends, are paying a great deal of attention to strategic issues. This session will examine issues of quality and utilization when focusing on strategy as an evaluand, or unit of analysis and impact.
|
|
Evaluating Strategy: Issues in Treating Strategy as the Evaluand
|
| Michael Quinn Patton, Utilization-Focused Evaluation, mqpatton@prodigy.net
|
|
Strategy is a new unit of analysis for evaluation, one that involves a different purpose and use: strategic use. Traditionally, evaluation has focused on projects and programs. Organizational development makes the organization the unit of analysis for assessing effectiveness, usually focused on mission fulfillment. Strategy as the unit of analysis, or evaluand, involves evaluating how decisions are made and executed with regard to how a broad initiative expects to have an impact (strategy as perspective) and where that impact will be made (strategy as position or niche). Issues of evaluation quality and utilization at the level of strategy begin with conceptualizing just what is meant by strategy and identifying characteristics of the strategy that can be evaluated. Strategy as an evaluand, or unit of analysis and impact, can be evaluated using such criteria as consistency, alignment, coherence, utility, and capacity to inform choices and decisions.
|
|
|
Strategy Evaluation Case Example: A Retrospective Strategy Evaluation of a 20-year Philanthropic Grant Program
|
| Patricia Patrizi, Patrizi & Associates, patti@patriziassociates.com
|
|
In 2006 the Robert Wood Johnson Foundation commissioned a strategic assessment of its investments to improve care at the end of life. The evaluators sought to document strategy and field level effects related to the portfolio as a whole of 337 grants made over 20 years as opposed to those associated with individual grant programs. This presentation addresses the many lessons surfaced during this evaluation affecting every aspect of evaluation design. For instance: what is strategy?
| |
|
Experience From the International Development Research Centre in Doing and Using Strategic Evaluation
|
| Tricia Wind, International Development Research Centre, twind@idrc.ca
|
|
The International Development Research Centre (IDRC) is a Canadian crown corporation that supports research in developing countries. IDRC undertakes evaluate work at different levels: strategic evaluations; program and project evaluation; and ongoing learning. “Strategic” evaluations examine issues that cut across the Centre’s eighteen programs. They have focused on programming modalities and broad corporate result areas like capacity building and policy influence. In a decentralized, utilization-focused approach to evaluation, IDRC has taken the view that strategic-level evaluation requires separate studies, as opposed to “rolling up” data from program or project evaluations. This panel will highlight how IDRC has framed and used strategic evaluations. It will also discuss the findings of a 2009 study, which reviewed recent strategic and program evaluations in light of IDRC’s Corporate Strategy. This study drew out a number of “tensions” that exist in IDRC programming. Naming these tensions has proved helpful for subsequent evaluation and strategy development.
| |
|
An Experiment in Evaluating Strategy: The WK Kellogg Foundation’s Devolution Initiative
|
| Kay Sherwood, Independent Consultant, kay.sherwood@verizon.net
|
|
In 1996, the United States government began a historic strategic shift called “devolution” – shifting powers, responsibilities, and funding from the federal level of government to the state level, and sometimes the local level, for a number of social welfare programs, beginning with the cash assistance program for low-income families. The law overturned a social safety net that had been built over several decades. It was politically divisive.
Soon after this historic welfare reform became law, the Kellogg Foundation began its own devolution journey, eventually committing seven years and $56 million to a project with 31 grantees as well as $3.6 million to an external evaluation of the “Devolution Initiative.”This case looks at strategy, and the evaluation of strategy, at three levels, beginning with the Congressional action of 1996, moving to the Kellogg Foundation’s responding Devolution Initiative, and then to the external evaluation of the Devolution Initiative.
| |