Introducing the AJE Podcast Series! This podcast will dive deeper into selected articles published in AJE. Each episode will include an interview with the author to uncover insights form each article. In addition, each article will be open to all for two weeks (June 25 - July 8) at the time of publication. Click on the videos below to watch each episode.
American Journal of Evaluation Section on International Developments in Evaluation: Transforming Evaluation for Times of Global Transformation
Zenda Ofir, PhD and Deborah Rugg, PhD, pp. 47-52
Abstract
Humanity faces not only the multiple interconnected crises that define the era of the Anthropocene, but also the challenge to engineer or accelerate transformations that will help address such wicked and super-wicked problems. Evaluative practices have the potential to accompany and accelerate efforts at transformation as well as responses to transformations. Evaluations may even in themselves be transformative if they are catalytic or focus on potential tipping points. But if we are to use the full potential of evaluation, much has to change, both technically and in dealing with the political economy of evaluation. The urgency with which evaluation has to change to be a practice fit for this era prompted this inaugural section. The four articles address some of the important developments at the intersection of evaluation and transformation – supporting a call to action for all in our field who want to help create a flourishing future for our planet and all its living beings.
Zenda Ofir, a South African citizen, is an independent evaluator and scientist who has worked from local to global level across many countries and fields of work. With a PhD in Ecological Chemistry, she has a special interest in the Global South, in the relationship between people and nature, and in how the many different approaches to evaluative practice can help accelerate urgent large-scale transformations. She is a former President of the African Evaluation Association (AfrEA) and former Vice-President of the global evaluation networks IOCE and IDEAS. She has also been an Honorary Professor at Stellenbosch University and a Fellow at the Robert Bosch Academy in Berlin. She continues to advise international organisations and is currently a member of the UNDP International Evaluation Advisory Panel and Blue Marble Evaluation (BME) Advisory Council.
Deborah Rugg is an award-winning evaluation pioneer working in over 100 countries during her 40-year career. She is a lifelong advocate for global health, the underserved, and promoting evaluation evidence to improve national policies and systems. As Chair of the UN Evaluation Group she shepherded the landmark UN Resolution 69/237 on strengthening national evaluation capacity; as SDG Adviser to the US Mission to the UN in New York she successfully promoted inclusion of the “country-led evaluation” principle in the 2030 Agenda for Sustainable Development. As UNAIDS/Geneva Evaluation Director and Chair of the HIV/AIDS Monitoring & Evaluation Reference Group, she led the Global AIDS Reporting System in 193 countries and launched the first-ever M&E Field Adviser program in 7 regions in over 66 countries. She is now President of Evaluation Consultants LLC, and co-editor of International Developments in Evaluation in the American Journal of Evaluation, which highlights urgent need for transforming evaluation and evaluating sustainable development.
Evaluation Criteria for Evaluating Transformation: Implications for the Coronavirus Pandemic and the Global Climate Emergency
Michael Quinn Patton, pp.53-89
Fundamental systems transformations are needed to address the global emergency brought on by climate change and related global trends, including the COVID-19 pandemic, which, together, pose existential threats to the future of humanity. Transformation has become the clarion call on the global stage. Evaluating transformation requires criteria. The revised Organization for Economic Cooperation and Development/Development Assistance Committee criteria are adequate for business as usual summative and accountability evaluations but are inadequate for addressing major systems transformations. Six criteria for evaluating transformations are offered, discussed, and illustrated by applying them to the pandemic and the Global Alliance for the Future of Food. The suggested criteria illustrate possibilities. The criteria for judging any intervention should be developed in the context of and aligned with the purpose of a specific evaluation and information needs of primary intended users. This article concludes that the greatest danger for evaluators in times of turbulence is not the turbulence—it is to act with yesterday’s criteria.
Michael Quinn Patton is Professor of Practice in the Claremont Evaluation Center, Claremont Graduate University, and an independent evaluation and organizational development consultant based in Minnesota, USA. He is former President of the American Evaluation Association (AEA) and author of eight major evaluation books including a forthcoming 5th edition of Utilization-Focused Evaluation and a 4th edition of Qualitative Research and Evaluation Methods. He is recipient of the Myrdal Award for Outstanding Contributions to Useful and Practical Evaluation Practice, the Lazarsfeld Award for Lifelong Contributions to Evaluation Theory, and the 2017 Research on Evaluation Award, all from AEA. EvalYouth recognized him with the first Transformative Evaluator Award in 2020. His latest book on evaluating global systems transformations is Blue Marble Evaluation.
Advancing Evaluation and Learning on Transformational Change: Lessons From the Climate Investment Funds’ Transformational Change Learning Partnership
Anna Williams, Joseph Dickman, and Regan Smurthwaite, pp.90-109
Inspiring the transformational changes required to address the climate crisis, as well as other global challenges, requires acting boldly and taking risks. New approaches that integrate transformational change thinking and concepts into evaluations, and that further deepen the nexus of evaluation and learning, are urgently needed. In 2017, Climate Investment Funds established the Transformational Change Learning Partnership (TCLP) within its Evaluation and Learning Initiative to facilitate an evidence-based learning collaboration on transformational change in climate action. The TCLP developed and implemented a set of frameworks for evaluating transformational change, including dimensions, stages, and signals of transformational change in climate action. Building on this work, we have arrived at several reflections related to making evaluation a more potent tool in understanding transformation. Let us embrace this challenge and leverage the potential of evaluation to contribute to a safer, more sustainable, and more prosperous world for all.
Anna Williams is an evaluator and strategic learning expert specializing in climate change and other profound challenges affecting lives and our common future. Between 2016-2020, Anna’s work focused on the Evaluation & Learning Initiative, a bold, trailblazing effort she helped design for the $8 billion Climate Investment Funds. Anna was also the architect and leader of the Initiative’s flagship Transformational Change Learning Partnership, a global collaboration focused on advancing context-informed, systemic, equitable, and resilient change at scale. Anna holds Master’s degrees in Public Policy and Environmental Management from Duke University and a Bachelor’s degree from Colorado College.
Joe Dickman is a Senior Evaluation and Learning Specialist at the Climate Investment Funds. jdickman@worldbank.org. Joe leads the CIF Evaluation and Learning Initiative, managing a large portfolio of studies and learning activities covering key topics in climate finance and drawing upon CIF experience in clean energy, energy access, sustainable forestry and climate resilience. Prior to joining the CIF, Joe was Deputy Director of Research, Evaluation and Learning at The MasterCard Foundation, and has previously led design, monitoring and evaluation activities for several non-governmental organizations, including Mercy Corps and CARE International. Joe also served as a United States Peace Corps Volunteer in Bolivia, and holds a Master’s in Public Affairs from the Woodrow Wilson School at Princeton University.
Regan Smurthwaite is an international development evaluation professional interested in using systems thinking to address climate change and ensure access to clean water and sanitation. She currently works on issues related to transformational change in climate action with the Climate Investment Funds (CIF) Evaluation & Learning Initiative. Prior to joining the CIF, Regan completed a graduate fellowship in the U.S. Embassy in Kathmandu and a Human Rights and Economic Justice fellowship in the office of an Indian Member of Parliament. She holds a Master’s degree in Public Policy from the Harvard Kennedy School.
Development Trajectories and Complex Systems–Informed Theories of Change
Aaron Eduardo Zazueta, Thuy Thu Le, and Nima Bahramalian, pp.110-129
This article presents an approach to use complex adaptive systems thinking to construct a theory of change to plan and evaluate transformational interventions. The article draws on the concepts of domains, scales, agents, and emergence to build a model of the system targeted by the intervention, to identify the chains of causality driving the system, and to identify the most influential enabling conditions affecting the system's development trajectory. Using the case study of the Indonesia SMART-Fish project, the article illustrates how to use such a theory of change to understand how a program interacts with the phenomena and how to assess the extent to which a program contributes to a development trajectory consistent with the intended long term transformational objectives. The article also illustrates how to use network analysis tools and simple data visualization techniques in ways that engage stakeholders in evaluation design, data collection and analysis.
Aaron E. Zazueta has led multiple evaluations on global environmental issues and sustainable development over the last 30 years for the World Bank, the Global Environment Facility, the Green Climate Fund, several UN agencies, bilateral organizations, and private foundations.
Thuy Thu Le is Evaluation Officer of the United Nations Industrial Development Organization (UNIDO). For the last 20 years, she has enjoyed conducting and managing evaluations, research studies and impact assessment of development programmes and projects around the world, across thematic areas including environment management, private sector development and poverty reduction at different UN and international development organizations.
Nima Bahramalian is an industrial development officer at the United Nations Industrial Development Organization (UNIDO) with over 10 years of experience in managing technical assistance projects promoting sustainable development of productive sectors in South America, Africa and Asia.
Reflecting on Our Times: Valuing Transformative Leadership in Real-World “Living Systems”
John Atkinson, Florence Lasbennes, and David Nabarro, pp. 130–138
We present this brief reflection on key aspects of the 2030 Agenda for Sustainable Development and the implications of the worldwide focus on achieving the sustainable development goals as external observers of the evaluation endeavor around the world. We have conducted and participated in evaluations, but it is not our primary field of work and we are not engaged in the global community of evaluation specialists. However, we believe that the urgency of the challenges confronting the world today should inspire those influencing and shaping evaluation internationally to focus much more fiercely on the value of evaluation and its implications for leadership at all levels and in all fields of work. We propose that evaluation as practice should support and help inspire, value, and evaluate the type of leadership that the world needs now—dynamic and purposeful “living systems” leaders working toward large-scale, drastic change.
John Atkinson is a designer, architect, mentor and catalyst for whole system transformation. He has instigated and led projects around the world in public and corporate settings that helped people generate fundamental change in their work and lives. He has been at the forefront of developing understanding of working with human organisations as living systems for over 25 years. John has played a key role in several recent UN processes. As advisor to the Secretary-General’s special facilitator for Nature-Based Solutions (NBS) and to the United Nations Environment Programme he helped shape approaches that made a fundamental and significant impact on people’s relationship with nature. This included the design and delivery of the NBS Momentum Day as part of the Climate Action Summit at the UN General Assembly 2019.
Florence Lasbennes is a French agronomist, specialized in rural economy. Florence has worked in several African countries where she led multi-stakeholder dialogues and partnerships on land and natural resources management, food and nutrition security and post-conflict economic reconstruction. She has worked for the French Ministry of Foreign Affairs for 10 years, before joining the United Nations where she managed large scale multi-stakeholder and multi-sector initiatives on food and nutrition for another 10 years. Since 2018, Florence is the Managing Director of 4SD (Skills, Systems and Synergies for Sustainable Development), a social enterprise that she created with David Nabarro to enable changemakers to be effective for equity, justice and regenerative futures.
David Nabarro is the Co-Director of the Imperial College Institute of Global Health Innovation and supports systems leadership for sustainable development through his Switzerland based social enterprise 4SD (Skills, Systems and Synergies for Sustainable Development). Currently David is Special Envoy of WHO on COVID-19 and Senior Advisor to the United Nations Food Systems Summit. David secured his medical qualification in 1974 and has worked in over 50 countries in multiple roles. He had functions within the UN System between 1999 and 2017. He held a range of positions including Special Representative of the UN Secretary-General (S-G) for Food Security and Nutrition, and Under-Secretary-General (and Special Adviser to the S-G) on the 2030 Agenda for Sustainable Development and Climate Change. In October 2018, David received the World Food Prize together with Lawrence Haddad for their leadership in building coalitions for action for better nutrition across the Sustainable Development Goals.
Please note, articles will be open access from June 25th at 12:01 am PDT - June 8 at 11:59 pm PDT.