Evaluation + Design


    Everything we evaluate is designed. Every evaluation we conduct is designed. Every report, graph, or figure we present is designed. In our profession, design and evaluation are woven together to support the same purpose—making the world a better place. By considering both as parts of a whole, we can advance that purpose.  

    This year, we will consider the integration of design and evaluation in three areas. 

    Program Design: We call the intentional actions that organizations take to improve the social or natural world programs. Today,organizations of every kind are designing and implementing programs. They include:

    • nonprofit organizations serving the most critical needs of communities with education, human services, and health programs;
    • government agencies piloting innovative solutions to longstanding social problems;
    • philanthropists supporting collaborative programs designed to create collective impact;
    • corporations implementing social responsibility programs that promote social equity and environmental sustainability;
    • and public-private initiatives, such as pay-for-success funding, social enterprises, and impact investing, that leverage private capital for social good.

    Every program is designed, yet the field of evaluation has not developed a systematic approach to designing programs. Should we? Can we? What would it look like? What role should evaluators play? How can evaluation be built into a program from the start? Can we design for sustainability? What does it mean for a program to have an exemplary design?

    Evaluation Design: An evaluation design integrates evaluation theories, approaches, and methods to achieve a set of intended purposes in a specific context. An evaluation design encompasses more than research design. It also includes:

    • the use of evaluation as a direct means of creating change, for example by working with stakeholders in ways that promote equity and empower communities;
    • strategies for understanding, working with, and building consensus among stakeholders;
    • the thoughtful combination of quantitative and qualitative methods;
    • the application of evaluation for formative, summative, and developmental purposes;
    • and the development of evaluative criteria that establish what matters, to whom, and why.

    Evaluation design is dynamic, changing as programs develop and real-world challenges emerge. Given this complexity, what constitutes an exemplary evaluation design? Does flexibility and responsiveness come at the expense of quality, credibility, or usefulness? Can we learn to design evaluations that are faster, cheaper, and better? How do we balance the competing interests of stakeholders?

    Information Design: Evaluators must communicate to diverse audiences about the complexities of programs and their impacts. They strive to be accurate, compelling, and clear. At the same time, they are constrained by the time, attention, and training of audiences. To be successful, evaluators must develop strong information design skills, such as:

    • data visualization techniques that transform mind-boggling complexity into clear, meaningful images;
    • real-time data displays that help managers make decisions better and more quickly;
    • storytelling that speaks to the part of our brains hardwired by evolution to learn from narrative;
    • and multi-faceted communication strategies that leverage social psychology and social media to promote appropriate and timely use.

    Information design plays a central role in evaluation. But what is good information design? How should it be taught and learned? Can we be persuasive and accurate at the same time? Have evaluation reports become obsolete? What roles can online and interactive technologies play? 

    In 2016, professionals from around the world who conduct, use, support, and study evaluation will come together in Atlanta. As a global community, we will explore how design and evaluation can be integrated to strengthen programs, benefit the environment, and improve the lives of people. I hope you will be a part of it. 


    Participants will have the opportunity to learn about exemplary evaluations, enabling evaluation environments, evaluation policies, evaluation theories and methods, and research on evaluation from a range of cultures and regions throughout the world. In 2015, we encourage participants to reflect on our practice, and to present examples of success and challenges to contribute to the goal of highlighting exemplary evaluations conducted in countries at all points along the development continuum. And, we look to explore what facilitates exemplary evaluations—what are the enabling factors, actions, and drivers of success? 

    The Exemplary Evaluations theme is intended to inspire and energize evaluation professionals from around the world to spotlight what has gone well in evaluation. As we listen to each other’s exceptional experiences, we listen for themes of success—how were these evaluation exemplars managed? How did evaluators engage with stakeholders? How evaluation conclusions were developed, what lent them credibility, and what led to evaluation use? There is much to learn from examples of high quality, ethically defensible, culturally responsive evaluation practices that have clearly contributed to decision-making processes, program improvement, policy formulation, effective and humane organizations, and ideally to the enhancement of the public good.