Research on Evaluation (RoE) TIG Week: From Evaluation to Engagement: Designing RoE to Understand What Matters by Tatiana E. Bustos, Esther C. Nolton, and Jessica S. Saucedo

Date: Sunday, June 8, 2025

Hello! We–Tatiana Bustos, Esther Nolton, and Jessica Saucedo–are evaluators passionate about systematic approaches to engagement, driven by the need for more empirical evidence about what works and for whom.

We recently completed a scoping review of Research on Evaluation (RoE) on engagement, expanding on a prior study from 2014. With 27 RoE studies, we synthesized how engagement was conceptualized, methodological approaches used to empirically examine it, and the strategies or practices that guided engagement quality (e.g., types of partners or activities described).

Lessons Learned

1. RoE Scarcity: RoE on engagement remains limited in both scope and methods, with little progress over the past decade. While the complexities of engagement may explain this stagnation, there is potential for innovation in RoE to identify conditions that lead to successful engagement.
2. Inconsistent Language: Engagement is defined differently across RoE studies, presenting benefits and pitfalls. Developing shared, inclusive language about “engagement” that emphasizes mutual benefits can improve consistency and operational clarity. However, it is equally important to maintain flexibility to accommodate cultural contexts within projects and interest-holders. 
3. Overgeneralization of Partners: Thoughtful consideration of partner selection processes is crucial for fostering meaningful and equitable engagement. Yet, terms like “diverse partners” described in studies overgeneralize the nuanced practices of engaging specific groups, limiting understanding on what practices may work for some groups over others.
4. Necessity for Measurement: Establishing reliable methods to measure engagement outcomes can help assess the effectiveness of various strategies and guide future research. This requires measurement tools that account for factors like partner benefits, relationship quality, and the direct involvement of interest-holders–all of which should be defined in collaborations.

Our findings highlight gaps and opportunities for advancing RoE to develop a research agenda that guides theory and practice for effective engagement. We hope calling attention to these lessons will inspire others to see the importance of building scholarship to position engagement as an essential practice that holds mutual benefits for interest-holders and the evaluand. The following are ways to incorporate systematic approaches to engagement in your existing work that are low-cost and -effort but powerful in advancing research aspects to engagement.

Hot Tips

• Promote More RoE: In proposals, consider including optional tasks related to engagement that incorporate meaningful measurement approaches.
• Clarify Language: When developing evaluation plans or initiating discussions with key partners, collaboratively define engagement to establish a consensus on its intended applications. Keep in mind that definitions may evolve over time as contexts are dynamic.
• Engage Partners: With consent, clearly characterize your partners rather than simply labeling them as a broad category. Provide a summary of who they are, their vested interests, roles and contributions to the project.
• Develop Measures: During planning, identify which engagement-related outcomes are essential to capture for a more comprehensive understanding of impact.

Engagement is a fundamental evaluation practice. Yet, the empirical evidence base on engagement has remained limited for decades across all fields. RoE has the potential to bridge this gap by generating actionable insights into the conditions that foster successful engagement and positive outcomes. Without this progress, we struggle to understand what works, for whom, and under what circumstances.

We invite the evaluation community to engage with our post and join the ongoing conversation.

Rad Resources

• Evaluation engagement: Historical perspectives and new directions with community-based participatory research principles by Tatiana Bustos, Oyesola Oluwafunmilayo Ayeni, and Jessica Saucedo
• Engagement in evaluation by Esther Nolton and Laura Forsythe

The American Evaluation Association is hosting Research on Evaluation (ROE) Topical Interest Group Week. The contributions all this week to AEA365 come from our ROE TIG members. Do you have questions, concerns, kudos, or content to extend this AEA365 contribution? Please add them in the comments section for this post on the AEA365 webpage so that we may enrich our community of practice. Would you like to submit an AEA365 Tip? Please send a note of interest to AEA365@eval.org. AEA365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators. The views and opinions expressed on the AEA365 blog are solely those of the original authors and other contributors. These views and opinions do not necessarily represent those of the American Evaluation Association, and/or any/all contributors to this site.