Why Alumni Stories Matter: Evaluating the Impact of Policy, Systems, and Environmental Change in Learning Collaboratives by Rita Nahta

Date: Wednesday, September 10, 2025

Hello, AEA365 community! Liz DiLuzio here, Lead Curator of the blog. This week is Individuals Week, which means we take a break from our themed weeks and spotlight the Hot Tips, Cool Tricks, Rad Resources and Lessons Learned from any evaluator interested in sharing. Would you like to contribute to future individuals weeks? Email me at AEA365@eval.org with an idea or a draft and we will make it happen.

Hi, I’m Rita Nahta, and I oversee evaluation of cross-sectoral learning collaboratives for health professionals. These learning collaboratives are designed to promote policy, systems, and environmental (PSE) change for improved health outcomes. While the learning programs are short-term, the PSE changes and real-world impacts of the programs can take much longer to emerge. Having strategies in place to understand the long-term PSE impacts of learning collaboratives is essential. 

Lessons Learned 

From my work supporting evaluation of these collaboratives, I observed that proactive and consistent engagement with alumni participants is essential for understanding and elevating impact. This is particularly true in rapidly changing landscapes, where new needs and threats to program sustainability often emerge simultaneously. Rather than relying solely on outreach through surveys, I have seen that one-to-one conversations with alumni provide a meaningful set of data, including impact stories, case studies, and recommendations from the field. These conversations also serve to strengthen relationships between program staff and alumni, who often become program champions. 

Why Alumni Voices Matter 

Alumni stories are powerful tools. They offer insights about how useful the learning collaborative was towards realizing PSE change within a real-world context. Alumni who have gone through the steps of trying to implement what they learned are uniquely positioned to share gaps, as well as emerging needs that the curriculum should address. In addition to informing program improvement, alumni stories can be used to show return on investment to partners, including funders, prospective participants, and community members. Including alumni stories in the evaluation process allows alumni to remain invested in the learning community, often returning as mentors or advocating for the sustainability and continued funding of these programs. Alumni stories also provide new program participants with inspiration and tangible approaches for PSE change, helping translate concepts into real-world impact. 

I’ve found that having alumni stories curated and ready to go is especially important when funding becomes uncertain. When partners request data to advocate for continued funding, the ability to retrieve high-impact stories in a timely manner is crucial. 

Hot Tips 

• Build alumni engagement into your longitudinal evaluation plan. 
• Maintain an organized alumni database, such as a spreadsheet or digital platform, for internal use. 
• Ask current participants how they would like to be engaged when the program ends. 
• Include alumni in opt-in newsletters or virtual communities. 
• Proactively conduct interviews, aiming to collect a diverse range of alumni stories, and de-identify as requested. 
• Collaborate with data management and communications staff at your organization to curate alumni stories in an organized database, allowing for easy, searchable retrieval. 
• Collaborate with other backbone organizations if your learning collaborative is part of a larger network. Working together to share best practices and lessons learned can increase collective impact towards PSE change. 
• Curate alumni stories in multiple modes, based on alumni preferences, and ensure accessibility for a broad audience. 

Rad Resource 

• The Collaborative Learning Networks Measurement & Learning Framework from Collaborative Impact and Results for Development (R4D) provides planning and evaluation tools for learning collaboratives. The lists of “measurement and learning questions” and “measurement and learning activities and methods” in the toolkit can be adapted for alumni engagement. 
• Evaluating the long-term PSE change impact of learning collaboratives can be complex. Alumni stories are essential for adapting collaboratives to meet real-world needs and for communicating the true impact of these programs. 

Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org . aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators. The views and opinions expressed on the AEA365 blog are solely those of the original authors and other contributors. These views and opinions do not necessarily represent those of the American Evaluation Association, and/or any/all contributors to this site.