Date: Monday, August 17, 2026
My name is Élyse McCall-Thomas, PhD Candidate at the University of Ottawa. In this post, I reflect on how working outside traditional timelines and closely with programs can facilitate the use of evaluation findings.
In the Canadian federal government, policy dictates that evaluations of grant and contributions programs must occur every five years and be conducted in a neutral manner. While working at the Natural Sciences and Engineering Research Council, a federal agency that funds research and innovation, the evaluation division was approached by Ed Irving and Jeffrey Nerenberg, the directors of the newly implemented Alliance Grants, who wanted to understand why participation was lower than anticipated. They had theories but wanted evidence to confirm their thinking and support potential program changes. Although the formal, policy-required evaluation was still two years away, they needed answers now, and it was our responsibility to help them.
To meet the program’s needs, my director, Susan Morris, suggested a rapid program review to generate quality evidence quickly for learning and decision-making. The review would be narrowly focused and completed in six months.
Rather than producing a formal written report with recommendations, we summarised our findings and opportunities for improvement in a presentation deck. This made the information easier to digest and faster to share. The program could also repurpose the deck for senior management, and other interested parties.
My colleagues, Kelly-Lyn Christie and Sara Grondin, and I worked closely with the program to identify what they needed to know, maintaining ongoing dialogue about emerging findings and ideas for change.
We found opportunities to join consultations the program was already planning with the research community. This was a novel approach for us as government evaluations are often conducted independently to meet policy expectations around neutrality. We supplemented the consultations with independent data collection, keeping the work efficient without sacrificing rigour.
Because of their involvement in the review, the program was able to make quick use of the findings, redesigning elements of the Alliance Grants to increase uptake. They called this the Alliance Refresh. In my eighteen years as an evaluator, this was the first time I witnessed such timely and concrete use of evaluation findings to inform program decisions.
If the organization has an evaluation policy or specific expectations for how evaluations are to be conducted, it helps to know those up front and identify potential “gray zones.” Flexibility is only possible if you understand what is expected and how you can meet those expectations without sacrificing the utility of evaluation.
Do you have questions, concerns, kudos, or content to extend this AEA365 contribution? Please add them in the comments section for this post on the AEA365 webpage so that we may enrich our community of practice. Would you like to submit an AEA365 Tip? Please do so using the form on our website. AEA365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.