Date: Friday, May 29, 2026
We are Melanie DiLoreto and John Pecore, faculty members in the School of Education at the University of West Florida. We are writing to highlight a language disconnect between academic faculty and field-based practitioners that we recently observed while serving as external evaluators for a grant awarded to a science center and museum that promotes hands-on learning in math, engineering, and science.
When working with informal learning environments, it’s important to recognize that you may not be speaking the same language. For academics with training or backgrounds in measurement and evaluation, this disconnect can be especially noticeable. Many organizations write and submit grants, and sometimes even receive funding, before identifying who will evaluate the project. In many cases, grant teams have ambitious goals and outcomes but have not identified how these will be measured. This can leave evaluators trying to connect broad ideas, such as environmental literacy, youth empowerment, or standards alignment, to practical and meaningful sources of evidence.
Imagine agreeing to serve as the evaluator for a grant that outlines a long list of outcomes, many of which are difficult to measure, or for which no assessment tools have been identified. Faced with this situation, we returned to our roots as educators. Through guided informal conversations, we highlighted the importance of using existing instruments with established evidence of reliability and validity. We also discussed the near-impossible task of creating new tools from scratch, given the lengthy process required to establish that evidence.
Our project team wanted to measure a wide range of outcomes, including environmental literacy, critical thinking, systems thinking, design thinking, and youth empowerment, which created a significant challenge. We measured some goals through pre- and post-assessments, while other goals required teacher surveys, interviews, observations, or standards-alignment rubrics.
For example, one project goal focused on increasing accessibility for grades 3-5 students. While that sounds simple, the team had to define what “accessibility” meant in practice. Did it refer to number of students served, number of schools participating, or whether students from underserved schools were represented? Another outcome focused on alignment with state standards and Next Generation Science Standards, which required more than a survey item; it required a rubric-based review of project activities and lesson materials.
Through multiple conversations with informal educators, we identified and gained permission to use appropriate instruments to measure the various outcomes outlined in the grant. The project team initially provided sample items they had created which served as a starting point for discussion to identify viable instruments. We helped the project team connect each intended outcome with a realistic and meaningful source of evidence. We examined student learning through pre- and post-assessments, while teacher surveys and interviews captured teacher perceptions more effectively. More complex outcomes required a combination of methods. By aligning project goals with validated instruments and multiple sources of evidence, we ensured that the evaluation would be both meaningful and methodologically sound.
Ultimately, successful evaluation in informal learning environments requires more than finding the “right” instrument; it requires helping grant teams connect ambitious goals with evidence that is both meaningful and feasible to collect. Center for Advancement of Informal Science Education (CAISE) maintains InformalScinece.org, which includes a repository of thousands of project resources, including evaluation reports and materials developed in museums and science centers. National Informal STEM Education Network also contains valuable information and resources for evaluation. Drawing on these resources can help project teams move from broad ideas to practical, measurable evidence.
The American Evaluation Association is hosting STEM Education and Training TIG Week with our colleagues in the STEM Education and Training Topical Interest Group. The contributions all this week to AEA365 come from our STEM Education and Training TIG members. Do you have questions, concerns, kudos, or content to extend this AEA365 contribution? Please add them in the comments section for this post on the AEA365 webpage so that we may enrich our community of practice. Would you like to submit an AEA365 Tip? Please send a note of interest to AEA365@eval.org. AEA365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators. The views and opinions expressed on the AEA365 blog are solely those of the original authors and other contributors. These views and opinions do not necessarily represent those of the American Evaluation Association, and/or any/all contributors to this site.