Date: Tuesday, June 16, 2026
Andrew Feig, Senior Program Director, RCSA
I’m Andrew Feig – Senior Program Director at Research Corporation for Science Advancement (RCSA). At RCSA, we have two major programs, Cottrell Scholars and Scialog (as well as a nascent RCSA Fellows Program), but for this discussion, I will focus on Scialog. Scialog is a contraction of Science + Dialog. The program brings together groups of scientists (Scialog Fellows) who do not yet know each other but should. After a structured, data-driven process to introduce Fellows to one another and 1.5 days of facilitated discussions, participants spend the last half-day forming teams and writing proposals. Themes typically run for three years, and Fellows return year-on-year for the duration of the initiative, with additional rounds of grant-making each year. RCSA’s goal is to fertilize new areas of science, seed new interdisciplinary collaborations, and help early-career faculty become leaders in the thematic area, where their bold ideas change the direction of scientific thinking.
At one level, our Board wants big outcome assessments. What were the ground-breaking scientific advances that derived from these grants? How did these grants change the field? Unfortunately, since we fund early-stage ideas, it often takes 5, 10, or even 15 years for the most successful projects to evolve and yield that level of impact. These stories do not provide actionable insights to guide the program and thus have only modest value in my opinion. Collect them, but don’t spend too much time or resources here.
Far more important are the early hallmarks of success that we can use as proxies for what may eventually come from these teams. Do the collaborations persist? What occurred during Scialog that helped the teams gel? Was there sufficient early success that the team sought or acquired additional funding based on data our funding helped them obtain? We collect early hallmark data from structured grant reports, and supplement those reports with annual analyses of every past Scialog – back to its inception more than 15 years ago. Using APIs, we pull publication histories of every participant (more than 2000 to date). We tag papers co-authored by 2 or more Fellows who met at Scialog, noting whether or not they received funding or wrote a proposal together. We use these papers to track the long-term network effects of the community we seeded. It turns out that many of these papers come from teams that never received a grant and thus would never have submitted a progress report. They liked the idea they proposed and chose to pursue it with their colleagues anyway.
Even more important, however, are the questions about the activities during Scialog. What activities effectively build trust between Fellows? What motivates two individuals to choose to collaborate just 24 hours after having met? Through a project with Prof. Danny Abrams (Applied Mathematics @ Northwestern), we have measured the interpersonal interactions during our meetings, the nature and role of facilitation in discussion groups, and the dynamics of group conversations that break homophily and drive creative thinking. These studies have led to changes in our conference schedule, modifications of our facilitator training, and optimization of the event in transformative ways. This is the type of assessment I lean towards: INTENTIONAL – based on data collections we build into the planning and execution of our meeting; it is part of what we do not an add-on; ACTIONABLE – allowing us to use it to refine our convening formats; and IMPACTFUL – providing a feedback loop that lets us tell the story of how we use assessment in the day-to-day work of our foundation and pursue continuous program improvement.
The American Evaluation Association is celebrating RTD TIG Week with our colleagues in the Research Technology and Development TIG. All of the blog contributions this week come from our RTD TIG members. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.