AEA Newsletter: July 2016

Message from the President - What Are We to Make of the World?

From John Gargani, AEA President

RS_gargani_photo_large-13RT.jpgIn light of the tragic news from Louisiana, Minnesota, Dallas, Turkey, and elsewhere, I threw out the column I wrote. I find myself shocked by a world that has made me numb to shocking news. I ask, “What am I to make of it all?” I find no satisfactory answer.

In the midst of confusion and despair, it is our professional community that gives me hope. As a community, not only can we make sense of the world, we can remake the world. We can do it in ways that are collaborative, culturally responsive, and inclusive of multiple perspectives; we can do so with legitimacy and humility.

As I like to say: Evaluation matters. It has never mattered more. Let’s hold that thought firmly in our mind when we come together in October.

 

Diversity - The Latinx Responsive Evaluation Discourse

From Lisa Aponte-Soto, National Deputy Director of RWJF New Connections at Equal Measure and Chair of LA RED TIG

Aponte-Soto.jpg

The Latinx Responsive Evaluation Discourse (LA RED) is an AEA TIG that was established in 2014. The mission of LA RED is to increase representation, engagement, and leadership of Latinx and other evaluators in the theory, research, and practice of culturally responsive evaluation, especially involving the Latinx community through the development of sustainable avenues and venues for the development and application of Latinx responsive evaluation theory and practice.

LA RED has organized an introductory four-session AEA Coffee Break series to build collaborative cross-cultural discourse to support Latinx responsive evaluation. Our introductory series highlights work of AEA members and TIG leaders who are from a Latin background and/or have experience conducting evaluation with Latinx communities.

The series opened on July 19 with Dr. Arthur E. Hernandez, The Rise of Latinx presence, perceptions and contributions to notions of CRE and AEA, Professor and Dean at Texas A&M University Corpus Christi, and Director of the AEA Minority Serving Institution (MSI) initiative. During his session, Dr. Hernandez noted that the investment in programs like MSI has resulted in “a significant growth in the number, the sophistication, and the participation of folks from underrepresented backgrounds in AEA and in the profession.” He also poignantly defined the meaning of Latinx, “While Latino and Latina participants, Latinx participants were not specifically targeted for by the AEA initiative, the largest impact on the program so far has been in the influx and participation of folks from that particular background.”

This session was followed by “A Recipe for Building Evaluation Capacity of Community-Based Organizations,” a presentation held on July 26 led by Grisel Marie Robles-Schrader, Martha Hernandez-Martinez, and Dr. Josie Serrata. Ms. Robles-Schrader is a research portfolio manager for Community and Stakeholder Engagement at Northwestern University. Dr. Serrata is the director of research and evaluation at the National Latin@ Network and Ms. Hernandez-Martinez is a research associate at the National Latin@ Network. This Coffee Break shared a culturally responsive process of a toolkit development for community based organization (CBO) engagement toward increasing the evaluation capacity of Latinx-serving CBOs in the field of domestic violence. To check out the recording of this webinar, click here.

In addition, upcoming sessions will build on prior sessions and expand on perspectives and approaches to working with Latinx programs, participants, and community stakeholder for conducting culturally responsive and community engaged evaluations. Tune in for the next sessions: on August 2 “Trabajando con Latinxs: Good Practices for Evaluators Working with Latino Communities in the U.S. by Efrain Gutierrez, associate director at FSG and on August 11 by Dr. Lisa Aponte-Soto, national deputy director of RWJF New Connections at Equal Measure and chair of LA RED TIG.

To learn more about the LA RED TIG and their exciting ongoing activities, contact their leadership below.

Lisa Aponte-Soto: laponte-soto@equalmeasure.org

Saul I. Maldonado: smaldonado@sdsu.edu

 

Potent Presentations Initiative - Time to Begin Preparing That Presentation 

From Sheila B. Robinson, Potent Presentations Initiative Coordinator 

Sheila Robinson-RS 2.png

“Begin at the beginning," the King said, very gravely, "and go on till you come to the end: then stop.”

 -Lewis Carroll, Alice in Wonderland

Hello, P2i-ers! Did you receive an acceptance notice for Evaluation 2016 in Atlanta, Georgia, or for any other conference or meeting? If so, congratulations! It's time to start crafting your potent presentation!

Where do you begin? While designing your presentation is rarely a linear process, it makes sense to start by crafting your message before delving too deeply into Design or Delivery.

Of course, our p2i message tools are here to help. In our Messaging Demo slides, you’ll find this six word story about how to structure your key presentation message: Memorable messages are distilled and structured. Use the advice from these slides along with our Messaging Model handout to identify and assemble the key components of your message — Background, Bottom Line, Explanation, So What, and Call to Action — and allocate the appropriate percentage of your presentation time to each. Then, tell the story of a program creating emotional connections.

Think you can’t possibly do all of this in a 15-minute presentation? Guess again! These resource materials were created for exactly for this type and length of presentation and, of course, can be applied to presentations of any length.

In a recent blog post, Presentation Lessons from Steve Pinker's Advice on Writing, presentation guru Garr Reynolds shares lessons learned from Steven Pinker’s The Sense of Style: The Thinking Person's Guide to Writing in the 21st Century. Reynolds describes how much of Pinker’s sage advice for writing also applies to presentations. One of Pinker’s tips we can apply to messaging is to embrace simplicity (not simplism), and Reynolds explains it this way:Cutting the superfluous is one of the hardest things for writers to do. But clear communicators make careful considerations during the preparation stage about what is important to include and what is not. Good writers and effective presenters know that if everything is important then nothing is important.” 

I highly recommend reading Reynolds’s article in its entirety, as it contains a wealth of inspiration for all stages of presentation design.

Good luck getting started on your presentations! And don’t forget:

"It's always best that you start at the beginning…”

– Glinda, the Good Witch, from The Wizard of Oz

 

Policy Watch –  Foreign Aid Transparency and Accountability Act of 2016

From Cheryl Oros, Consultant to the Evaluation Policy Task Force (EPTF)

Cheryl Oros.jpgThe Foreign Aid Transparency and Accountability Act of 2016 was signed by the president on July 15, 2016. It requires the President to establish guidelines for the establishment of measurable goals, performance metrics, and evaluation plans for U.S. foreign development and economic assistance. The bill had bipartisan support in both houses, garnering unanimous approval in the Senate.

This bill provides a good example of Evaluation Policy Task Force (EPTF) efforts to support adoption of enlightened policies on federal evaluation.  The EPTF monitored congressional efforts to draft and move this bill since December 2012, providing staff the AEA Roadmap, discussing options with congressional staff, reviewing and commenting on drafts, meeting with staff, and drafting support letters for the AEA president to send to committee chairs who were primary authors and sponsors.

This legislation is unusual and heartening in its scope across many foreign aid programs, rather than only addressing evaluation program by program.  Covered foreign assistance programs include: 

  1. Assistance authorized for many programs under Part I of the Foreign Assistance Act of 1961;
  2. Economic Support Fund;
  3. Millennium Challenge Act of 2003; and
  4. Food for Peace Act.

In summary, the Act requires that guidelines must be developed by the president in the next 18 months and provide direction on how to:

  1. Establish annual evaluation objectives and timetables to plan and manage the process of evaluating, analyzing progress, and applying learning toward achieving results;
  2. Develop specific project evaluation plans, including measurable goals and metrics, and to identify the resources necessary to conduct such evaluations, which should be covered by program costs; 
  3. Apply rigorous evaluation methodologies to such programs, including impact evaluations, ex-post evaluations, or other methods, as appropriate, that clearly define program logic, inputs, outputs, intermediate outcomes, and end outcomes;
  4. Disseminate guidelines for the development and implementation of evaluation of programs to all personnel, especially in the field;
  5. Establish methodologies for the collection of data;
  6. Evaluate, at least once in their lifetime, all programs whose dollar value equals or exceeds the median program size for the relevant office or bureau to ensure the majority of program resources are evaluated; 
  7. Conduct impact evaluations on all pilot programs before replicating, or conduct performance evaluations and provide a justification for not conducting an impact evaluation when such an evaluation is deemed inappropriate or impracticable; 
  8. Develop a clearinghouse capacity for the collection, dissemination, and preservation of knowledge and lessons learned to guide future  programs;
  9. Distribute evaluation reports internally; 
  10. Publicly report each evaluation, including a description of the evaluation methodology; key findings; appropriate context, including quantitative and qualitative data when available; and recommendations made; 
  11. Undertake collaborative partnerships and coordinate efforts with the academic community, implementing partners, and national and  international institutions, as appropriate, that have expertise in program monitoring, evaluation, and analysis when such partnerships provide needed expertise or significantly improve  the evaluation and analysis;
  12. Ensure verifiable, reliable, and timely data are available to evaluation personnel to permit the objective evaluation of programs, including an assessment of assumptions and limitations in such evaluations; and
  13. Ensure that standards of professional evaluation organizations for evaluation efforts are employed, including ensuring the integrity and independence of evaluations, permitting and encouraging the exercise of professional judgment, and providing for quality control and assurance in the monitoring and evaluation process. 

Agencies must apply the findings and conclusions of such evaluations to proposed project and program design. This should help to make formulating policy more open and consultative

Not later than 18 months after this Act, the president must submit a report to congressional committees on the guidelines.  That is likely to come from the next administration.   Next, the comptroller general must submit to congressional committees not later than 18 months after the president’s report, which (1) analyzes the guidelines; and (2) assesses the implementation of the guidelines by agencies.

If you know of other legislative opportunities to advance the use of evaluation in the federal government, please let us know at EvaluationPolicy@eval.org.

 

International Policy Update - 13 Different Ways AEA Will Support the Global Evaluation Agenda

From Mike Hendricks, AEA Representative to the International Organization for Cooperation in Evaluation (IOCE), with contributions from Jim Rugh, EvalPartners Co-Coordinator

Mike-Hendricks.jpgJim Rugh 2010.01.16.JPG

 

 

 

 

 

 

In an exciting and important development, the AEA Board of Directors has accepted the recommendations of its International Working Group and soon the AEA management will initiate 13 separate actions to support the brand-new (and first-ever) Global Evaluation Agenda 2016–2020, also known as EvalAgenda2020.  To our knowledge, this forward-thinking decision makes AEA the first Voluntary Organization for Professional Evaluation (VOPE) in the world to develop such an action plan.

In addition, the table below shows a second admirable aspect to this effort. As you can see, AEA will work to support all three dimensions of EvalAgenda2020: (1) the enabling environment for evaluation (including especially more demand for evaluation), (2) institutional capacities to commission and utilize evaluations, and (3) individual capacities to conduct evaluations.

Furthermore, AEA will work not just within the AEA membership or just within the United States, but will also to try to create knowledge sharing opportunities with other VOPEs and other countries. We can all be very proud of the way AEA has decided to be a resource and create opportunities for relationships within the evaluation community around the world.

 

Capture.PNG

Now, of course, the real work begins. Envisioning these activities is one thing, but making them a reality is quite different. If you would like to become involved in any of these 13 different ways, you are most welcome. To submit your name into the pool of volunteers, please email tmadan@eval.org and tell us which effort(s) interest you.

Also, if you are involved in a Topical Interest Group (TIG) or a local affiliate, discuss with your colleagues where their interests overlap with these 13 initiatives. A quick review of the list of TIGs and local affiliates suggests that there are lots of possibilities here.

Finally, come to the special session at our October conference to hear President-elect Kathy Newcomer and Executive Director Denise Roosendaal report on AEA’s most up-to-date plans. This will be a perfect opportunity to meet personally with AEA’s leaders and other colleagues who also want to get involved. Click here to download the 13 Initiatives. Click here to download the entire AEA International Strategy Document.

These are exciting times for evaluation globally. Kudos to the AEA Board of Directors for helping to shape the future of our field.

Recent Stories
AEA Newsletter: February 2017

AEA Newsletter: January 2017

AEA Newsletter: December 2016