|
Session Title: From a Distance: Conducting Successful Long-distance Evaluations
|
|
Panel Session 401 to be held in Avalon A on Thursday, Nov 3, 2:50 PM to 4:20 PM
|
|
Sponsored by the Independent Consulting TIG
and the Cluster, Multi-site and Multi-level Evaluation TIG
|
| Chair(s): |
| Randi Nelson, Nelson Evaluation Services, rnelson.nert@gmail.com
|
| Abstract:
The panel members will discuss lessons learned and useful resources for planning and implementing long-distance evaluations, defined as evaluations in which the client and/or programs being evaluated are geographically distant from the evaluator's primary place of business. Although each panelist will base their remarks on experiences conducting specific evaluations, the discussion focuses on strategies and resources for overcoming global challenges of conducting long-distance evaluations. The discussion highlights issues related to evaluation planning, data collection, and communication with clients and stakeholders and applies to a wide variety of program contexts and evaluation approaches. The discussion will provide useful insights to evaluators, clients, and program staff who may be involved in long-distance evaluations. Panelists invite audience members to contribute to the discussion by sharing their own experiences, perspectives, and resources to advance the practice of conducting long-distance evaluations.
|
|
Planning Long-distance Evaluations
|
| Randi Nelson, Nelson Evaluation Services, rnelson.nert@gmail.com
|
|
Independent evaluators typically initiate planning by reading requests for proposals or in client discussions. Although many planning issues apply to all evaluations, those conducted at a distance introduce additional issues. This discussion addresses distance evaluations at the earliest stage of planning, when-¼-¼ determining if it makes sense to propose a distance evaluation.
The discussion is organized around a series of questions that can be used to determine the feasibility of successfully conducting a distance evaluation on time and within budget. Questions address the client's attitude toward or experience with prior local and distance evaluations, their views on collaborative approaches to evaluation-¼-¼-especially as it relates to the capacity and willingness of program staff or clients to participate in data collection, the viability of Internet-based data collection methods such as online surveys or focus groups, and client expectations concerning frequency and type of communication with the evaluator.
|
|
|
Booklets, Telephone Focus Groups, and Training Videos: Getting the Data Collected From a Distance
|
| Cheryl Kessler, Blue Scarf Consulting, cheryl@bluescarfconsulting.com
|
|
Three different clients, three separate projects, three different approaches to getting data collected from a distance. A six-week parenting workshop at a children's museum required data collection that was easily integrated into sessions to accommodate the limited time of each session and concerns with participants' literacy skills. A project with limited travel budget and eight communities that were distant from each other as well as from the evaluator required strong client-evaluator-participant communication and consideration of participants' relatively low technical abilities. A large, multi-site project required easily accessible training for staff with varying levels of experience collecting data and understanding of evaluation. Solving the specific data collection issues of each project required collaboration with clients, follow-up with participants, and vigilant oversight of the data collection to insure protocols were followed, explain why half the questions were unanswered, and know when the data promised the prior week would actually arrive.
| |
|
Helping Build Commitment and Participation Among Reluctant Long-distance Clients
|
| Denise Roseland, MetroPoint Research & Evaluation, metropointeval@gmail.com
|
|
Sometimes, the client doesn't choose you as an evaluator and may struggle with the unique challenges of the long-distance relationship. This is especially true in many multi-site evaluations, where the primary "client" is a funder but the work of the independent evaluator is with grantees or local project sites spread across a state or the country. This presentation will share a number of lessons learned (and a few promising practices from the literature) about using a variety of participatory approaches to evaluation to build client trust across the miles and to build capacity of project staff to effectively collaborate using technological tools.
| |
|
Evaluation from a Distance: Conquering Communication and Data Collection Challenges
|
| Kathleen Dowell, EvalSolutions, kathy@eval-solutions.com
|
|
Long distance evaluations - where the evaluator and the program being evaluated are separated geographically - pose unique problems for evaluators. Two of the major issues revolve around communication and data collection. This presentation will focus on how one independent evaluator worked in partnership with a client to plan and manage a long-distance, multi-site evaluation that presented multiple challenges to both the client and the evaluator. Challenges encountered included developing a mutually beneficial partnership with the client organization; maintaining adequate communication; and remotely planning, monitoring, and supporting data collection efforts. While technology exists to mitigate some of the challenges that arise during long-distance evaluations, problems still occur that require flexibility and creativity on the part of the evaluator. Strategies to overcome data collection and communication challenges will be discussed and lessons learned in successfully working on a long-distance evaluation will be presented.
| |