AEA EVALUATION
POLICY TASK FORCE CHARGE
JUNE 2007 (Scroll down for Updates as of July 2009)
Background
At its Winter 2007 meeting, the Board of Directors of
the American Evaluation Association (AEA) discussed its interest in
the Association enhancing its ability to identify and influence
evaluation policies that have a broad effect on evaluation practice.
To that end the Board established an Evaluation Policy Task Force
that can advise AEA on how best to proceed in this arena.
In advance of the Task Force, an Advisory Group was
convened over the Spring of 2007 to address the following tasks:
-
Discuss the boundaries and scope of the Task Force
-
Develop a draft charge for the Task Force
-
Identify and recommend potential Task Force members
The results of the Advisory Group’s discussions
informed the current Charge to the Evaluation Policy Task Force.
Scope of Work
The Evaluation Policy Task Force is appointed to
assist AEA in developing an ongoing capability to influence
evaluation policies that are critically important to the practice of
evaluation. The following outlines the intended scope of this work.
-
Focus on Evaluation Policies. The
Evaluation Policy Task Force should focus on evaluation
policy, rather than policy in general. While evaluation can help
inform substantive policies in a wide range of areas, and this
is a recognized central purpose for much evaluation, influencing
substantive policy is not the emphasis of this Task Force
effort. Instead, the Task Force should concentrate on evaluation policies. Examples of general areas of evaluation
policy might include (although may not be limited to) policies
regarding:
-
Evaluation definition.
How, if at all, is evaluation defined in an agency or in
legislation? In such contexts, how is evaluation formally
distinguished from or related to other functions such as
program planning, monitoring, performance measurement or
implementation?
-
Requirements of evaluation.
When are evaluations required? What programs or entities are
required to have evaluations? How often are evaluations
scheduled? What procedures are used to determine when or
whether evaluation takes place?
-
Evaluation methods.
What approaches or methods of evaluation are recommended or
required by legislation or regulation, for what types of
programs or initiatives?
-
Human resources regarding evaluation.
What requirements exist for people who conduct evaluations?
What types of training, experience or background are
required?
-
Evaluation budgets.
What are the standards for budgeting for evaluation work?
-
Evaluation implementation.
What types of evaluation implementation issues are guided by
policies? For instance, when are internal versus external
evaluations required and how are these defined?
-
Evaluation ethics.
What are the policies for addressing ethical issues in
evaluation?
-
Focus on U.S. Federal Evaluation Policy.
At least initially, it would be desirable to focus this Task
Force on the Federal level because evaluation policy decisions
at that level have broad implications for many AEA members and
for the field generally. It may be possible in the future to
expand this effort to other arenas (nonprofits,
state-governments, businesses, international, etc.), but we
expect that initially at least the effort will be focused on
Federal policy.
-
Target Only a Few Specific Legislative and Executive Policies.
It will be necessary to limit the effort to one or two
potentially important evaluation policy formulation
opportunities on both the congressional and administrative side
of the Federal government. For instance, we might identify one
or two major pieces of legislation and one or two areas where
administrative regulations are likely over the next few years.
-
Focus on Selected Substantive Areas.
The workgroup anticipates that it will be necessary to focus the
Task Force’s work on evaluation policy within a limited set of
substantive areas. For instance, the fields of education and
health evaluation are areas of primary interest to a broad range
of AEA members where AEA could build on its current strengths.
The Task Force should focus its evaluation policy shaping
activities in substantive areas such as these that are related
to AEA’s history and current emphases.
-
Be
Appropriately Opportunistic. Given
the preceding thoughts about limiting the Task Force’s work, we
recognize the value of remaining flexible and of responding
(thoughtfully and selectively) as unexpected opportunities arise
that might fall outside the stated initial scope. The Task Force
should look to take advantage of such opportunities and should
consult with the AEA Board as such arise.
Tasks and Responsibilities
The Task Force will advise and assist the AEA Board,
Executive Committee and AEA staff in addressing two primary parallel
tasks. The first task will consist of a focused, targeted
consultative campaign designed to identify and provide useful
consultation to U.S. federal legislators and executive branch staff
on legislation and regulation in a select number of areas. The
second and parallel task will be to develop the public presence
that AEA projects in direct support of the consultative campaign, in
particular through the development of collateral materials that can
be utilized in that campaign, and perhaps through other forms of
outreach as well (e.g., website presence).
The primary role of the Task Force will be to provide
overall guidance on these two tasks. The Task Force is not
responsible for implementation of the tasks, although some Task
Force members may be directly involved in the implementation. To
accomplish the work envisioned here, AEA intends to engage the
services of a consultant who will play a major role in
implementation of the consultative campaign. The Task Force is
intended to advise and support this consultant, and to make policy
recommendations to the AEA Executive Committee and Board on both of
the tasks as appropriate.
Consultative Campaign Task.
The primary purpose of this task is to influence federal evaluation
policy in a manner consistent with the Mission of AEA in select
areas identified by this Task Force. This task is intended to be a
relatively “quiet” effort designed to develop experience and
expertise on how best to influence federal evaluation policy
directly.
Specific sub-tasks on which the Task Force will
provide input and guidance include:
-
Identifying and summarizing attempts to influence
federal policy by other professional organizations that are
similar in nature and scope to this effort.
-
Identify and coordinate with evaluators within
the Federal Government who are connected to the specific
evaluation policy context.
-
Advising the AEA Board on legal and ethical
issues and concerns associated with efforts to influence
evaluation policy.
-
Developing processes for engaging and
communicating with the AEA membership on this evaluation policy
effort, including communicating with membership and inviting
their input at sessions at the annual conference of the American
Evaluation Association.
-
Formulating and making recommendations to the AEA
Board as appropriate.
-
Consulting on evaluation policy planning
processes, including methods for: scanning and identifying
prospective policy opportunities; managing networks of contacts;
entering into and managing specific policy influence efforts;
responding to requests for input on policy formulation;
monitoring policy efforts; and managing relationships with any
consultants engaged by AEA to accomplish this task.
-
Providing guidance and advice on efforts to scan
federal legislative and regulatory opportunities for relevant
policies that could be targets for influence.
-
Advising on the selection of specific policy
opportunities.
-
Advising on how best to manage the emerging
network of contacts in both the legislative and executive branch
that can assist AEA in influencing evaluation policy.
-
Monitoring and providing input on the evaluation
policy efforts and progress made.
-
Reporting to the Executive Committee and Board on
progress of the consultative campaign.
-
Preparing and making recommendations to AEA Board
on how the evaluation policy influence effort should be
implemented subsequent to the two-year appointment period of the
Task Force.
Because of the potentially broad nature of this work,
the consultative campaign will initially be limited to only a few
policy targets. The consultant, AEA Staff, and any Task Force
members directly involved in implementation will be primarily
responsible for identifying an initial set of potential policy
targets and will bring these to the Task Force for consideration.
The Task Force will be expected to play a major role in discussing
and advising on what policies should be targeted in this effort. On
the congressional/legislative side, this might involve: identifying
one or two major areas of upcoming legislation (e.g., one each in
education and health); identifying congressional staff members that
are critical to the legislation; developing contacts and building a
network of personal relationships with staff members; conducting
background research on evaluation policies that currently exist in
the area or other relevant aligned areas; identifying potential
policies that would enhance evaluation; making contact with
identified staff; and, responding as needed to any requests for
input on policy formulation (e.g., testifying at congressional
hearings; drafting potential language for legislation). On the
executive side, this effort might involve: identifying one or two
major areas where evaluation policy is likely to be formulated
within the next few years; determining who the major agencies and
people are who will formulate the policies; determining how direct
personal contact will be made with them; advising on the development
of a network of personal relationships; conducting background
research on existing relevant evaluation policies; and, consulting
on potential policy changes or regulatory language. The Task Force
will provide general guidance and advice on these activities;
implementation will be handled by the consultant, AEA staff, and
select Task Force members appointed to engage in these activities.
Public Presence Task.
The primary purpose of this task is to support the consultative
campaign through efforts that help position AEA as the leading U.S.
association in evaluation and through the development of collateral
materials that represent AEA and can be utilized in discussions with
policymakers. Again, while the Task Force will be expected to advise
and provide guidance on this effort, and may elect to draft
materials that support it, the primary implementation will be the
responsibility of the consultant, AEA staff, and Task Force members
who are directly involved in implementation. The Task Force will be
expected to provide advice and guidance on the following:
-
Development of general “talking points” on
evaluation and its role that would be likely to have broad
consensus from AEA members and could be used in consultative
work on evaluation policies (possible examples of talking points
might be, for example, that: evaluation is a critically
important endeavor; legislation and regulation should have
explicit written requirements for evaluation of federally-funded
programs; a broad range of evaluation methods are likely to be
applicable and legislation and regulation should require that
multiple methods and approaches appropriate to the program being
assessed should be considered; etc.)
-
Development and maintenance of a roster of
leading evaluators who can be called upon as needed as
spokespersons or contact persons for the media, for meetings
with legislators or executives in government, for testifying in
formal committees or meetings, and/or as consultants on
evaluation policy issues.
-
Development and enhancement of AEA collateral
materials (e.g., brochures, Guiding Principles, public
statements) that will support the consultative campaign
-
Enhancement of the AEA website so that it is more
useful for influencing evaluation policy formulation and
positioning AEA as a leader in advising on evaluation policy.
This should include working with the consultant to develop a
“Policy Watch” function for notifying and engaging AEA members
regarding important impending evaluation policy changes and
initiatives.
The Task Force is not expected to address the AEA’s
public presence generally. Their efforts should be concentrated on
guidance that will specifically support the consultative campaign.
In this work, the Task Force will communicate and collaborate with
other AEA Committees such as the Public Affairs or Professional
Development committees on areas of common relevance.
Appointment, Time Commitment, Structure and Support
The Evaluation Policy Task Force is appointed for a
two-year period beginning in July, 2007. The Evaluation Policy Task
Force is a Task Force of the AEA Executive Committee and, as such,
will report through the EC and may use the EC as a sounding Board as
desired.
The Task Force will consist of 5 members, the
Executive Director, and a liaison from the Public Affairs Committee
(either the Chair or a PAC member). The Chair of the Task Force will
be a member of the Executive Committee. The Evaluation Policy
Consultant will be integrally involved in the Task Force and will
participate in meetings and discussions as appropriate. The Task
Force Chair and AEA Executive Director will identify potential
members, subject to approval of the Executive Committee, with the
following criteria to guide selection of members:
-
Commitment to supporting the mission and goals of the American
Evaluation Association.
-
Knowledge of and a history of prior involvement with the
American Evaluation Association.
-
Familiarity with the field of evaluation and capacity to
understand and represent the field to others.
-
A
broad perspective on evaluation and willingness to advocate for
the many and diverse views of evaluation.
-
Knowledge of the federal policymaking process.
-
Experience with policy development initiatives in the Federal
Government.
-
Experience with public presentations of evaluation to a variety
of audiences.
-
Diversity of the Task Force and representativeness of the
breadth of members and interests of AEA.
The Task Force will provide a written annual report
to the AEA Board for review at its Winter Board meeting and updates
as needed through the Task Force Chair at other Board meetings. At
the completion of the first year of their work, the Task Force will
be expected to report to the Board with their recommendations about
the feasibility and desirability of continuation of this effort and
the best mechanisms for doing so (e.g., continuation of the Task
Force; establishment of a standing committee or assignment of this
effort as a subcommittee to an existing committee). The AEA Board
will provide ongoing feedback and guidance based on its reviews of
these reports.The Task Force will be expected to convene at least
once every two months over the two year appointment period. Most of
these meetings will be by teleconference, although the Task Force
should attempt to meet face-to-face at least annually at the AEA
annual conference. The Task Force will set its own agenda and
specific activities in pursuit of the tasks outlined above. The Task
Force is encouraged to develop work groups that invite people who
are not on the Task Force to assist in work to address specific
tasks as needed.
The Task Force will most likely need to develop a
mechanism that enables rapid response to situations that arise under
the consultative campaign. For instance, they might wish to develop
a roster of senior evaluators who are willing to be called to help
respond in the event that a meeting is needed to be held
immediately, legislative or statutory language needs to be drafted
quickly or public testimony is requested.
The AEA office will provide administrative support
for the Task Force, including assistance in setting and managing
meetings and Task Force materials.
AEA will provide contracted consultative support for
technical assistance in identifying, contacting and managing
specific policy shaping opportunities.
Over the course of the two year timeframe, the
success of this initiative will be judged on: the level and
appropriateness of the activities; the extent of its influence on
specific policies and policy language; and, the quality and
potential value of the network developed that connects AEA, the
consultant, and policymakers.
MODIFICATIONS AS
OF JULY 2009
The Evaluation Policy Task Force is appointed for an
additional two-year period beginning in July,
2009.
The Task Force will consist of
up to 10 members, including
the Executive Director, a member of
the Presidential rotation, and a liaison from the Public
Affairs Committee (either the Chair or a PAC member).
The Chair of the Task Force will be
a member of the Executive Committee.
Return to the
AEA Evaluation Policy Task Force Main Page |
