|
CULTURAL READING
An
Introduction to the Cultural Reading
of The Program Evaluation Standards (2nd edition)
Welcome
to the American Evaluation Association (AEA) site of the Final Report of
A Cultural Reading of The Program Evaluation Standards (2nd
edition). This is a report of the Diversity Committee of the
American Evaluation Association that was approved by the AEA Board,
November 2004. This site contains the primary source materials,
intermediate working documents and final reports of the Diversity
Committee’s Cultural Reading Task Force as well as an opening
orientation to our work and its many participants.
Overview
The Cultural Reading began as text conversations among a
group of professional evaluators who share expertise in and concern for
issues of cultural diversity and cultural context in evaluation. The
purpose of the cultural reading was to review The Program
Evaluation Standards (2nd edition) with respect to
coverage of cultural diversity, treatment of cultural concerns, and
attention to cultural competence. This conversation was intended to
build knowledge among Diversity Committee members and other relevant
stakeholders within the AEA; to identify specific passages that are in
need of revision; and to gather ideas on relevant resources and
suggested changes for input to The Joint Committee on Standards for
Educational Evaluation. Throughout this reflection, culture is
broadly defined, inclusive of race, ethnicity, gender, age, sexual
orientation, social class, disability, language, and educational level
or disciplinary background. It includes both individual
characteristics and those of a group or collective (e.g., community or
organizational culture).
Background
At a meeting of the Diversity Committee convened by Chair Edith Thomas
at the 2002 Annual Meeting of the AEA, discussion turned to agenda items
for 2003. It was noted that The Program Evaluation Standards (2nd
edition) would be up for consideration by the Joint Committee in fall
2003, and Karen Kirkhart expressed the strong opinion that these
standards needed to be revised to update and improve their attention to
dimensions of cultural diversity. This stimulated interest among members
present. Kirkhart proposed that the Diversity Committee do a group
reading of the Standards and record their critical reflections,
exchange ideas, etc. so that the Committee would be in a more informed
position to offer comment, reflect interpretations from different
perspectives, and impact the process of Standards revision
through established channels, via the AEA representative to the Joint
Committee, Dianna Newman. The cultural reading became a 2003 action item
for the Diversity Committee, chaired by Satish Verma. Karen Kirkhart
volunteered to chair the work group. The Board approved this action item
at its February 2003 as an element of the Diversity Committee’s 2003
Action Plan.
Participants
Initial readers were members of the Diversity Committee in 2002 (when
the idea was initiated) and 2003 (when Draft 1 was completed). Charles
Thomas joined the reading group in Spring 2003 to provide continuity
with the Building Diversity Initiative’s Task Force on Guiding
Principles for Evaluators Working Across Cultures, which he chaired. In
alphabetical order, the initial readers were:
Claude
F. Bennett, U.S. Department of Agriculture
Denice
A. Cassaro, Cornell University
Melvin
E. Hall, Northern Arizona University
Stafford
Hood, Arizona State University
Lester
Horvath, Evaluation Consultant, Connecticut
Elmima
C. Johnson, National Science Foundation
Karen
E. Kirkhart, Syracuse University
Donna
M. Mertens, Gallaudet University
Sharon
Rallis, University of Connecticut
Edith
P. Thomas, U.S. Department of Agriculture
Charles
L. Thomas, George Mason University
Satish
Verma, Louisiana State University
Elizabeth
Whitmore, Carleton University
The
work of the 2003 Task Force was supported by:
Crystal
Collette, Syracuse University
David
Schlesselman, Syracuse University
In
2004, the Diversity Committee, chaired by Melvin Hall, appointed a Task
Force to synthesize the text generated by the initial readers and to
prepare comments in formats suitable for dissemination to the Joint
Committee and other relevant audiences. The members of the 2004 Task
Force were (in alphabetical order):
Denice
A. Cassaro, Cornell University
Cindy
A. Crusto, Yale University
Melvin
E. Hall, Northern Arizona University
Elmima
C. Johnson, National Science Foundation
Karen
E. Kirkhart, Syracuse University
Joan
LaFrance, Mekinak Consulting
Donna
M. Mertens, Gallaudet University
Craig
W. Russon, Kellogg Foundation
Procedure
The first round of the Cultural Reading began in January 2003 and
concluded in August 2003, yielding roughly 100 single-spaced pages of
text material, summarizing and critiquing the Standards. Members
read the standards together, posting their comments to the entire group
via email, then conversing and exchanging ideas around the postings.
Readers created their own “archives” by saving the emails to a
folder. Each standard was read in order of presentation in the book. Not
everyone logged in comments on every standard, and some readers were
more active than others in the process. Karen’s Spring 2003 Graduate
Assistant, David Schlesselman pulled together the comments at the end of
each of the four sections—Utility, Feasibility, Accuracy and
Propriety—as well as an integrated document at the end. This became
Draft 1. In Fall 2003, Crystal Collette, then a Graduate Assistant at
Syracuse University, searched the literature and compiled a bibliography
of culturally relevant references by standard. This bibliography was
provided to the Joint Committee as a freestanding document in 2004 and
was subsequently incorporated in the second draft of the Cultural
Reading.
In 2004, moving the cultural
reading from a private to a public document remained an action item of
the Diversity Committee. The second round of the Cultural Reading began
in January 2004 and concluded in August 2004, yielding a set of
prioritized action items and summary matrixes organized by individual
standard as well as by category (Utility, Feasibility, Propriety,
Accuracy), a narrative summary, and an edited version of the full text.
The content and format of these products are summarized below.
Cultural
Reading Documents Click on the links below to view the documents
in PDF format.
To
download a free PDF reader go to: http://www.adobe.com/products/acrobat/readstep2.html
- Executive
Summary:
Narrative summary, including overview of recommended corrective
actions.
- Priority
Recommendations
Important corrective actions summarized for all standards, matrix
format.
- Standards
Overview by Categories
Summarizes the relevance to cultural competence, current strengths,
concerns or limitations, and recommendations for standards by major
category—Utility, Feasibility, Propriety and Accuracy, matrix
format.
- Summary
of Utility Standards:
Summarizes the relevance to cultural competence, current strengths,
concerns or limitations, and recommendations for each of the Utility
standards, matrix format.
- Summary
of Feasibility Standards:
Summarizes the relevance to cultural competence, current strengths,
concerns or limitations, and recommendations for each of the
Feasibility standards, matrix format.
- Summary
of Propriety Standards:
Summarizes the relevance to cultural competence, current strengths,
concerns or limitations, and recommendations for each of the
Propriety standards, matrix format.
- Summary
of Accuracy Standards:
Summarizes the relevance to cultural competence, current strengths,
concerns or limitations, and recommendations for each of the
Accuracy standards, matrix format.
- A
Cultural Reading of the Program Evaluation Standards
(2nd ed.): Narrative discussion among
readers of each of the thirty standards, inclusive of Overview,
Guidelines and Common Errors, Case Illustrations and Analyses, and
suggestions for Supporting Documentation. (This is a large document which
could take a while to download depending on your connection.)
Current
Status These documents were approved by the Diversity Committee on
September 28, 2004 and sent to the AEA Board of Directors for review and
approval. The Board made minor revisions to the Executive Summary to
improve clarity and approved the revised documents as the Final Report
of the Cultural Reading Task Force on November 3, 2004. These documents
are available to support the work of AEA members, other AEA Committees,
and the Task Force of the Joint Committee on Standards for Educational
Evaluation charged with revising the second edition of The Program
Evaluation Standards (Donald Yarbrough, Chair). The documents may
also be useful to other persons or organizations that have an interest
in educational evaluation. Elmima Johnson, a participant in both phases
of the cultural reading, assumed the position of AEA representative to
the Joint Committee in January 2005. The Diversity Committee will seek
continued opportunity for input as the Joint Committee Task Force moves
through the revision process.
We
welcome dialogue and discussion. Comments may be addressed to:
|
