[Federal Register: November 4, 2003 (Volume 68, Number 213)]
[Notices]
[Page 62445-62447]
From the Federal Register Online via GPO Access [wais.access.gpo.gov]
[DOCID:fr04no03-38]
-----------------------------------------------------------------------
DEPARTMENT OF EDUCATION
RIN 1890-ZA00
Scientifically Based Evaluation Methods
AGENCY: Department of Education.
ACTION: Notice of proposed priority.
-----------------------------------------------------------------------
SUMMARY: The Secretary of Education proposes a priority that may be
used for any appropriate programs in the Department of Education
(Department) in FY 2004 and in later years. We take this action to
focus Federal financial assistance on expanding the number of programs
and projects Department wide that are evaluated under rigorous
scientifically based research methods in accordance with the Elementary
and Secondary Education Act (ESEA) as reauthorized by the No Child Left
Behind Act of 2001 (NCLB). Establishing the priority on a Department-
wide basis would permit any office to use the priority for a program
for which it is appropriate.
DATES: We must receive your comments on or before December 4, 2003.
ADDRESSES: Address all comments about this proposed priority to Margo
K. Anderson, U.S. Department of Education, 400 Maryland Avenue, SW.,
room 4W333, Washington, DC 20202-5910. If you prefer to send your
comments through the Internet, use the following address: comments@ed.gov.
You must include the term ``Evaluation'' in the subject line of
your electronic message.
FOR FURTHER INFORMATION CONTACT: Margo Anderson. Telephone: (202) 205-3010 or via Internet at Margo.Anderson@ed.gov.
If you use a telecommunications device for the deaf (TDD), you may
call the Federal Information Relay Service (FIRS) at 1-800-877-8339.
Individuals with disabilities may obtain this document in an
alternative format (e.g., Braille, large print, audiotape, or computer
diskette) on request to the contact person listed under FOR FURTHER
INFORMATION CONTACT.
SUPPLEMENTARY INFORMATION:
Invitation To Comment
We invite you to submit comments regarding this proposed priority.
We invite you to assist us in complying with the specific
requirements of Executive Order 12866 and its overall requirement of
reducing regulatory burden that might result from this proposed
priority. Please let us know of any further opportunities we should
take to reduce potential costs or
[[Page 62446]]
increase potential benefits while preserving the effective and
efficient administration of the Department's programs.
During and after the comment period, you may inspect all public
comments about this proposed priority in room 4W333, 400 Maryland
Avenue, SW., Washington, DC between the hours of 8:30 a.m. and 4 p.m.,
Eastern time, Monday through Friday of each week except Federal
holidays.
Assistance to Individuals With Disabilities in Reviewing the Rulemaking
Record
On request, we will supply an appropriate aid, such as a reader or
print magnifier, to an individual with a disability who needs
assistance to review the comments or other documents in the public
rulemaking record for this proposed priority. If you want to schedule
an appointment for this type of aid, please contact the person listed
under FOR FURTHER INFORMATION CONTACT.
General
The ESEA as reauthorized by the NCLB uses the term scientifically
based research more than 100 times in the context of evaluating
programs to determine what works in education or ensuring that Federal
funds are used to support activities and services that work. This
proposed priority is intended to ensure that Federal funds are used to
support projects and activities that are consistent with a statutory
purpose of Department programs, and evaluated using scientifically
based research. Establishing this priority makes it possible for any
office in the Department to encourage or to require appropriate
projects to use scientifically based evaluation strategies to determine
the effectiveness of a project intervention.
Discussion of Proposed Priority
We will announce the final priority in a notice in the Federal
Register. We will determine the final priority after considering public
comments on this proposed priority and other information available to
the Department. This notice does not preclude the Secretary from
proposing or funding additional priorities, subject to meeting
applicable rulemaking requirements.
Note: This notice does not solicit applications. In any year in
which we choose to use this proposed priority, we invite
applications for new awards under the applicable program through a
notice in the Federal Register. When inviting applications we
designate the priority as absolute, competitive preference, or
invitational. The effect of each type of priority follows:
Absolute priority: Under an absolute priority we consider only
applications that meet the priority (34 CFR 75.105(c)(3)).
Competitive preference priority: Under a competitive preference
priority we give competitive preference to an application by either
(1) awarding additional points, depending on how well or the extent
to which the application meets the competitive preference priority
(34 CFR 75.105(c)(2)(i)); or (2) selecting an application that meets
the competitive priority over an application of comparable merit
that does not meet the priority (34 CFR 75.105(c)(2)(ii)).
Invitational priority: Under an invitational priority we are
particularly interested in applications that meet the invitational
priority. However, we do not give an application that meets the
invitational priority a competitive or absolute preference over
other applications (34 CFR 75.105(c)(1)).
Proposed Priority
The Secretary proposes a priority for program projects proposing an
evaluation plan that is based on rigorous scientifically based research
methods to assess the effectiveness of a particular intervention. The
Secretary intends that this priority will allow program participants
and the Department to determine whether the project produces meaningful
effects on student achievement or teacher performance.
Evaluation methods using an experimental design are best for
determining project effectiveness. Thus, the project should use an
experimental design under which participants--e.g., students, teachers,
classrooms, or schools--are randomly assigned to participate in the
project activities being evaluated or to a control group that does not
participate in the project activities being evaluated.
If random assignment is not feasible, the project may use a quasi-
experimental design with carefully matched comparison conditions. This
alternative design attempts to approximate a randomly assigned control
group by matching participants--e.g., students, teachers, classrooms,
or schools--with non-participants having similar pre-program
characteristics.
In cases where random assignment is not possible and an extended
series of observations of the outcome of interest precedes and follows
the introduction of a new program or practice, regression discontinuity
designs may be employed.
For projects that are focused on special populations in which
sufficient numbers of participants are not available to support random
assignment or matched comparison group designs, single-subject designs
such as multiple baseline or treatment-reversal or interrupted time
series that are capable of demonstrating causal relationships can be
employed.
Proposed evaluation strategies that use neither experimental
designs with random assignment nor quasi-experimental designs using a
matched comparison group nor regression discontinuity designs will not
be considered responsive to the priority when sufficient numbers of
participants are available to support these designs. Evaluation
strategies that involve too small a number of participants to support
group designs must be capable of demonstrating the causal effects of an
intervention or program on those participants.
The proposed evaluation plan must describe how the project
evaluator will collect--before the project intervention commences and
after it ends--valid and reliable data that measure the impact of
participation in the program or in the comparison group.
If the priority is used as a competitive preference priority,
points awarded under this priority will be determined by the quality of
the proposed evaluation method. In determining the quality of the
evaluation method, we will consider the extent to which the applicant
presents a feasible, credible plan that includes the following:
(1) The type of design to be used (that is, random assignment or
matched comparison). If matched comparison, include in the plan a
discussion of why random assignment is not feasible.
(2) Outcomes to be measured.
(3) A discussion of how the applicant plans to assign students,
teachers, classrooms, or schools to the project and control group or
match them for comparison with other students, teachers, classrooms, or
schools.
(4) A proposed evaluator, preferably independent, with the
necessary background and technical expertise to carry out the proposed
evaluation. An independent evaluator does not have any authority over
the project and is not involved in its implementation.
In general, depending on the implemented program or project, under
a competitive preference priority, random assignment evaluation methods
will receive more points than matched comparison evaluation methods.
Executive Order 12866
This notice of proposed priority has been reviewed in accordance
with Executive Order 12866. Under the terms of the order, we have
assessed the potential costs and benefits of this regulatory action.
The potential costs associated with the notice of proposed priority
are those
[[Page 62447]]
we have determined as necessary for administering applicable programs
effectively and efficiently.
In assessing the potential costs and benefits--both quantitative
and qualitative--of this notice of proposed priority, we have
determined that the benefits of the proposed priority justify the
costs.
We have also determined that this regulatory action does not unduly
interfere with State, local, and tribal governments in the exercise of
their governmental functions.
Intergovernmental Review
Some of the programs affected by this proposed priority are subject
to Executive Order 12372 and the regulations in 34 CFR part 79. One of
the objectives of the Executive order is to foster an intergovernmental
partnership and a strengthened federalism. The Executive order relies
on processes developed by State and local governments for coordination
and review of proposed Federal financial assistance.
This document provides early notification of our specific plans and
actions for these programs.
Electronic Access to This Document
You may view this document, as well as all other Department of
Education documents published in the Federal Register, in text or Adobe
Portable Document Format (PDF) on the Internet at the following site:
http://www.ed.gov/news/fedregister.
To use PDF you must have Adobe Acrobat Reader, which is available
free at this site. If you have questions about using PDF, call the U.S.
Government Printing Office (GPO), toll free, at 1-888-293-6498; or in
the Washington, DC, area at (202) 512-1530.
Note: The official version of this document is the document
published in the Federal Register. Free Internet access to the
official edition of the Federal Register and the Code of Federal
Regulations is available on GPO Access at: http://www.gpoaccess.gov/nara/index.html
.
(Catalog of Federal Domestic Assistance Number does not apply.)
Program Authority: ESEA, as reauthorized by the No Child Left
Behind Act of 2001, Pub. L. 107-110, January 8, 2002.
Dated: October 29, 2003.
Rod Paige,
Secretary of Education.
[FR Doc. 03-27699 Filed 11-3-03; 8:45 am]
Click
here to access the AEA Statement
Click
here to access the Request for Comment in the Federal Register
Click
here to access the AEA cover letter to the Dept of Education
Click
here to access the AEA Press Release
Click here to access the
AERA response to the Request for Comment
Click
here to access the NEA Response as a pdf file
|