[Federal Register: November 4, 2003 (Volume 68, Number 213)]
[Notices]               
[Page 62445-62447]
From the Federal Register Online via GPO Access [wais.access.gpo.gov]
[DOCID:fr04no03-38]                         

-----------------------------------------------------------------------

DEPARTMENT OF EDUCATION

RIN 1890-ZA00

 
Scientifically Based Evaluation Methods

AGENCY: Department of Education.

ACTION: Notice of proposed priority.

-----------------------------------------------------------------------

SUMMARY: The Secretary of Education proposes a priority that may be 
used for any appropriate programs in the Department of Education 
(Department) in FY 2004 and in later years. We take this action to 
focus Federal financial assistance on expanding the number of programs 
and projects Department wide that are evaluated under rigorous 
scientifically based research methods in accordance with the Elementary 
and Secondary Education Act (ESEA) as reauthorized by the No Child Left 
Behind Act of 2001 (NCLB). Establishing the priority on a Department-
wide basis would permit any office to use the priority for a program 
for which it is appropriate.

DATES: We must receive your comments on or before December 4, 2003.

ADDRESSES: Address all comments about this proposed priority to Margo 
K. Anderson, U.S. Department of Education, 400 Maryland Avenue, SW., 
room 4W333, Washington, DC 20202-5910. If you prefer to send your 
comments through the Internet, use the following address: comments@ed.gov.
    You must include the term ``Evaluation'' in the subject line of 
your electronic message.

FOR FURTHER INFORMATION CONTACT: Margo Anderson. Telephone: (202) 205-3010 or via Internet at Margo.Anderson@ed.gov.
    If you use a telecommunications device for the deaf (TDD), you may 
call the Federal Information Relay Service (FIRS) at 1-800-877-8339.
    Individuals with disabilities may obtain this document in an 
alternative format (e.g., Braille, large print, audiotape, or computer 
diskette) on request to the contact person listed under FOR FURTHER 
INFORMATION CONTACT.

SUPPLEMENTARY INFORMATION:

Invitation To Comment

    We invite you to submit comments regarding this proposed priority.
    We invite you to assist us in complying with the specific 
requirements of Executive Order 12866 and its overall requirement of 
reducing regulatory burden that might result from this proposed 
priority. Please let us know of any further opportunities we should 
take to reduce potential costs or

[[Page 62446]]

increase potential benefits while preserving the effective and 
efficient administration of the Department's programs.
    During and after the comment period, you may inspect all public 
comments about this proposed priority in room 4W333, 400 Maryland 
Avenue, SW., Washington, DC between the hours of 8:30 a.m. and 4 p.m., 
Eastern time, Monday through Friday of each week except Federal 
holidays.

Assistance to Individuals With Disabilities in Reviewing the Rulemaking 
Record

    On request, we will supply an appropriate aid, such as a reader or 
print magnifier, to an individual with a disability who needs 
assistance to review the comments or other documents in the public 
rulemaking record for this proposed priority. If you want to schedule 
an appointment for this type of aid, please contact the person listed 
under FOR FURTHER INFORMATION CONTACT.

General

    The ESEA as reauthorized by the NCLB uses the term scientifically 
based research more than 100 times in the context of evaluating 
programs to determine what works in education or ensuring that Federal 
funds are used to support activities and services that work. This 
proposed priority is intended to ensure that Federal funds are used to 
support projects and activities that are consistent with a statutory 
purpose of Department programs, and evaluated using scientifically 
based research. Establishing this priority makes it possible for any 
office in the Department to encourage or to require appropriate 
projects to use scientifically based evaluation strategies to determine 
the effectiveness of a project intervention.

Discussion of Proposed Priority

    We will announce the final priority in a notice in the Federal 
Register. We will determine the final priority after considering public 
comments on this proposed priority and other information available to 
the Department. This notice does not preclude the Secretary from 
proposing or funding additional priorities, subject to meeting 
applicable rulemaking requirements.

    Note: This notice does not solicit applications. In any year in 
which we choose to use this proposed priority, we invite 
applications for new awards under the applicable program through a 
notice in the Federal Register. When inviting applications we 
designate the priority as absolute, competitive preference, or 
invitational. The effect of each type of priority follows:
    Absolute priority: Under an absolute priority we consider only 
applications that meet the priority (34 CFR 75.105(c)(3)).
    Competitive preference priority: Under a competitive preference 
priority we give competitive preference to an application by either 
(1) awarding additional points, depending on how well or the extent 
to which the application meets the competitive preference priority 
(34 CFR 75.105(c)(2)(i)); or (2) selecting an application that meets 
the competitive priority over an application of comparable merit 
that does not meet the priority (34 CFR 75.105(c)(2)(ii)).
    Invitational priority: Under an invitational priority we are 
particularly interested in applications that meet the invitational 
priority. However, we do not give an application that meets the 
invitational priority a competitive or absolute preference over 
other applications (34 CFR 75.105(c)(1)).

Proposed Priority

    The Secretary proposes a priority for program projects proposing an 
evaluation plan that is based on rigorous scientifically based research 
methods to assess the effectiveness of a particular intervention. The 
Secretary intends that this priority will allow program participants 
and the Department to determine whether the project produces meaningful 
effects on student achievement or teacher performance.
    Evaluation methods using an experimental design are best for 
determining project effectiveness. Thus, the project should use an 
experimental design under which participants--e.g., students, teachers, 
classrooms, or schools--are randomly assigned to participate in the 
project activities being evaluated or to a control group that does not 
participate in the project activities being evaluated.
    If random assignment is not feasible, the project may use a quasi-
experimental design with carefully matched comparison conditions. This 
alternative design attempts to approximate a randomly assigned control 
group by matching participants--e.g., students, teachers, classrooms, 
or schools--with non-participants having similar pre-program 
characteristics.
    In cases where random assignment is not possible and an extended 
series of observations of the outcome of interest precedes and follows 
the introduction of a new program or practice, regression discontinuity 
designs may be employed.
    For projects that are focused on special populations in which 
sufficient numbers of participants are not available to support random 
assignment or matched comparison group designs, single-subject designs 
such as multiple baseline or treatment-reversal or interrupted time 
series that are capable of demonstrating causal relationships can be 
employed.
    Proposed evaluation strategies that use neither experimental 
designs with random assignment nor quasi-experimental designs using a 
matched comparison group nor regression discontinuity designs will not 
be considered responsive to the priority when sufficient numbers of 
participants are available to support these designs. Evaluation 
strategies that involve too small a number of participants to support 
group designs must be capable of demonstrating the causal effects of an 
intervention or program on those participants.
    The proposed evaluation plan must describe how the project 
evaluator will collect--before the project intervention commences and 
after it ends--valid and reliable data that measure the impact of 
participation in the program or in the comparison group.
    If the priority is used as a competitive preference priority, 
points awarded under this priority will be determined by the quality of 
the proposed evaluation method. In determining the quality of the 
evaluation method, we will consider the extent to which the applicant 
presents a feasible, credible plan that includes the following:
    (1) The type of design to be used (that is, random assignment or 
matched comparison). If matched comparison, include in the plan a 
discussion of why random assignment is not feasible.
    (2) Outcomes to be measured.
    (3) A discussion of how the applicant plans to assign students, 
teachers, classrooms, or schools to the project and control group or 
match them for comparison with other students, teachers, classrooms, or 
schools.
    (4) A proposed evaluator, preferably independent, with the 
necessary background and technical expertise to carry out the proposed 
evaluation. An independent evaluator does not have any authority over 
the project and is not involved in its implementation.
    In general, depending on the implemented program or project, under 
a competitive preference priority, random assignment evaluation methods 
will receive more points than matched comparison evaluation methods.

Executive Order 12866

    This notice of proposed priority has been reviewed in accordance 
with Executive Order 12866. Under the terms of the order, we have 
assessed the potential costs and benefits of this regulatory action.
    The potential costs associated with the notice of proposed priority 
are those

[[Page 62447]]

we have determined as necessary for administering applicable programs 
effectively and efficiently.
    In assessing the potential costs and benefits--both quantitative 
and qualitative--of this notice of proposed priority, we have 
determined that the benefits of the proposed priority justify the 
costs.
    We have also determined that this regulatory action does not unduly 
interfere with State, local, and tribal governments in the exercise of 
their governmental functions.

Intergovernmental Review

    Some of the programs affected by this proposed priority are subject 
to Executive Order 12372 and the regulations in 34 CFR part 79. One of 
the objectives of the Executive order is to foster an intergovernmental 
partnership and a strengthened federalism. The Executive order relies 
on processes developed by State and local governments for coordination 
and review of proposed Federal financial assistance.
    This document provides early notification of our specific plans and 
actions for these programs.

Electronic Access to This Document

    You may view this document, as well as all other Department of 
Education documents published in the Federal Register, in text or Adobe 
Portable Document Format (PDF) on the Internet at the following site: 
http://www.ed.gov/news/fedregister.
    To use PDF you must have Adobe Acrobat Reader, which is available 
free at this site. If you have questions about using PDF, call the U.S. 
Government Printing Office (GPO), toll free, at 1-888-293-6498; or in 
the Washington, DC, area at (202) 512-1530.

    Note: The official version of this document is the document 
published in the Federal Register. Free Internet access to the 
official edition of the Federal Register and the Code of Federal 
Regulations is available on GPO Access at: http://www.gpoaccess.gov/nara/index.html
.


(Catalog of Federal Domestic Assistance Number does not apply.)


    Program Authority:  ESEA, as reauthorized by the No Child Left 
Behind Act of 2001, Pub. L. 107-110, January 8, 2002.

    Dated: October 29, 2003.
Rod Paige,
Secretary of Education.
[FR Doc. 03-27699 Filed 11-3-03; 8:45 am]

BILLING CODE 4000-01-P