Abt Associates: Bold thinkers driving real-world impact
An evaluation completed by Abt Associates for the U.S. Department of Education's Institute of Education Sciences found that while the federal Reading First Program increased instructional time on key components of reading instruction promoted by the program, the impact on student reading comprehension test scores was not statistically significant.
The Reading First Program, established under The No Child Left Behind Act of 2001, is a major federal reading initiative meant to address a persistent education problem — large numbers of the nation's children do not develop the basic reading skills necessary to be successful in school. Reading First provides substantial resources at both the state and local levels to help ensure that all children can read at or above grade level by the end of third grade by improving the quality of reading instruction — and thereby improve the reading skills and achievement of children in the primary grades. The Program promotes instructional practices that have been validated by scientific research.
The Reading First Impact Study is a congressionally mandated evaluation of the Program designed to answer the questions:
The current report presents preliminary answers to the first two questions. The study's final report will address all three questions.
Key findings of the study are that:
The Reading First Impact Study (RFIS) employs a regression discontinuity design that capitalizes on the systematic process used by a number of school districts to allocate their Reading First funds. A regression discontinuity design is the strongest quasi-experimental method that exists for estimating program impacts. Under certain conditions, outlined below, all of which are met by the RFIS, this method can produce unbiased estimates of program impacts:
Under these conditions, there should be no systematic differences between eligible schools that did and did not receive Reading First grants (Reading First and non-Reading First schools respectively), except for the characteristics associated with the school rating used to determine the funding decision. By controlling for differences in schools' ratings, one can then control statistically for all systematic pre-existing differences between the two groups. This makes it possible to estimate the impact of Reading First by comparing the outcomes for Reading First schools and non-Reading First schools in the study sample, controlling for differences in their ratings. Non-Reading First schools in a regression discontinuity analysis thereby play the same role as do control schools in a randomized experiment — they represent the best indications of what outcomes would have been for the treatment group (Reading First schools) in the absence of the program being evaluated.
The study sample included 258 schools (129 Reading First, 129 Non-Reading First) in 18 sites in 13 states; more than 40,000 1st through 3rd grade students were assessed in Fall 2004, Spring 2005 and Spring 2006; approximately 1400 1st and 2nd grade classrooms were observed in Spring 2005, Fall 2005 and Spring 2006.
The study's final report, which is due early 2009, will provide an additional year of follow-up data, and will examine whether the magnitude of impacts on the use of scientifically based reading instruction is associated with improvements in reading comprehension.
Reading First Impact Study: Interim Report
Abt Associates is a mission-driven, global leader in research, evaluation and program implementation in the fields of health, social and environmental policy, and international development. Known for its rigorous approach to solving complex challenges, Abt Associates is regularly ranked as one of the top 20 global research firms and one of the top 40 international development innovators. The company has multiple offices in the U.S. and program offices in more than 40 countries.