Authors: Mark Berends, Ph.D., Senior Social Scientist; Sheila Kirby, Ph.D., Senior Economist; Scott Naftel, Research Programmer; Christopher McKelvey, Research Assistant; RAND Education
Relation of author to model: independent evaluator
Date evaluation report completed: July, 2000
Months/years covered by evaluation: 3 Years of MRSH Implementation, Spring, 1997 through Spring, 1999
Number of schools involved: Total Number of Schools = 104 schools in 7 districts, 6 states Total Number of NAS (New American Schools) designs studied = 7 Number of MRSH Schools = 11 schools in 3 districts
Demographics of sample population: Jurisdictions, overall, that were predominantly urban, high poverty, and high minority partnered with New American Schools' designs. In the analysis sample, MRSH and one other design tended to be in the poorest (economically) schools.
Purpose of study: A progress report that monitors implementation and progress in NAS schools three years into NAS' district scale-up strategy. The purpose is to describe in detail the implementation and performance progress of a longitudinal sample of NAS schools.
Methodology: Survey data were collected in 1997 and 1998 and provide information two and three years into NAS' scale-up phase. These quantitative data were complemented by interviews of key decision-makers in the different jurisdictions undertaken in 1997 through 1999 to gather information about district-level policy changes and their effects on NAS implementation and outcomes. Incorporating the interview and focus group information provides a richer description of what has happened in the NAS schools and districts.
Student performance analyses are presented in trends in student achievement across the seven jurisdictions. Data on test score trends are used to classify schools as those that experienced gains in test scores relative to the district or state.
Measures/indicators used to assess effectiveness: Teacher surveys administered to all the teachers in the NAS schools; principal phone interviews that took about an hour to complete; data provided by districts on school performance indicators (e.g., mandated test scores, attendance rates, promotion and drop out rates, and school demographic characteristics); site visits to schools and school districts to gather information about district and school administrators' and teachers' reports of the progress of the NAS initiative.
Findings: Figure 4 shows the results from a study of student achievement for several comprehensive school reform designs. MRSH schools evidenced the highest rates of student poverty among the seven designs studied and were disproportionately located in districts with poor or weak support for implementing designs. Despite those adversities, MRSH was the design most likely to foster gains on state assessments in reading and mathematics that were greater than district-wide gains. In this study, 73 percent of the MRSH schools had gains in reading that were greater than the district gains. For mathematics, the percent of schools was 64. One other CSR design had success rates in mathematics that were roughly equivalent to MRSH (CSRD E), but their success with reading achievement was substantially lower.
Figure 4
RAND Study of Comprehensive School Reform:
Percent of sampled elementary schools with gains on state assessments*
reading and mathematics that are greater than the district gains by design.
|
 |
*Using data from 6 states, 4th grade cohort comparisons made for 1995-1998.
 % Schools Gaining in Math  % Schools Gaining in Reading
|
|
