Author: Allan Sterbin, Research Associate, Mid-South Center for School Evaluation and Reform, University of Memphis
Relation of author to model: independent evaluator
Date evaluation report completed: 2001
Months/years covered by evaluation: 1995-2000
Number of schools involved: 28
Demographics of sample population: Title I schools in Memphis, predominantly African-American
Purpose of study: Assess relative effectiveness of MRSH elementary school in Memphis with all other Title I schools.
Methodology: The present study compared longitudinal data from Rozelle Elementary on classroom practices and teacher perceptions with those from selected Title I schools in Memphis. Additionally, longitudinal data on student achievement and value-added scores from Rozelle Elementary, Title I schools, all Memphis City Schools, and Tennessee schools were compared.
Measures/indicators used to assess effectiveness: Each spring Tennessee students in grades 3-8 take an achievement test as part of the Tennessee Comprehensive Assessment Program (TCAP). The primary aim of the TCAP is to provide a measure of basic skills in Reading, Language, Math, Social Studies, and Science. In addition to reporting mean national percentiles, the state also computes Tennessee Value-Added Assessment System (TVAAS) performance scores. TVAAS constitutes value added (year-to-year gain scores) on the Terra Nova (TCAP) in five subjects and allows for a highly precise assessment of achievement that eliminates confounding influences from socioeconomic status and student ability (Sanders & Horn, 1995a, 1995b). TVAAS scores were expressed as cumulative percentage of the norm.
School Climate Inventory (SCI). The SCI was developed by researchers at the Center for Research in Educational Policy (CREP) at the University of Memphis in 1989. The SCI consists of seven dimensions linked with factors associated with effective school organizational climates, including Order, Leadership, Environment, Involvement, Instruction, Expectations, and Collaboration. Each dimension is comprised of seven items, with 49 statements comprising the inventory.
School Observation Measure (SOM). The SOM was developed by researchers at CREP and has high inter-rater reliability. Observers visit 10 randomly selected classrooms in the school for 15 minutes per classroom and observe the frequency of 24 different classroom practices including instructional orientation, classroom organization, instructional strategies, student activities, technology use, assessment, and academic focus/student interest (see Appendix). At the conclusion of the three-hour visit, observers summarize the frequency with which each of the strategies was observed across all classes. Frequency is recorded via a 5-point rubric that ranges from (0) Not Observed to (4) Extensively.
School Reform Teacher Questionnaire (SRTQ). The SRTQ was developed by researchers at the University of Memphis - CREP and is composed of 30 items to which teachers respond using a five-point Likert-type scale, ranging from strongly agree to strongly disagree. Face validity of the items and logical ordering of the items by scales were established by the research team during the development of the inventory. The four scales of the SRTQ include Professional Development, Resources, Pedagogical Change, and Outcomes.
Findings: TVAAS (gain) scores provide the longitudinal data on student achievement. In 1995 (as shown in Table 2), prior to the adoption of MRSH, Rozelle achieved a much lower gain score than the comparison groups (M=68.9), but then steadily increased its relative standing each year of the next two, surpassing all groups in 1997, 1998, and 1999. While experiencing a lower gain in 2000, it still surpassed the MCS and Title I schools while being comparable to the state.
Table 2 Student Achievement - Gain Scores - 1995- 2000
|
1995 |
1996 |
1997 |
1998 |
1999 |
2000 |
|
Tennessee |
107.97 |
99.25 |
94.62 |
112.34 |
105.23 |
99.63 |
MCS |
110.24 |
89.94 |
94.93 |
119.11 |
109.17 |
84.93 |
Title 1 |
113.75 |
88.43 |
102.18 |
112.19 |
103.97 |
83.14 |
Rozelle |
68.89 |
85.99 |
103.81 |
122.66 |
146.98 |
96.21 |
|
Source: Sterbin, Allan. 2001. " Rozelle Elementary School : A Longitudinal Analysis, 1995-2000." Mid-South Center for School Evaluation and Reform, University of Memphis . |
In terms of school climate, Rozelle teachers were significantly (.001) more likely to agree to these statements than comparison schools:
- Student tardiness and absence from school is a major problem
- Information about school activities is communicated to parents on a consistent basis
- Teachers are proud of this school and it's students
- Teachers, administrators, and parents assume joint responsibility for student discipline
- The goals of this school are reviewed and updated regularly
- Student behavior is generally positive in this school
Significant differences obtained at .01 for these items:
- Faculty and staff cooperate a great deal in trying to achieve school goals
- Teachers use appropriate evaluation methods to determine student achievement
- Parents are often invited to visit classrooms
- The school is a safe and secure place in which to work
- The principal is an effective instructional leader
- Teachers have high expectations for all students
- The principal is highly visible throughout the school
- People in this school really care about each other
|
