**Table 34. **SAGE and Comparison Gain
**From First-Grade Pre-Test to Second Grade From First-Grade Post-Test to Second Grade**
**SCALE**
**SCORE**
**SAGE Gain Comparison**
**Gain**
**Gain**
**Difference**
**SAGE Gain Comparison**
**Gain**
**Gain**
**Difference**
Language Arts 77.07 71.74 5.33* 25.67 22.76 2.91
Reading 72.78 69.62 3.16 22.33 22.01 3.32
Mathematics 77.54 70.07 7.47* 22.86 21.97 0.89
Total 75.90 70.80 5.1* 23.67 22.36 1.31
*significant at .05 level
Regression Analysis
Regression Models. The effect of the SAGE program on student achievement for second
graders was also tested through a series of ordinary least squares regression models for each subtest
and total scale score. Control variables were again entered into the models in blocks, with
the SAGE/comparison student variable entered into the models last. In addition, two different
regressions were done for each sub-test and total scale score. The first regression used the firstgrade
pre-test as a predictor variable and the second regression used the first-grade post-test as a
predictor variable.
The first block of control variables included student score on the first-grade pre-test or
post-test and eligibility for subsidized lunch as an indicator of family income. Because
attendance data were not reliably reported by districts for second graders during 1997-98,
attendance was not included in the analytical model for second graders. As with the first graders
(discussed earlier), the second block of control variables included dummy variables for
race/ethnicity. Finally, a dummy variable for SAGE or comparison school student was entered
on the third block. As with the first graders, this variable is coded 0 if a student is from a
comparison school and 1 if a student is from a SAGE school.
Regression Results. Results of the regression analyses are presented in Tables 35-42.
When either the first-grade pre-test or the first-grade post-test is used as the predictor variable,
39
membership in SAGE emerges as a significant predictor of student achievement on the total
score and for all sub-tests except reading. The magnitude of the effect of SAGE on student
achievement, as denoted by the “b” coefficient, varies depending on the CTBS sub-test.
The largest effects of SAGE are found when the first-grade mathematics pre-test is used
to predict the second-grade test. When all cases are analyzed, the goodness-of-fit of the models
(as denoted by the adjusted R square statistic), ranges from .19 to .47. Most of the variance, as
was the case with the first graders, is explained by the baseline scores (either the first-grade pretest
or the first-grade post-test). “Family Income” and “Race” show some relatively large effects
(as denoted by the b coefficients) and these effects are usually statistically significant. Most
importantly, membership in SAGE schools has a consistently statistically significant positive
effect on the language arts sub-test, the mathematics sub-test, and the total score.
**Table 35. **SCALE SCORES OLS Regression for Language Arts: Pre-Test as Control
**Block 1 Block 2 Block 3**
**Variable **b t b t b t
**Pre-Test Score **.45 20.24* .43 19.22* .43 19.18*
**Subsidized Lunch Eligibility **-3.82 -3.71* -2.22 -2.09* -2.25 -2.12*
**African American **14.58 5.60* 14.22 5.47*
**White **2.48 1.07 2.55 1.11
**SAGE **6.44 3.46*
**Constant **370.29 30.15* 382.62 31.39* 392.46 31.46*
**Adjusted R Squared **.24 .26 .27
**Standard Error of Estimate **36.03 35.43 35.31
*significant at .05 level
**Table 36. **SCALE SCORES OLS Regression for Language Arts: Post-Test as Control
**Block 1 Block 2 Block 3**
**Variable **b t b t b t
**Post-Test Score **.49 20.08* .46 18.62* .45 18.36*
**Subsidized Lunch Eligibility **-4.63 -4.60* -3.08 -2.96* -3.14 -3.02*
**African American **12.97 5.12* 12.60 4.99*
**White **3.18 1.41 3.27 1.46
**SAGE **6.18 3.36*
**Constant **328.73 22.72* 347.62 23.92* 359.60 24.10*
**Adjusted R Squared **.23 .25 .25
**Standard Error of Estimate **36.29 35.77 35.66
*significant at .05 level
40
**Table 37. **SCALE SCORES OLS Regression for Reading: Pre-Test as Control
**Block 1 Block 2 Block 3**
**Variable **b t b t b t
**Pre-Test Score **.37 15.98* .35 14.77* .35 14.77*
**Subsidized Lunch Eligibility **-6.46 -6.89* -5.14 -5.27* -5.14 -5.28*
**African American **6.02 2.50* 5.84 2.42*
**White **5.19 2.44* 5.22 2.45*
**SAGE **3.18 1.85
**Constant **416.18 32.96* 426.13 33.47* 430.56 33.26*
**Adjusted R Squared **.19 .20 .20
**Standard Error of Estimate **32.93 32.68 32.65
*significant at .05 level
**Table 38. **SCALE SCORES OLS Regression for Reading: Post-Test as Control
**Block 1 Block 2 Block 3**
**Variable **b t b t b t
**Post-Test **.46 20.98* .44 19.95* .44 19.85*
**Subsidized Lunch Eligibility **-6.07 -7.06* -4.55 -5.08* -4.57 -5.10*
**African American **5.46 2.51* 5.31 2.43*
**White **6.23 3.23* 6.26 3.25*
**SAGE **2.42 1.53
**Constant **344.12 26.28* 352.98 26.84* 357.25 26.58*
**Adjusted R Squared **.26 .27 .27
**Standard Error of Estimate **31.08 30.76 30.75
*significant at .05 level
**Table 39. **SCALE SCORES OLS Regression for Mathematics: Pre-Test as Control
**Block 1 Block 2 Block 3**
**Variable **b t b t b t
**Pre-Test Score **.63 -2.80* .58 25.04* .58 25.23*
**Subsidized Lunch Eligibility **-2.66 -2.80* -1.21 -1.26 -1.22 -1.27
**African American **17.55 7.35* 17.34 7.32*
**White **1.50 .70 1.45 .68
**SAGE **7.94 4.73*
**Constant **262.50 22.76* 288.48 24.88* 299.02 25.49*
**Adjusted R Squared **.36 .39 .40
**Standard Error of Estimate **32.80 31.96 31.74
*significant at .05 level
41
**Table 40. **SCALE SCORES OLS Regression for Mathematics: Post-Test as Control
**Block 1 Block 2 Block 3**
**Variable **b t b t b t
**Post-Test Score **.55 28.17* .50 25.38* .50 25.14*
**Subsidized Lunch Eligibility **-4.78 -5.36* -3.01 -3.30* -3.05 -3.35*
**African American **15.14 6.73* 14.90 6.63*
**White **4.10 2.06* 4.20 2.12*
**SAGE **3.97 2.44*
**Constant **274.87 25.18* 300.38 27.30* 307.76 27.01*
**Adjusted R Squared **.36 .39 .39
**Standard Error of Estimate **32.22 31.41 31.37
*significant at .05 level
**Table 41. **SCALE SCORES OLS Regression for Total: Pre-Test as Control
**Block 1 Block 2 Block 3**
**Variable **b t b t b t
**Pre-Test Score **.65 30.92* .62 28.85* .62 28.93*
**Subsidized Lunch Eligibility **-2.63 -3.42* -1.82 -2.32* -1.84 -2.36*
**African American **12.04 6.19* 11.86 6.12*
**White **.01 .01 .04 .02
**SAGE **5.21 3.80*
**Constant **258.65 22.99* 276.79 24.39* 284.31 24.78*
**Adjusted R Squared **.42 .44 .45
**Standard Error of Estimate **26.10 25.66 25.55
*significant at .05 level
**Table 42. **SCALE SCORES OLS Regression for Total: Post-Test as Control
**Block 1 Block 2 Block 3**
**Variable **b t b t b t
**Post-Test Score **.69 33.44* .65 31.12* .65 30.88*
**Subsidized Lunch Eligibility **-3.31* -4.60* -2.29 -3.11* -2.33 -3.16*
**African American **9.75 5.42* 9.58 5.32*
**White **2.40 1.51 2.47 1.56
**SAGE **2.95 2.26*
**Constant **204 16.93* 225.24 18.47* 231.32 18.54*
**Adjusted R Squared **.45 .46 .47
**Standard Error of Estimate **25.31 24.91 24.87
*significant at .05 level
African-American Students
Like the first graders, African-American second-grade students definitely comprise the
largest subgroup of valid test scores – roughly 21% of all SAGE students and 25% of all
comparison students. In the analyses to follow, African-American students are first compared
across SAGE and comparison schools on the CTBS sub-tests and total scale score. Second,
42
African-American students are compared to white students across SAGE and comparison
schools on the CTBS Total Scale Score.
SAGE vs. Comparison. Table 43 provides comparisons of means on the CTBS secondgrade
test, as well as change scores from the first-grade pre-test to the second-grade test and
from the first-grade post-test to the second-grade test. On the second-grade test, African-
American SAGE students scored higher than comparison school students on every sub-test and
on the total scale score. However, the differences between SAGE and comparison students on
the second-grade test scores are not statistically significant. When using the first-grade pre-test
as the baseline score, statistically significant change scores are found on all scores except for
reading. However, using the first-grade post-test as the baseline score shows no statistically
significant differences between SAGE and comparison schools.
43
**Table 43. **African-American Post-Test and Change Scores, by SAGE or Comparison
**SCORE SAGE COMPARISON**
**Language Arts**
Mean Second-Grade Score 591.09 587.19
Mean Change From First-Grade
Pre-Test to Second Grade
72.41* 62.26
Mean Change From First-Grade
Post-Test to Second Grade
19.41 20.62
**Reading**
Mean Second-Grade Score 594.54 543.48
Mean Change From First-Grade
Pre-Test to Second Grade
73.07 68.48
Mean Change From First-Grade
Post-Test to Second Grade
19.30 20.91
**Mathematics**
Mean Second-Grade Score 545.44 543.94
Mean Change From First-Grade
Pre-Test to Second Grade
72.59* 61.08
Mean Change From First-Grade
Post-Test to Second Grade
16.46 20.58
**Total**
Mean Second-Grade Score 577.04 575.94
Mean Change From First-Grade
Pre-Test to Second Grade
72.44* 65.44
Mean Change From First-Grade
Post-Test to Second Grade
17.99 21.14
*significant at .05 level
African-American Males. Table 44 further distinguishes African-American SAGE and
comparison school students by gender. These results show no significant gender-related gain
differences when using either the first-grade pre-test or the first-grade post-test as a baseline
measure.
44
**Table 44. **African-American Post-Test and Change Scores by Gender
**COMPARISON SAGE**
Male Female Male Female
**Language Arts**
Mean Post-Test Scale Score 579.63 591.49 590.74 592.22
Change (Pre-Test as Baseline) 62.03 62.52 73.74 72.09
Change (Post-Test as Baseline) 23.97 18.31 19.48 19.35
**Reading**
Mean Post-Test Scale Score 584.84 596.65 590.50 598.16
Change (Pre-Test as Baseline) 66.23 69.74 71.13 74.34
Change (Post-Test as Baseline) 25.24 17.92 13.62 24.21
**Mathematics**
Mean Post-Test Scale Score 538.82 547.33 550.29 540.34
Change (Pre-Test as Baseline) 55.68 65.23 74.99 69.35
Change (Post-Test as Baseline) 21.91 25.46 21.65 30.79
**Total**
Mean Post-Test Scale Score 568.54 580.32 577.36 576.87
Change (Pre-Test as Baseline) 62.09 67.74 73.17 72.11
Change (Post-Test as Baseline) 22.24 20.36 16.60 19.21
*significant at .05 level
African-American and White Achievement. African-American students scored lower
than white students on the first-grade pre-test total scale score, as shown in table 45. This result
is statistically significant for both SAGE and comparison schools, though the gap between
African Americans and whites is larger in the SAGE schools. The change from first-grade posttest
to the second-grade test shows that the SAGE African Americans kept pace with white
students, but did not further close the achievement gap in second grade.
**Table 45. **African-American versus White Achievement on Total Scale
**First-Grade**
**Pre-Test**
**First-Grade**
**Post-Test**
**Second**
**Grade**
**Change From**
**Pre to Second**
**Change From**
**Post to Second**
**SAGE**
African
American
504.09 559.02 577.04 72.44 17.99
White 533.21 582.56 606.81 73.61 24.50
F 142.06* 94.70* 132.69* .291 8.46*
**Comparison**
African
American
510.22 553.48 575.94 65.44 21.14
White 528.79 578.28 602.67 73.60 23.80
F 32.24* 61.80* 66.28* 7.46* .925
*significant at .05 level
45
Hierarchical Linear Modeling
Hierarchical linear models were used to evaluate the second-grade achievement results
using the same series of models used to assess first-grade results. Two sets of analyses were
done. The first, shown in Table 45 used first-grade pre-test as the initial achievement level of the
students. The second, shown in Table 46, used first-grade post-test as the initial achievement
level of the students.
HLM Results. Tables 46 and 47 provide a summary of the effects of each of the level-1
and level-2 variables for each of these analyses. Level-1 effects can be interpreted as the
weighted average of the within classroom effects of the level-1 variables. Level-2 effects can be
interpreted as the classroom effects of the level-2 variables. Level-1 coefficients may be thought
of as the average effect of the modeling variable on the criterion score at the individual level.
The level-1 results indicate that lower SES is related to lower post-test scores and higher pre-test
scores are related to higher post-test scores.
The coefficients associated with the level-2 variables can be thought of as classroom
effects. For example, in the Model A total score, an increase of one student in class size resulted
in a drop of .764 points for the class average. Likewise, SAGE participation resulted in a 6.322
point gain in the class average on total score for Model B. A discussion of each model follows.
**Model A. **Depending on the test, an increase in class size of one person can be expected
to produce a .66 to 1.01 loss in average post-test performance. The results for all scores show
this effect to be significant. It is noteworthy that the results of Tables 41 and 42 for the Model A
are similar, indicating that the second-grade class size has little effect on the two year gain (first
grade and second grade) versus the one year gain (second grade only).
**Model B. **Participation in SAGE shows no statistically significant class average
increases in all post-test scores with the exception of mathematics in the pre-test condition. A
46
comparison of these results with the first-grade only results indicates second-grade SAGE effects
of lesser magnitude than the first-grade effects. These results tend to support the hypothesis that
SAGE is not continuing to produce the same advantage through second grade. The advantage of
SAGE students is, however, maintained.
**Model C. **Combining class size and SAGE participation in a single analysis isolates the
effects that SAGE might have beyond those produced by lower class size. The results show that
once class size has been accounted for, SAGE has no significant effect on class average
performance.
**Model D. **Since socioeconomic status is known to have an influence on academic test
scores, a replacement for this variable was used as both a level-1 and level-2 predictor. The
level-2 variable was the average SES for the class and estimates the effect of the overall class
SES level beyond that associated with the individual, which is accounted for in the level-1
model. This model combines class SES and class size. The results indicate that class SES has a
significant effect on the class average post-test performance. The effect of a 1 point class
average gain in SES equates to between a 10 point and 13 point gain on the average post-test
score, depending on the test. SES was measured on a three-point family income scale; thus a one
point difference on average would be quite pronounced. Class size still has a significant effect
on the post-test scores once SES has been accounted for.
**Model E. **When class SES and SAGE participation are entered in the same level-2
model, class SES has a significant effect on class average post-test performance. However, for
the most part, SAGE has no significant effect on class average post-test performance once class
SES is controlled. In other words, the effects of SAGE participation on class average post-test
scores, beyond those produced by SES differences, are not significant on post-test scores. The
single exception is for the mathematics score across the two-year time span. In general, the
47
SAGE effects are roughly the same as when SAGE is the only variable in the model (see Model
B), suggesting that SAGE classrooms and comparison classrooms are about equal on class SES.
**Model F. **This model combines SES, SAGE participation, and class size in a single
analysis. For most sub-tests, class SES once again has a significant effect on the class average
post-test score. Again, class size has a significant effect on the class average post-test score for
all scores except language. Finally, SAGE has no significant effect on any sub-test, once class
size is accounted for.
**Table 46. **HLM Results for 1997-98 Second-Grade Students – Pre-Test as Initial Achievement
**Source Total Reading Language Arts Mathematics**
**Level 1**
Pre-Test 0.703 0.440 0.509 .662
SES -1.929 -4.503 -1.069 -3.196
**Level 2**
A. Class Size -0.764* -0.630* -0.738* -1.007*
B. SAGE 6.322* 3.302* 7.354* 8.612*
C. Class Size -0.840* -0.886* -0.703 -1.073*
SAGE 1.533 5.152 0.731 1.372
D. Class SES -9.129* -12.214* -7.932 -6.661
Class Size 0.678* -0.523* -0.666* 0.946*
E. Class SES -10.232* -13.214* -8.944* -8.251*
SAGE 5.939 3.024 7.042 8.392*
F. Class SES -9.086* -12.018* -8.017 -6.613
Class Size -0.714* -0.715* -0.593 -0.979*
SAGE 0.708 3.825 -1.487 0.681
*significant at .05 level
**Table 47. **HLM Results for 1997-98 Second-Grade Students – Post Test as Initial Achievement
**Source Total Reading Language Arts Mathematics**
**Level 1**
Pre-Test 0.750 0.506 0.551 .574
SES -2.742 -4.142 -4.640 -4.703
**Level 2**
A. Class Size -0.811* -0.669* -0.732* -0.959*
B. SAGE 6.296 3.651 7.432 7.741
C. Class Size -0.916* -0.916* -0.683 -1.038*
SAGE 2.177 5.043 1.004 1.622
D. Class SES -9.106* -12.888* -9.069* -5.906*
Class Size 0.726* -0.553* -0.646* 0.911*
E. Class SES -10.339* -13.836* -10.079* -7.254
SAGE 6.007 3.300 7.040 7.616
F. Class SES -9.023* -12.637* -9.172* -5.835*
Class Size -0.787* -.739* -0.556 -0.962*
SAGE 1.243 3.737 1.825 1.029
*significant at .05 level
48
**Effects Within SAGE Classrooms**
It is of some interest to examine achievement gains in light of various classroom and
teacher characteristics, teacher behaviors and student behaviors. Except for the “Teacher
Experience and Achievement” analyses, data for these questions were collected only for SAGE
first-grade classrooms.
Teacher Experience and Achievement
Student gains (at the classroom level) were correlated with teacher experience. In those
classrooms with more than one teacher, teacher experience was averaged to obtain an appropriate
experience variable for that classroom. Correlations between teacher experience and
achievement gains ranged from .019 to .140, but none were significant for either the first-grade
cohort or the second-grade cohort.
Student Participation and Achievement
The student participation questionnaire factored into two scales as was noted earlier.
These scales were “active learning” and “on-task behavior.” Interest here centered on the
relationship between these variables and the classroom characteristics of class size and teacher
experience as well as achievement gains. Composite active learning and on-task variables were
formed for each classroom by averaging over the administrations and across students in each
class. (Note, there were no significant changes in either of these variables across time). These
variables were then correlated with class size, teacher experience and achievement gain in each
of the three achievement sub-score areas. The on-task variable showed no significant
relationships with any of the classroom variables. For the 1996-97 first-grade cohort, active
learning was significantly correlated with class size (r=-.204; p<.05). For the 1997-98 first-grade
cohort, active learning correlated significantly with teacher experience (r=.307; p<.05). Since
these correlations did not replicate across cohorts, little practical significance should be placed in
49
these findings.
Proximity to the Curriculum and Achievement
From the teacher questionnaire, variables associated with proximity to the curriculum
were constructed. The only significant correlation was the language arts proximity score with
reading gain (r=.270) for the 1997-98 first-grade cohort. Again, since these results did not
replicate across cohorts, little practical significance should be placed in this finding.
Class Organization and Achievement
SAGE students are organized into different types of classrooms as discussed earlier in the
report. These classrooms include 15:1 Regular classrooms, 30:2 2-Team Teaching classrooms,
15:1 Shared-Space classrooms, and 30:2 Floating Teacher classrooms. Available data consisted
of fifty-nine 15:1 classrooms, and thirty-one 30:2 classrooms. Neither the 15:1 Shared Space
nor the 30:2 Floating Teacher class organization contained enough classrooms to analyze each of
these four types separately. This analysis uses Hierarchical Linear Modeling to make this
comparison. The HLM results show no statistically significant relationship between type of
classroom organization and achievement on any of the sub-tests or the total score.
50
ANALYSES OF SAGE TEACHERS, CLASSROOMS, AND SCHOOLS 1997-98
**Share with your friends:** |