The student achievement guarantee



Download 0.91 Mb.
Page4/8
Date conversion15.02.2016
Size0.91 Mb.
1   2   3   4   5   6   7   8

Table 34. SAGE and Comparison Gain

From First-Grade Pre-Test to Second Grade From First-Grade Post-Test to Second Grade

SCALE

SCORE

SAGE Gain Comparison

Gain

Gain

Difference

SAGE Gain Comparison

Gain

Gain

Difference

Language Arts 77.07 71.74 5.33* 25.67 22.76 2.91

Reading 72.78 69.62 3.16 22.33 22.01 3.32

Mathematics 77.54 70.07 7.47* 22.86 21.97 0.89

Total 75.90 70.80 5.1* 23.67 22.36 1.31

*significant at .05 level

Regression Analysis

Regression Models. The effect of the SAGE program on student achievement for second

graders was also tested through a series of ordinary least squares regression models for each subtest

and total scale score. Control variables were again entered into the models in blocks, with

the SAGE/comparison student variable entered into the models last. In addition, two different

regressions were done for each sub-test and total scale score. The first regression used the firstgrade

pre-test as a predictor variable and the second regression used the first-grade post-test as a

predictor variable.

The first block of control variables included student score on the first-grade pre-test or

post-test and eligibility for subsidized lunch as an indicator of family income. Because

attendance data were not reliably reported by districts for second graders during 1997-98,

attendance was not included in the analytical model for second graders. As with the first graders

(discussed earlier), the second block of control variables included dummy variables for

race/ethnicity. Finally, a dummy variable for SAGE or comparison school student was entered

on the third block. As with the first graders, this variable is coded 0 if a student is from a

comparison school and 1 if a student is from a SAGE school.

Regression Results. Results of the regression analyses are presented in Tables 35-42.

When either the first-grade pre-test or the first-grade post-test is used as the predictor variable,

39

membership in SAGE emerges as a significant predictor of student achievement on the total



score and for all sub-tests except reading. The magnitude of the effect of SAGE on student

achievement, as denoted by the “b” coefficient, varies depending on the CTBS sub-test.

The largest effects of SAGE are found when the first-grade mathematics pre-test is used

to predict the second-grade test. When all cases are analyzed, the goodness-of-fit of the models

(as denoted by the adjusted R square statistic), ranges from .19 to .47. Most of the variance, as

was the case with the first graders, is explained by the baseline scores (either the first-grade pretest

or the first-grade post-test). “Family Income” and “Race” show some relatively large effects

(as denoted by the b coefficients) and these effects are usually statistically significant. Most

importantly, membership in SAGE schools has a consistently statistically significant positive

effect on the language arts sub-test, the mathematics sub-test, and the total score.



Table 35. SCALE SCORES OLS Regression for Language Arts: Pre-Test as Control

Block 1 Block 2 Block 3

Variable b t b t b t

Pre-Test Score .45 20.24* .43 19.22* .43 19.18*

Subsidized Lunch Eligibility -3.82 -3.71* -2.22 -2.09* -2.25 -2.12*

African American 14.58 5.60* 14.22 5.47*

White 2.48 1.07 2.55 1.11

SAGE 6.44 3.46*

Constant 370.29 30.15* 382.62 31.39* 392.46 31.46*

Adjusted R Squared .24 .26 .27

Standard Error of Estimate 36.03 35.43 35.31

*significant at .05 level



Table 36. SCALE SCORES OLS Regression for Language Arts: Post-Test as Control

Block 1 Block 2 Block 3

Variable b t b t b t

Post-Test Score .49 20.08* .46 18.62* .45 18.36*

Subsidized Lunch Eligibility -4.63 -4.60* -3.08 -2.96* -3.14 -3.02*

African American 12.97 5.12* 12.60 4.99*

White 3.18 1.41 3.27 1.46

SAGE 6.18 3.36*

Constant 328.73 22.72* 347.62 23.92* 359.60 24.10*

Adjusted R Squared .23 .25 .25

Standard Error of Estimate 36.29 35.77 35.66

*significant at .05 level

40

Table 37. SCALE SCORES OLS Regression for Reading: Pre-Test as Control

Block 1 Block 2 Block 3

Variable b t b t b t

Pre-Test Score .37 15.98* .35 14.77* .35 14.77*

Subsidized Lunch Eligibility -6.46 -6.89* -5.14 -5.27* -5.14 -5.28*

African American 6.02 2.50* 5.84 2.42*

White 5.19 2.44* 5.22 2.45*

SAGE 3.18 1.85

Constant 416.18 32.96* 426.13 33.47* 430.56 33.26*

Adjusted R Squared .19 .20 .20

Standard Error of Estimate 32.93 32.68 32.65

*significant at .05 level



Table 38. SCALE SCORES OLS Regression for Reading: Post-Test as Control

Block 1 Block 2 Block 3

Variable b t b t b t

Post-Test .46 20.98* .44 19.95* .44 19.85*

Subsidized Lunch Eligibility -6.07 -7.06* -4.55 -5.08* -4.57 -5.10*

African American 5.46 2.51* 5.31 2.43*

White 6.23 3.23* 6.26 3.25*

SAGE 2.42 1.53

Constant 344.12 26.28* 352.98 26.84* 357.25 26.58*

Adjusted R Squared .26 .27 .27

Standard Error of Estimate 31.08 30.76 30.75

*significant at .05 level



Table 39. SCALE SCORES OLS Regression for Mathematics: Pre-Test as Control

Block 1 Block 2 Block 3

Variable b t b t b t

Pre-Test Score .63 -2.80* .58 25.04* .58 25.23*

Subsidized Lunch Eligibility -2.66 -2.80* -1.21 -1.26 -1.22 -1.27

African American 17.55 7.35* 17.34 7.32*

White 1.50 .70 1.45 .68

SAGE 7.94 4.73*

Constant 262.50 22.76* 288.48 24.88* 299.02 25.49*

Adjusted R Squared .36 .39 .40

Standard Error of Estimate 32.80 31.96 31.74

*significant at .05 level

41

Table 40. SCALE SCORES OLS Regression for Mathematics: Post-Test as Control

Block 1 Block 2 Block 3

Variable b t b t b t

Post-Test Score .55 28.17* .50 25.38* .50 25.14*

Subsidized Lunch Eligibility -4.78 -5.36* -3.01 -3.30* -3.05 -3.35*

African American 15.14 6.73* 14.90 6.63*

White 4.10 2.06* 4.20 2.12*

SAGE 3.97 2.44*

Constant 274.87 25.18* 300.38 27.30* 307.76 27.01*

Adjusted R Squared .36 .39 .39

Standard Error of Estimate 32.22 31.41 31.37

*significant at .05 level



Table 41. SCALE SCORES OLS Regression for Total: Pre-Test as Control

Block 1 Block 2 Block 3

Variable b t b t b t

Pre-Test Score .65 30.92* .62 28.85* .62 28.93*

Subsidized Lunch Eligibility -2.63 -3.42* -1.82 -2.32* -1.84 -2.36*

African American 12.04 6.19* 11.86 6.12*

White .01 .01 .04 .02

SAGE 5.21 3.80*

Constant 258.65 22.99* 276.79 24.39* 284.31 24.78*

Adjusted R Squared .42 .44 .45

Standard Error of Estimate 26.10 25.66 25.55

*significant at .05 level



Table 42. SCALE SCORES OLS Regression for Total: Post-Test as Control

Block 1 Block 2 Block 3

Variable b t b t b t

Post-Test Score .69 33.44* .65 31.12* .65 30.88*

Subsidized Lunch Eligibility -3.31* -4.60* -2.29 -3.11* -2.33 -3.16*

African American 9.75 5.42* 9.58 5.32*

White 2.40 1.51 2.47 1.56

SAGE 2.95 2.26*

Constant 204 16.93* 225.24 18.47* 231.32 18.54*

Adjusted R Squared .45 .46 .47

Standard Error of Estimate 25.31 24.91 24.87

*significant at .05 level

African-American Students

Like the first graders, African-American second-grade students definitely comprise the

largest subgroup of valid test scores – roughly 21% of all SAGE students and 25% of all

comparison students. In the analyses to follow, African-American students are first compared

across SAGE and comparison schools on the CTBS sub-tests and total scale score. Second,

42

African-American students are compared to white students across SAGE and comparison



schools on the CTBS Total Scale Score.

SAGE vs. Comparison. Table 43 provides comparisons of means on the CTBS secondgrade

test, as well as change scores from the first-grade pre-test to the second-grade test and

from the first-grade post-test to the second-grade test. On the second-grade test, African-

American SAGE students scored higher than comparison school students on every sub-test and

on the total scale score. However, the differences between SAGE and comparison students on

the second-grade test scores are not statistically significant. When using the first-grade pre-test

as the baseline score, statistically significant change scores are found on all scores except for

reading. However, using the first-grade post-test as the baseline score shows no statistically

significant differences between SAGE and comparison schools.

43

Table 43. African-American Post-Test and Change Scores, by SAGE or Comparison

SCORE SAGE COMPARISON

Language Arts

Mean Second-Grade Score 591.09 587.19

Mean Change From First-Grade

Pre-Test to Second Grade

72.41* 62.26

Mean Change From First-Grade

Post-Test to Second Grade

19.41 20.62



Reading

Mean Second-Grade Score 594.54 543.48

Mean Change From First-Grade

Pre-Test to Second Grade

73.07 68.48

Mean Change From First-Grade

Post-Test to Second Grade

19.30 20.91



Mathematics

Mean Second-Grade Score 545.44 543.94

Mean Change From First-Grade

Pre-Test to Second Grade

72.59* 61.08

Mean Change From First-Grade

Post-Test to Second Grade

16.46 20.58



Total

Mean Second-Grade Score 577.04 575.94

Mean Change From First-Grade

Pre-Test to Second Grade

72.44* 65.44

Mean Change From First-Grade

Post-Test to Second Grade

17.99 21.14

*significant at .05 level

African-American Males. Table 44 further distinguishes African-American SAGE and

comparison school students by gender. These results show no significant gender-related gain

differences when using either the first-grade pre-test or the first-grade post-test as a baseline

measure.

44

Table 44. African-American Post-Test and Change Scores by Gender



COMPARISON SAGE

Male Female Male Female



Language Arts

Mean Post-Test Scale Score 579.63 591.49 590.74 592.22

Change (Pre-Test as Baseline) 62.03 62.52 73.74 72.09

Change (Post-Test as Baseline) 23.97 18.31 19.48 19.35



Reading

Mean Post-Test Scale Score 584.84 596.65 590.50 598.16

Change (Pre-Test as Baseline) 66.23 69.74 71.13 74.34

Change (Post-Test as Baseline) 25.24 17.92 13.62 24.21



Mathematics

Mean Post-Test Scale Score 538.82 547.33 550.29 540.34

Change (Pre-Test as Baseline) 55.68 65.23 74.99 69.35

Change (Post-Test as Baseline) 21.91 25.46 21.65 30.79



Total

Mean Post-Test Scale Score 568.54 580.32 577.36 576.87

Change (Pre-Test as Baseline) 62.09 67.74 73.17 72.11

Change (Post-Test as Baseline) 22.24 20.36 16.60 19.21

*significant at .05 level

African-American and White Achievement. African-American students scored lower

than white students on the first-grade pre-test total scale score, as shown in table 45. This result

is statistically significant for both SAGE and comparison schools, though the gap between

African Americans and whites is larger in the SAGE schools. The change from first-grade posttest

to the second-grade test shows that the SAGE African Americans kept pace with white

students, but did not further close the achievement gap in second grade.

Table 45. African-American versus White Achievement on Total Scale

First-Grade

Pre-Test

First-Grade

Post-Test

Second

Grade

Change From

Pre to Second

Change From

Post to Second

SAGE

African


American

504.09 559.02 577.04 72.44 17.99

White 533.21 582.56 606.81 73.61 24.50

F 142.06* 94.70* 132.69* .291 8.46*



Comparison

African


American

510.22 553.48 575.94 65.44 21.14

White 528.79 578.28 602.67 73.60 23.80

F 32.24* 61.80* 66.28* 7.46* .925

*significant at .05 level

45

Hierarchical Linear Modeling



Hierarchical linear models were used to evaluate the second-grade achievement results

using the same series of models used to assess first-grade results. Two sets of analyses were

done. The first, shown in Table 45 used first-grade pre-test as the initial achievement level of the

students. The second, shown in Table 46, used first-grade post-test as the initial achievement

level of the students.

HLM Results. Tables 46 and 47 provide a summary of the effects of each of the level-1

and level-2 variables for each of these analyses. Level-1 effects can be interpreted as the

weighted average of the within classroom effects of the level-1 variables. Level-2 effects can be

interpreted as the classroom effects of the level-2 variables. Level-1 coefficients may be thought

of as the average effect of the modeling variable on the criterion score at the individual level.

The level-1 results indicate that lower SES is related to lower post-test scores and higher pre-test

scores are related to higher post-test scores.

The coefficients associated with the level-2 variables can be thought of as classroom

effects. For example, in the Model A total score, an increase of one student in class size resulted

in a drop of .764 points for the class average. Likewise, SAGE participation resulted in a 6.322

point gain in the class average on total score for Model B. A discussion of each model follows.



Model A. Depending on the test, an increase in class size of one person can be expected

to produce a .66 to 1.01 loss in average post-test performance. The results for all scores show

this effect to be significant. It is noteworthy that the results of Tables 41 and 42 for the Model A

are similar, indicating that the second-grade class size has little effect on the two year gain (first

grade and second grade) versus the one year gain (second grade only).

Model B. Participation in SAGE shows no statistically significant class average

increases in all post-test scores with the exception of mathematics in the pre-test condition. A

46

comparison of these results with the first-grade only results indicates second-grade SAGE effects



of lesser magnitude than the first-grade effects. These results tend to support the hypothesis that

SAGE is not continuing to produce the same advantage through second grade. The advantage of

SAGE students is, however, maintained.

Model C. Combining class size and SAGE participation in a single analysis isolates the

effects that SAGE might have beyond those produced by lower class size. The results show that

once class size has been accounted for, SAGE has no significant effect on class average

performance.



Model D. Since socioeconomic status is known to have an influence on academic test

scores, a replacement for this variable was used as both a level-1 and level-2 predictor. The

level-2 variable was the average SES for the class and estimates the effect of the overall class

SES level beyond that associated with the individual, which is accounted for in the level-1

model. This model combines class SES and class size. The results indicate that class SES has a

significant effect on the class average post-test performance. The effect of a 1 point class

average gain in SES equates to between a 10 point and 13 point gain on the average post-test

score, depending on the test. SES was measured on a three-point family income scale; thus a one

point difference on average would be quite pronounced. Class size still has a significant effect

on the post-test scores once SES has been accounted for.



Model E. When class SES and SAGE participation are entered in the same level-2

model, class SES has a significant effect on class average post-test performance. However, for

the most part, SAGE has no significant effect on class average post-test performance once class

SES is controlled. In other words, the effects of SAGE participation on class average post-test

scores, beyond those produced by SES differences, are not significant on post-test scores. The

single exception is for the mathematics score across the two-year time span. In general, the

47

SAGE effects are roughly the same as when SAGE is the only variable in the model (see Model



B), suggesting that SAGE classrooms and comparison classrooms are about equal on class SES.

Model F. This model combines SES, SAGE participation, and class size in a single

analysis. For most sub-tests, class SES once again has a significant effect on the class average

post-test score. Again, class size has a significant effect on the class average post-test score for

all scores except language. Finally, SAGE has no significant effect on any sub-test, once class

size is accounted for.

Table 46. HLM Results for 1997-98 Second-Grade Students – Pre-Test as Initial Achievement

Source Total Reading Language Arts Mathematics

Level 1

Pre-Test 0.703 0.440 0.509 .662

SES -1.929 -4.503 -1.069 -3.196

Level 2

A. Class Size -0.764* -0.630* -0.738* -1.007*

B. SAGE 6.322* 3.302* 7.354* 8.612*

C. Class Size -0.840* -0.886* -0.703 -1.073*

SAGE 1.533 5.152 0.731 1.372

D. Class SES -9.129* -12.214* -7.932 -6.661

Class Size 0.678* -0.523* -0.666* 0.946*

E. Class SES -10.232* -13.214* -8.944* -8.251*

SAGE 5.939 3.024 7.042 8.392*

F. Class SES -9.086* -12.018* -8.017 -6.613

Class Size -0.714* -0.715* -0.593 -0.979*

SAGE 0.708 3.825 -1.487 0.681

*significant at .05 level

Table 47. HLM Results for 1997-98 Second-Grade Students – Post Test as Initial Achievement

Source Total Reading Language Arts Mathematics

Level 1

Pre-Test 0.750 0.506 0.551 .574

SES -2.742 -4.142 -4.640 -4.703

Level 2

A. Class Size -0.811* -0.669* -0.732* -0.959*

B. SAGE 6.296 3.651 7.432 7.741

C. Class Size -0.916* -0.916* -0.683 -1.038*

SAGE 2.177 5.043 1.004 1.622

D. Class SES -9.106* -12.888* -9.069* -5.906*

Class Size 0.726* -0.553* -0.646* 0.911*

E. Class SES -10.339* -13.836* -10.079* -7.254

SAGE 6.007 3.300 7.040 7.616

F. Class SES -9.023* -12.637* -9.172* -5.835*

Class Size -0.787* -.739* -0.556 -0.962*

SAGE 1.243 3.737 1.825 1.029

*significant at .05 level

48

Effects Within SAGE Classrooms

It is of some interest to examine achievement gains in light of various classroom and

teacher characteristics, teacher behaviors and student behaviors. Except for the “Teacher

Experience and Achievement” analyses, data for these questions were collected only for SAGE

first-grade classrooms.

Teacher Experience and Achievement

Student gains (at the classroom level) were correlated with teacher experience. In those

classrooms with more than one teacher, teacher experience was averaged to obtain an appropriate

experience variable for that classroom. Correlations between teacher experience and

achievement gains ranged from .019 to .140, but none were significant for either the first-grade

cohort or the second-grade cohort.

Student Participation and Achievement

The student participation questionnaire factored into two scales as was noted earlier.

These scales were “active learning” and “on-task behavior.” Interest here centered on the

relationship between these variables and the classroom characteristics of class size and teacher

experience as well as achievement gains. Composite active learning and on-task variables were

formed for each classroom by averaging over the administrations and across students in each

class. (Note, there were no significant changes in either of these variables across time). These

variables were then correlated with class size, teacher experience and achievement gain in each

of the three achievement sub-score areas. The on-task variable showed no significant

relationships with any of the classroom variables. For the 1996-97 first-grade cohort, active

learning was significantly correlated with class size (r=-.204; p<.05). For the 1997-98 first-grade

cohort, active learning correlated significantly with teacher experience (r=.307; p<.05). Since

these correlations did not replicate across cohorts, little practical significance should be placed in

49

these findings.



Proximity to the Curriculum and Achievement

From the teacher questionnaire, variables associated with proximity to the curriculum

were constructed. The only significant correlation was the language arts proximity score with

reading gain (r=.270) for the 1997-98 first-grade cohort. Again, since these results did not

replicate across cohorts, little practical significance should be placed in this finding.

Class Organization and Achievement

SAGE students are organized into different types of classrooms as discussed earlier in the

report. These classrooms include 15:1 Regular classrooms, 30:2 2-Team Teaching classrooms,

15:1 Shared-Space classrooms, and 30:2 Floating Teacher classrooms. Available data consisted

of fifty-nine 15:1 classrooms, and thirty-one 30:2 classrooms. Neither the 15:1 Shared Space

nor the 30:2 Floating Teacher class organization contained enough classrooms to analyze each of

these four types separately. This analysis uses Hierarchical Linear Modeling to make this

comparison. The HLM results show no statistically significant relationship between type of

classroom organization and achievement on any of the sub-tests or the total score.

50

ANALYSES OF SAGE TEACHERS, CLASSROOMS, AND SCHOOLS 1997-98


1   2   3   4   5   6   7   8


The database is protected by copyright ©essaydocs.org 2016
send message

    Main page