The student achievement guarantee



Download 0.91 Mb.
Page3/8
Date conversion15.02.2016
Size0.91 Mb.
1   2   3   4   5   6   7   8

Table 19. SCALE SCORES Regression for Language Arts for 1997-98 First-Grade Students

Block 1 Block 2 Block 3

Variable b t b t b t

Pre-Test Score .56 24.87* .55 24.18* .55 23.94*

Days Absent -.37 -3.31* -.35 -3.06* -.33 -2.90*

Subsidized Lunch Eligibility -2.63 -2.46* -1.95 -1.72 -2.06 -1.82

African American 2.78 1.02 2.50 .92

White 5.60 2.37* 5.80 2.47*

SAGE 7.25 3.64*

Constant 292.26 23.33* 291.59 23.06* 304.36 23.28*

Adjusted R Squared .33 .33 .34

Standard Error of Estimate 36.49 36.45 36.30

*significant at .05 level



Table 20. SCALE SCORES Regression for Reading for 1997-98 First-Grade Students

Block 1 Block 2 Block 3

Variable b t b t b t

Pre-Test Score .56 20.88* .55 20.33* .54 20.20*

Days Absent -.21 -1.96* -.18 -1.60 -.16 -1.43

Subsidized Lunch Eligibility -3.01 -2.85* -2.06 -1.85 -2.15 -1.94*

African American 3.07 1.15 2.80 1.05

White 7.21 3.14* 7.37 3.22*

SAGE 6.98 3.59*

Constant 285.03 19.25* 284.30 19.08* 296.11 19.48*

Adjusted R Squared .26 .27 .27

Standard Error of Estimate 35.73 35.64 35.50

*significant at .05 level



Table 21. SCALE SCORES Regression for Mathematics for 1997-98 First-Grade Students

Block 1 Block 2 Block 3

Variable b t b t b t

Pre-Test Score .62 30.65* .62 29.21 .61 29.13*

Days Absent -.21 -1.96* -.18 -1.60 -2.15 -1.94*

Subsidized Lunch Eligibility -2.75 -3.08* -2.44 -2.62* -2.56 -2.76*

African American .1.66 .74 1.44 .64

White 3.05 1.56 3.24 1.66

SAGE 7.06 4.31*

Constant 235.05 22.32* 235.69 21.88* 247.05 22.40

Adjusted R Squared .43 .43 .44

Standard Error of Estimate 30.05 30.05 29.88

*significant at .05 level

28

Table 22. SCALE SCORES Regression for Total for 1997-98 First-Grade Students

Block 1 Block 2 Block 3

Variable b t b t b t

Pre-Test Score .77 38.80* .77 37.58* .76 37.46*

Days Absent -.34 -3.08* -2.44 -2.62* -2.56 -2.76*

Subsidized Lunch Eligibility -.63 -.86 -.53 -.68 -.62 -.81

African American 3.91 2.1* 3.72 2.01*

White 3.00 1.86 3.20 2.00*

SAGE 6.33 4.68*

Constant 167.12 15.51* 165.11 15.04* 176.57 15.80*

Adjusted R Squared .54 .54 .55

Standard Error of Estimate 24.53 24.51 24.34

*significant at .05 level

African-American Students

Among minority students in SAGE and comparison schools, African Americans clearly

comprise the largest group of valid test scores – roughly 25% percent of SAGE students and 28%

percent of comparison school students. In the analyses to follow, African-American students are

first compared across SAGE and comparison schools on CTBS sub-test and total scale scores.

Second, African-American students are compared to white students across SAGE and

comparison schools on CTBS total scale scores.

SAGE vs. Comparison. Table 23 provides comparisons of means on the CTBS post-test,

and change scores from pre-test to post-test. On the post-test, African-American SAGE students

scored higher than African-American comparison school students on every sub-test and on total

scale score. The differences between SAGE and comparison schools on post-test scores are all

statistically significant. In addition, the differences between SAGE and comparison schools on

mean change scores from pre-test to post-test scores are statistically significant. In other words,

African-American SAGE students scored lower on the CTBS pre-test than African-American

comparison school students, but made significantly larger gains than comparison school students

from pre- to post-test, and surpassed African-American comparison school students on the posttest.

29

Table 23. African American Post-Test and Change Scores, by SAGE or Comparison School for

1997-98 First-Grade Students



SCALE SCORE SAGE COMPARISON F

Language Arts

Mean Post-Test 572.80 558.32 11.59*

Mean Change Pre to Post 56.05 38.27 16.41*

Reading

Mean Post-Test 573.82 554.11 25.31*

Mean Change Pre to Post 50.55 25.79 31.67*

Mathematics

Mean Post-Test 522.01 506.22 20.74*

Mean Change Pre to Post 49.06 27.50 41.99*

Total

Mean Post Test 556.72 539.73 25.48*

Mean Change, Pre to Post 52.15 32.78 43.51*

*significant at .05 level

African-American Males. Concern over the minority achievement gap on standardized

tests has occasionally focused on African-American male students. Table 24 further

distinguishes African-American SAGE and comparison school students by gender. The 1996-97

results showed that African-American male SAGE students attained comparable or higher

change scores from pre-test to post-test when compared to African-American female SAGE

students. The 1997-98 results show that African-American male SAGE students attained

comparable or higher change scores from pre-test to post-test on the language arts sub-test, the

mathematics sub-test, and the total score. However, none of these results is statistically

significant.

Table 24. African-American Post-Test and Change Scores by Gender

COMPARISON SAGE

Male Female Male Female



Language Arts

Mean Post-Test Scale Score 554.74 562.18 570.99 574.27

Mean Change Pre to Post 33.96 42.69 58.64 53.65

Reading

Mean Post-Test Scale Score 556.06 552.25 570.40 576.77

Mean Change Pre to Post 35.75 15.19 50.37 50.49

Mathematics

Mean Post-Test Scale Score 511.48 501.24 522.82 520.93

Mean Change Pre to Post 28.90 26.08 53.37 45.01

Total

Mean Post-Test Scale Score 540.94 538.70 554.85 558.23

Mean Change Pre to Post 34.78 30.65 53.47 50.93

*significant at .05 level

30

African-American and White Achievement. African-American students scored



significantly lower than white students on the CTBS pre-test total scale score, as shown in table

25. This result holds for both SAGE and comparison schools, though the gap between African

Americans and whites is larger in SAGE schools. When all cases are analyzed, African-

American SAGE students achieved greater gains on the CTBS total scale score than white SAGE

students from pre- to post-test, closing the achievement gap (though the gap remains statistically

significant). In contrast, African Americans in comparison schools achieved lesser gains and in

the comparison schools the achievement gap widened.

Table 25. African-American and White Achievement in SAGE and Comparison Schools

on Total Scale Scores for 1997-98 First-Grade Students



PRE-TEST POST-TEST CHANGE

SAGE SCHOOLS

African American 502.79 556.72 52.15

White 531.38 579.94 45.99

F 170.61* 96.09* 10.50*

COMPARISON SCHOOLS

African American 510.07 539.73 32.78

White 528.60 569.02 41.14

F 52.21 90.15* 10.72*

*significant at .05 level

Hierarchical Linear Modeling

Explanation. Many social science research analyses involve hierarchical data structures.

Hierarchical data structures are those in which individual units are nested within larger units, the

latter being the unit of interest. The SAGE data are a prime example: students are nested within

classrooms, and it is the classroom effect that is of particular interest to the SAGE project.

Hierarchical data structures pose special analytical challenges in that data analysis at the

individual level may result in a biased impression of the effect of the nesting unit (in the SAGE

case, the classroom). At the origin of this problem is the fact that different classrooms often

contain different numbers of students, thus those classrooms that contain greater numbers of

31

students have greater influence over the results of analyses done at the individual level. An



analytical approach known as “hierarchical linear modeling” (Bryk & Raudenbush, 1992) was

specifically designed to accommodate these types of data structures. Essentially hierarchical

linear modeling (HLM) estimates individual effects by analyzing data within each class and then

provides a weighted average of these effects. The effects of the class are then estimated as if all

classes contained the same number of students. HLM was used with the SAGE data to provide

an alternative and less biased account of the effects of SAGE experience on test scores. In these

models, variables associated with individual students are referred to as level-1 variables and

those associated with the classrooms are referred to as level-2 variables.

HLM Analyses. Analyses were conducted for each of the relevant criterion post-test

scores: reading, mathematics, language arts, and total. For all analyses, the level-1 variables

were pre-test scores and socioeconomic status (SES) measured as eligibility for subsidized lunch.

The post-test scores were adjusted for these two variables at the individual level, therefore the

effects may be thought of as being statistically independent of the effects of these variables. A

number of different level-2 models, each containing different level-2 variables, was specified for

each variable of interest. It is important to note that the “class size” variable used in these

analyses measures the student-teacher ratio.

HLM Results. Table 26 provides a summary of the effects of each of the level-1 and

level-2 variables for each of these analyses. Level-1 effects can be interpreted as the weighted

average of the within-classroom effects of the level-1 variables. Level-2 effects can be

interpreted as the classroom effects of the level-2 variables. Level-1 coefficients may be thought

of as the average effect of the modeling variable on the criterion score at the individual level.

The level-1 results indicate that lower SES is related to lower post-test scores and higher pre-test

scores are related to higher post-test scores.

32

The coefficients associated with the level-2 variables can be thought of as classroom



effects. For example, in the Model A total score, an increase of one student in class size resulted

in a drop of .828 points for the class average. Likewise, SAGE participation resulted in a 8.909

point gain in the class average on total score for Model B. A discussion of each model follows:

Model A. Class Size. These models examined the effect of class size on the adjusted

criterion score. Class size equals the number of students divided by the number of teachers.

Depending on the test, an increase in class size of one person can be expected to produce a .29 to

1.12 loss in average post-test performance. The results for all scores show this effect to be

significant.

Model B. SAGE. These models examined the effect of SAGE participation on the

adjusted criterion score. Participation in SAGE shows statistically significant class average

increases in all post-test scores as well. These score increases range from 7 points (reading) to

13 points (mathematics).



Model C. Class Size, SAGE. These models examined the effect of SAGE participation

on the adjusted criterion score after the classrooms were class size adjusted, viewed as the effect

of SAGE participation beyond the class size effect. Combining class size and SAGE

participation in a single analysis isolates the effects that SAGE might have beyond those

produced by lower class size. The results show that once class size has been accounted for,

SAGE has no significant effect on class average performance. This may suggest that the other

SAGE interventions (i.e., rigorous curriculum, lighted school house, and staff development) are

not having a significant impact on achievement in SAGE classrooms.



Model D. Class SES, Class Size. These models examined the effect of class size on the

adjusted criterion score after the classrooms were SES adjusted, viewed as the effect of class size

once the effects of the classroom SES are removed. Since socioeconomic status is known to

33

have an influence on academic test scores, a replacement for this variable was used as both a



level-1 and level-2 predictor. The level-2 variable was the average SES for the class and

estimates the effect of the overall class SES level beyond that associated with the individual,

which is accounted for in the level-1 model. This model combines class SES and class size. The

results indicate that class SES has a significant effect on the class average post-test performance.

The effect of a 1 point class average gain in SES equates to between a 10 point and 13 point gain

on the average post-test score, depending on the test. SES was measured on a three-point family

income scale, thus a one point difference on average would be quite pronounced. Class size still

has a significant effect on the post-test scores once SES has been accounted for.



Model E. Class SES, SAGE. These models examined the effect of SAGE participation

on the adjusted criterion score after the classrooms were SES adjusted; viewed as the effect of

SAGE participation once the effects of classroom SES are removed. When class SES and SAGE

participation are entered in the same level-2 model, class SES has a significant effect on class

average post-test performance. In addition, SAGE has a significant effect on class average posttest

performance. In other words, the effects of SAGE participation on class average post-test

scores, beyond those produced by SES differences, are significant on all post-test scores. In

general, these effects are roughly the same as when SAGE is the only variable in the model (see

model B), suggesting that SAGE classrooms and control classrooms are about equal on class

SES.


Model F. Class SES, Class Size, SAGE. These models examined the effect of SAGE

participation on the adjusted criterion score after the classrooms were adjusted for class size and

SES; viewed as the effect of SAGE participation beyond the class size and SES effects. This

model combines SES, SAGE participation, and class size in a single analysis. For all sub-tests,

class SES once again has a significant effect on the class average post-test score. Class size has

34

no significant effect on the class average post-test score. Finally, SAGE had significant effects



only on the mathematics sub-test.

Table 26. HLM Results for 1997-98 First-Grade Students

Source Total Reading Language Arts Mathematics

Level 1

Pre-Test 0.870 0.627 0.625 0.712

SES -0.784 -3.733 -1.612 -3.202

Level 2

A. Class Size -0.828* -0.289* -0.899* -1.115*

B. SAGE 8.909* 7.009* 10.148* 13.090*

C. Class Size -0.647 -0.734 -0.639 -0.722

SAGE 2.990 0.195 4.228 6.409

D. Class SES -12.959* -10.410* -12.971* -13.389*

Class Size 0.599* -0.574* -0.698* 0.883*

E. Class SES -14.707* -12.215* -15.201* -16.298*

SAGE 9.354* 7.320* 10.661* 13.428*

F. Class SES -14.883* -11.446* -15.168* -16.211*

Class Size -0.015 -.252 -0.011 -0.027

SAGE 9.074 4.957 10.556 13.172*

*significant at .05 level

Second-Grade Results 1997-98

Descriptive Statistics

Valid Test Scores. Analyses were conducted to assess the impact of SAGE on the 1997-

98 second-grade CTBS Complete Battery, Terra Nova Level 13 post-test results. There were

1702 persisting students (i.e., students present in both the 1996-97 SAGE and comparison firstgrade

classrooms and in the 1997-98 SAGE and comparison second-grade classrooms), while

there were 482 new second-grade students (students who were not in the program last year).

However, second-grade post-test results are compared to the first-grade pre-test, as well as first

grade post-test. Therefore, only those students who took both the first-grade pre-test and posttest,

as well as the second-grade post-test, were used in the 1997-98 second-grade analysis. As

would be expected, the number of second-grade students having all three valid test scores was

substantially less than the total number of students. The number of valid test scores for the

Fal1996 first-grade pre-test, the Spring 1997 first-grade post-test, and the Spring 1998 second-

35

grade post-test are presented in Table 27.



Table 27. Number of 1997-98 Second-Grade Students with Valid Test Scores

SAGE Comparison Total

Fall 1996 First-

Grade Pre-test

Reading 1033 562 1595

Language Arts 1033 562 1595

Mathematics 1020 559 1579

Total 1008 448 1456

Spring 1997 First-

Grade Post-Test

Reading 1011 545 1556

Language Arts 1011 545 1556

Mathematics 1007 538 1545

Total 1001 534 1535

Spring 1997

Second Grade

Reading 1037 561 1598

Language Arts 1037 562 1599

Mathematics 1043 559 1602

Total 1033 549 1582

Pre-Test (Baseline) Results. Both the first-grade pre-test and the first-grade post-test

served as a baseline. Table 28 provides descriptive statistics on the scale scores from the firstgrade

pre-test as well as the first-grade post-test.



Table 28. Descriptive Statistics on CTBS First-Grade Pre-Test and Post-Test (SAGE and

Comparison)



FIRST-GRADE PRE-TEST FIRST-GRADE POST-TEST

SCALE

SCORES

NORMAL CURVE

EQUIVALENT

SCALE

SCORES

NORMAL CURVE

EQUIVALENT

Reading 535.20 36.83 45.31 19.78 584.17 35.65 54.44 18.50

Language

Arts


532.70 42.03 45.03 20.74 583.05 37.78 54.53 17.89

Mathematics 494.55 38.27 44.11 18.05 546.59 41.56 55.88 20.41

Total 521.03 33.34 44.33 18.28 571.43 31.94 55.65 17.81

Difference of Means Test. The results from the difference of means tests between SAGE

and comparison student scale scores from the Fall 1996 first-grade pre-test and Spring 1997 firstgrade

post-test are reported in Tables 29-32. The differences between SAGE schools and

36

comparison schools on the first-grade pre-test are not found to be statistically significant at the



.05 level. Therefore, any differences between the first-grade pre-test and the second-grade test

can be more confidently attributed to the student-teacher ratio of 15:1 in the SAGE classrooms.

The differences between SAGE schools and comparison schools on the first-grade post-test are

found to be significant on the total score and on all sub-scores. Therefore, any conclusions

discussed regarding second-grade results must take into account the effects of the SAGE

program while these students were in first grade.



Table 29. Differences of Means Test on First-Grade Pre-Test and Post-Test:

Language Arts Scale Scores



FIRST-GRADE

PRE-TEST

FIRST-GRADE

POST-TEST

N MEAN STANDARD

DEVIATION

N MEAN STANDARD

DEVIATION

Comparison Schools 562 530.69 43.09 545 579.01 39.75

SAGE Schools 1033 533.80 41.42 1011 586.07* 36.06

*significant at .05 level



Table 30. Differences of Means Test on First-Grade Pre-Test and Post-Test: Reading Scale

Scores


FIRST-GRADE

PRE-TEST

FIRST-GRADE

POST-TEST

N MEAN STANDARD

DEVIATION

N MEAN STANDARD

DEVIATION

Comparison Schools 562 534.62 38.77 545 582.01 36.50

SAGE Schools 1033 535.52 35.75 1011 586.07* 36.06

*significant at .05 level



Table 31. Differences of Means Test on First-Grade Pre-Test and Post-Test: Mathematics Scale

Scores


FIRST-GRADE

PRE-TEST

FIRST-GRADE

POST-TEST

N MEAN STANDARD

DEVIATION

N MEAN STANDARD

DEVIATION

Comparison Schools 559 493.70 38.26 538 541.88 40.75

SAGE Schools 1020 495.01 38.29 1007 550.67* 41.17

*significant at .05 level

37

Table 32. Differences of Means Test on First-Grade Pre-Test and Post-Test: Total Scale Score

FIRST-GRADE

PRE-TEST

FIRST-GRADE

POST-TEST

N MEAN STANDARD

DEVIATION

N MEAN STANDARD

DEVIATION

Comparison Schools 548 519.96 33.59 534 567.64 32.29

SAGE Schools 1008 521.61 33.20 1001 574.72* 30.97

*significant at .05 level

As noted above, student populations varied in SAGE and comparison schools due to

withdrawals and within-year enrollments. The post-test results are based only on those second

graders who remained in class the entire 1996-97 first grade and 1997-98 second grade school

years.


Results of the difference of means test between SAGE and comparison schools on

the second-grade post-test can be seen in Table 33. Table 34 shows that when the first-grade

pre-test is used as the baseline score, significant results are found on the language arts sub-scale,

mathematics sub-scale, and total score. However, when the first-grade post-test is used as the

baseline score, no significant results are found. This suggests that the statistically significant

positive effects of SAGE occurred in the first grade. These positive effects were maintained, but

did not significantly increase in second grade.

Table 33. Difference of Means Test – Second-Grade Scale Scores

SAGE Schools Comparison Schools

N MEAN STANDARD

DEVIATION

N MEAN STANDARD

DEVIATION

Language Arts 1037 610.91* 41.10 562 602.70 41.38

Reading 1037 608.17 36.11 561 604.63 37.07

Mathematics 1043 572.11* 41.69 559 564.36 39.10

Total 1033 597.14* 34.29 549 591.25 34.10

*significant at .05 level

The largest gain in SAGE student scores from first-grade pre-test to the second-grade

post-test was on the mathematics sub-test, as shown in Table 34. The smallest relative gain for

38

SAGE students from pre-test to post-test was on the reading sub-scale; this gain was not



statistically significant.

1   2   3   4   5   6   7   8


The database is protected by copyright ©essaydocs.org 2016
send message

    Main page