Student participation in the classroom was measured by a Student Participation Questionnaire. SAGE kindergarten and first grade teachers completed the questionnaire for each of their students in both fall 1996 and spring 1997.
The Student Participation Questionnaire consists of 16 items, measured on a 1 to 5 scale, designed to assess student behavior and participation in SAGE classrooms (see Table 43 for a list of questions). Following principal component analysis (see Table 43), 15 of the 16 questions were combined into two additive scales. As shown in Table 44, one scale comprises 9 questions and measures the extent to which student behavior is "On Task." A second scale comprises 6 questions and measures the extent to which students are engaged in "Active Learning." Further analysis shows the On Task and Active Learning scales to be highly reliable indicators (as measured by the alpha coefficients shown in Table 44).
Student participation in the classroom represents both process and product. The
questionnaire measures the extent to which individual students are engaged, or actively participate in the learning process. But one of the ways in which a smaller student-teacher ratio could raise academic achievement is by first increasing students' level of participation in the learning process. Student participation represents an intervening measure between reduced student-teacher ratios and academic achievement; increasing student participation is an achievement in itself.
Student participation has indeed increased over the course of the first year of the SAGE program. Descriptive statistics on the fall 1996 and spring 1997 student participation scales are provided in Table 45 (because the two participation scales comprise a different number of items, the scores were transformed to a common scale ranging from 0-100). The mean On Task student participation score increased by 8.9 percent from fall 1996 to spring 1997; the mean Active Learning student participation score increased by 12 percent from fall 1996 to spring 1997.
Perfect (maximum) scores on the participation scale also increased from fall to spring. In fall, 143 (4.5 percent) SAGE students attained perfect scores in Active Learning, and 114 (3.6 percent) had perfect scores in On Task behavior. In spring, 345 (12.0 percent) students had maximum scores in Active Learning, and 201 (7.0 percent) attained perfect scores in On Task Behavior.
The Active Learning and On Task behavior scales were entered into OLS regression models (first grade only) to test their effect on achievement on the CTBS (see Tables 46-49).
After controlling for pre-test scores, attendance, eligibility for subsidized lunch, and race/ethnicity, both Active Learning and On Task behavior emerge as significant predictors of achievement on all sub-tests and total score on the CTBS. Active Learning had the largest effect on reading scale scores (b=.46). An increase of a single point on the Active Learning scale (on a 1-100 scale) predicts nearly a half a point increase in the reading scale score. The largest effect of On Task behavior is found for reading and mathematics (b=.37 for both).
Since the Student Participation Questionnaire was not administered to comparison school students the impact of SAGE on increasing student participation cannot be determined. Some increase in participation may be expected to result naturally as a function of a child's maturation. Further research and analysis of the antecedents and covariates of student participation is necessary.
Taken together the teacher interviews, classroom observations, teacher activity logs, and the teacher questionnaires provide a picture of teaching and learning in a 15:1 student-teacher ratio classroom. What emerges after one year of the SAGE program is a classroom where discipline problems and classroom management are greatly reduced, and when classroom management is needed, it is overwhelmingly positive. The direct beneficiary of this reduced time spent on managing the class is increased time spent on instruction, i.e., on actually teaching.
Further, the increased instructional time that is now available to teachers is used to attend to the learning needs of individual students. As much as half the time in classrooms teachers are helping individual students with difficulties they encounter or extending their learning beyond minimal competencies, checking their work and monitoring progress, and providing opportunities for them to become actively involved in learning by articulating current understandings and receiving feedback. The increased instructional time also permits greater emphasis on quantity and depth of reading-language arts and mathematics content. Well over 50 percent of the instructional time is specifically devoted to these areas.
The type of instruction that students encounter in SAGE classrooms is predominantly teacher-centered. Listening, practicing, receiving help, and answering account for between 50 to 75 percent of the teaching-learning that occurs. Although teachers indicated that their use of more student-centered activities such as creating, manipulating, and problem solving increased because of reduced class size, and there is evidence of use of these behaviors from both the observations and logs, student-centered teaching only plays a supplemental role in most SAGE classrooms.
Several pictures do not emerge from a composite of the interviews, observations, logs, and questionnaires. Although all teachers said some changes had taken place in their teaching during the first year of SAGE and observational and self-report log data substantiated these changes, a major change in classroom events from October to May was not observed. A possible explanation might be that the October data, rather than representing a baseline, show adjustments that teachers had already made in their teaching. That is, they quickly made changes in their teaching during September and part of October prior to initial data collection. These changes may have been fine-tuned throughout the year, but the first-year response to reduced class size in rudimentary form may have already been established by October when the first observation occurred.
Another picture that does not emerge is a large swing to student-centered teaching, a change that some might expect as an result of reduced class size. Most SAGE teachers appear to have made real, substantial changes in their teaching. However, these changes are not of the magnitude of substituting one set of coherent practices for another. Instead, SAGE teachers appear to have enriched their teaching and student learning with student-centered teaching, but the major change they have made is to use their teacher-centered teaching with individuals, a change that fewer students has permitted by reducing the need for management and increasing instructional time.
Still another picture that has not emerged is a clear difference among the four main types of SAGE classrooms. They differ in isolated behaviors and events, but generally they appear to reflect the patterns found for the total group of SAGE teachers. Differences among classroom types, however, as well as differences from the beginning to the end of the year in general and in relation to type of teaching may emerge over time. The present data represent only one year of the five-year SAGE program. Longitudinal data to be collected during the next four years are needed to bring all of the pictures of classroom life into sharper focus.
IV. OTHER SAGE INTERVENTIONS
During the first year of the SAGE program the focus of participating schools has clearly been on implementing the reduced student-teacher ratio. The other SAGE interventions -- rigorous curriculum, staff development, and lighted schoolhouse programs -- have been attended to by SAGE schools in varying degrees. This section of the report briefly describes the state of the other SAGE interventions near the end of the first year of the SAGE program.
The Teacher Questionnaire and Principal Interviews, both completed in May 1996, are the sources of data regarding rigorous curriculum.
The Teacher Questionnaire contains a section on classroom curriculum designed to determine the congruence of SAGE classroom curricula with professional curriculum standards developed by the International Reading Association (IRA), the National Council for Teachers of English (NCTE), and the National Council for Teachers of Mathematics (NCTM).
Teachers were asked to indicate, on a five point scale, the extent to which items on the questionnaire described the curriculum in their classrooms. A classroom’s curriculum would be, according to the teacher’s self-report, considered perfectly congruent with the professional curriculum standards of the IRA, NCTM, and NCTE if the teacher responded “5" on all curriculum items on the Teacher Questionnaire.
A total of 211 SAGE teachers completed the curriculum section (K=88, 1st=109, mixed grade=4). The responses of each teacher yielded mean scores for reading/language arts and for mathematics. As can be seen in Table 50 the mean scores for reading/language arts ranged from 2.11 to 4.84 with an overall average score of 3.88 (78 percent of perfect congruence). Mean scores for mathematics items ranged from 1.62 to 4.83 with an overall average score of 3.61 (72 percent of perfect congruence).
The responses of first grade and mixed grade teachers indicates that their reading/language arts curricula are more congruent with professional standards than SAGE kindergarten curricula. The responses of kindergarten, first grade and mixed grade teachers resulted in no significant difference in the degree to which their curricula were congruent with professional standards in the area of math.
It is interesting to note the three reading/language arts items that produced the highest means - and were thus closest to the recommended professional standard. According to SAGE teachers, students in their classrooms were most likely to be: 1) taught to apply a variety of decoding strategies (mean score=4.65), 2) taught the names of the parts of books (mean score=4.74), and 3) encouraged to choose books they are interested in reading (mean score=4.78). In mathematics the most notable divergence from the pattern mean responses at the 3 or 4 level was in response to the item on calculator use. This item had a mean response of 2.17 which suggests that calculators do not play much of a roll in SAGE kindergarten and first grade math instruction.
All SAGE principals were interviewed. Their responses to curriculum related questions suggests that, for SAGE principals, a rigorous curriculum includes basic skills, problem solving, and higher level thinking. Only a handful of principals seemed to believe that the curriculum of their school was rigorous. However, most SAGE principals regarded parts of their curriculum as strong.
The section of the Teacher Questionnaire that dealt with staff development asked teachers about their individual level of professional development as well as the extent to which their school district provides staff development programs. The questions were derived from standards for staff development published by the National Staff Development Council, in cooperation with the National Association of Elementary School principals. With respect to individual professional development, the SAGE contract requires that teachers and administrators develop professional development plans that focus on how they will help improve student academic achievement.
Despite of the contractual requirements, roughly percent responded that they had no "personal, formal, written development plan.” As can be seen in Table 51, those teachers who had individual plans were more likely to engage in activities aimed at professional development. Teachers were asked if, over the past school year, they had participated in any of twelve activities aimed at further developing their teaching skills. Of the twelve professional development activities, four were found to be significantly related to existence of an individual professional development plan. Teachers who reported that they had a formal individual development plan were more likely to have 1) collaborated with other schools and institutions, 2) conducted research connected to teaching, 3) attended a professional conference or skill building workshop, and 4) taken a course for graduate or CEU credit. Other questions concerned the extent to which their school district had a staff development program. Using categories established by the National Staff Development Council, teachers were asked to identify the stage of their district's program. Twenty one percent answered that their district was at the "initialization" phase, 66.4 percent responded that they were in the "implementation" stage, while 9.3 percent felt that their school district was currently "institutionalizing" their staff development program. Similar statistical analyses were conducted to determine the relationship between the presence of a school directed staff development program and professional development activities. Teachers who felt that their school district was either in the "implementation” or institutionalization” phase were significantly more likely to have participated in eight of the twelve personal development activities during the past school year, as shown in Table 52.
The professional development activities that were most highly related were those that occurred within the framework of the district's level of development. More simply, those school districts that were further along in implementing a staff development program were more likely to have their teachers participate in such activities as teacher-to-teacher mentoring programs, joint planning, collaborative teaching, collaborative evaluation of student progress, and school-wide instructional initiatives or themes. This finding indicates that individual professional development plans and school district staff development programs may differ in the impact that they have on individual actions aimed at professional development.
Individual, formal professional development plans seem to be related to those activities that are most directly controlled by the individual independent of the school district's commitment to professional staff development. For example, a teacher who has a formal professional development plan in place may be more likely to conduct research related to his or her teaching in spite of the fact that the school district may not have a well-developed staff development program. Additionally, the commitment of the school district toward staff development relates most directly to those activities that occur within the boundaries of everyday school life. In other words, staff development programs seem to correlate most highly with those activities that are directly under the district's control.
Data regarding implementation of lighted schoolhouse activities were obtained from the Principal Interviews and year end reports required by DPI. In addition, data regarding lighted schoolhouse activities existing prior to SAGE were obtained from the Baseline Data Questionnaire administered in May, 1996, and the school contracts completed for DPI prior to enrollment in the SAGE program.
Most schools have continued the activities they offered in previous school years. Principal Interview data suggest that SAGE schools have taken responsibility for the conception and operation of the lighted schoolhouse activities (as opposed to activities initiated by parents or community volunteers). However, they have not tended to focus heavily on their lighted schoolhouse activities in the first year of SAGE implementation, Some SAGE schools have experienced financial and transportation difficulties as a result of their lighted schoolhouse activities.
Regarding the differences between the pre-SAGE and SAGE years, Table 53 shows that of the thirty SAGE schools, 20 reported an increase in participation in lighted schoolhouse activities, while 3 reported a decrease in participation. Change related to the remaining 7 schools could not be determined due to incomplete or inaccurate information.
1 Distance (X, Y)= S (Xi - Yi )2
2 The SAGE evaluation team selected, a priori, .05 as the critical threshold for all tests of statistical significance.
3 See the Evaluation Design Plan for the Student Achievement Guarantee in Education (SAGE) Program.
4 CTB/McGraw-Hill uses a three parameter logistic model to create scale scores.The total scale score is computed by CTB/McGraw-Hill as the average of the three scale scores from reading, language arts and mathematics.
5 Additional models were tested including interaction terms for SAGE student by race/ethnicity. The interaction terms were not statistically significant.
6 It is worth noting that African American SAGE students are disproportionately found in large, two
teacher classes. The implications of this will become clearer in the next section, which uses a class size variable in hierarchical linear analysis.
Bryk, Anthony S. and Stephen W. Raudenbusch. 1992. Hierarchical Linear Models (Newbury
Park, CA: Sage).
Finn, Jermy D. and Charles M. Achilles. 1990. "Answers and Questions About Class Size: A
Statewide Experiment," American Educational Research Journal, Fall, Vol. 27 (3): 557-577.
Linn, Robert L. 1983. "Pearson Selection Formulas: Implications for Studies of Predictive Bias
and Estimates of Educational Effects in Selected Samples," Journal of Educational Measurement,
Spring, Vol. 20 (1): 1-15.