St. Mary’s College of California
The spontaneous gestures exhibited by fourteen undergraduate students taking a mathematics course for prospective elementary school teachers were analyzed, utilizing the work of David McNeill as a starting point. The analysis determined that the gestures fell into four types: iconic-physical, iconic-symbolic, metaphoric, and deictic. McNeill’s hypothesis that mathematical gestures have distinctive, semi-conventional forms received mixed support, with gestures related to specific fractions having the most structure in common. Introduction
Within the field of cognitive science, the pursuit of the understanding of human nature was initially based on certain assumptions. These included the assumption that cognition consists of the manipulation of symbols according to formal rules, that the mind is fundamentally a computational entity, and that the surest path to understanding human thinking is to build computational models that also process symbols according to formal rules. Underlying these assumptions was a more foundational one, inherited from Descartes: that the mind is separable from the body. That is, in order to understand human cognition, the fact that human brains reside in bodies is irrelevant. Equally unimportant is the fact that these bodies have evolved over hundreds of thousands of years, by coping with a range of environmental challenges within certain fixed physical constraints (the existence of gravity, the basic bilateral symmetry of mammalian bodies, etc.). In the “classical” cognitive science view, the senses function only as conduits for bringing “input” to the mind’s programs, and the motor system functions simply to carry out the results of the mind’s processing of information.
During the past two decades, an alternative paradigm has emerged that challenges the Cartesian assumptions of early cognitive science, and that instead acknowledges the embodied nature of cognition (Varela, Thompson & Rosch, 1991). This new paradigm sees the sensorimotor system as an essential aspect of cognition; as Varela states: “Embodied entails the following: (1) cognition dependent upon the kinds of experience that come from having a body with sensorimotor capacities; and (2) individual sensorimotor capacities that are themselves embedded in a more encompassing biological and cultural context...sensory and motor processes, perception and action, are fundamentally inseparable in lived cognition, and not merely contingently linked as input/output pairs” (Varela, 1999, p. 12).
Recently, research into the relationship between physical gesture and language has added a new dimension to the embodied cognition paradigm. According to work in this area, not only is it the case that one must consider the connections between the body, the mind, and the world, but that human gestures form an integral part of language and thought. Indeed, there is one school of thought that holds that gesture preceded and scaffolded speech in human evolution, and evidence from neuroscience indicates that the same areas of the brain are involved in the expressive use of gesture and oral language (Corballis, 1999).
Foundational work on gesture and language has been carried out by psychologist and linguist David McNeill (1992, 2000). McNeill has proposed that spoken language and gesture form an integrated system of communication; that is, that “gesture and speech arise from a single process of utterance formation” (McNeill 1992, p. 29). In contrast to theories that see gesture as in some way epiphenomenal to speech, as embellishments, or even as “translations” of previously conceptualized verbalizations, McNeill holds that speech and gesture are produced in concert in the mind, and act together to express the same intended meaning. McNeill offers several kinds of evidence for this hypothesis, including the neurolinguistic findings described above, but also the fact that gesture and speech are closely synchronized in time and meaning, that gesture occurs only during speech, and that gesture and spoken language develop together in children (ibid.). McNeill points out that spoken language and gesture have different yet complementary characteristics that allow them to synergistically support effective communication of meaning. Speech, on the one hand, is linearly segmented; it takes place one word at a time, with the order of the words playing an important, if not essential, role in the meaning of the utterance. Meaning is broken down into discrete segments, which are organized hierarchically (a discourse, a sentence, a phrase, a word, a phoneme); viewed another way, utterances are constructed by putting together these linearly organized segments, which combine to form more complex structures. In general, language is analytic, with one meaning (or a cluster of related meanings) attached to a given word, and a fairly arbitrary mapping between the form of words and their meanings. In sum, oral language is linearly-segmented, hierarchical, and analytic.
Gesture contrasts with oral language on all these dimensions. Rather than presenting meaning through a linearly segmented stream, gesture is what McNeill calls “global-synthetic”; the meaning comes from the motion as a whole, rather than being built up from simpler parts. In addition, one gesture can convey an entire complex of meanings, including aspects such as spatial location, physical shape, speed, and rhythm that are difficult to convey through words. Gestures are also non-hierarchical and non-combinatoric; that is, more complex gestures are not built up from simpler ones; instead, “With gestures, each symbol is a complete expression of meaning unto itself” (ibid., p. 21).
McNeill points out that that gesture can play a mediating role between internal, subjective imagery, and shared, conventional speech. His central hypothesis is that “Speech and gesture are elements of a single integrated process of utterance formation in which there is a synthesis of opposite modes of thought — global-synthetic and instantaneous imagery with linearly-segmented temporally extended verbalization. Utterances and thoughts realized in them are both imagery and language” (ibid., p. 35). Since mathematics itself has a dual identity comprising both an internal, individual experience based at least in part in imagery, as well as a shared, conventional symbol-based language, gesture has the potential of illuminating its nature within the human conceptual system.
Research on Gesture in Mathematics and Science
In recent years, a body of empirical research into the role of gesture in doing, teaching and learning mathematics has emerged. These studies address topics ranging from the way young children use gesture in counting (Alibali & diRusso, 1999; Graham, 1999); teachers’ gestures when providing instruction in problem-solving (Goldin-Meadow, Kim & Singer, 1999); and how students working together make sense of various kinds of graphs (Moschkovich, 1996; Reynolds & Reeve, 2002). Similar studies have examined gesture in science talk (Crowder, 1996; Roth & Welzel, 2001; see Roth, 2001 for a more thorough review). The results of these studies have suggested that when there is a mismatch between gesture and speech, it can indicate readiness to learn a concept, that students may be able to convey an understanding through gesture before they can do so in words, and that appropriate (matched) gesture can enhance the effectiveness of spoken instruction (summarized in Roth, 2001). Additional research has investigated bodily movement as well as gesture in the context of technology-based tools for understanding physics, mathematics, and the representation of both through graphing (Nemirovsky & Noble 1997; Nemirovsky, Tierney & Wright 1998).
Given that the study of gesture in mathematics is still in its infancy, it seems appropriate at this stage to undertake a descriptive analysis of kinds of gestures observed in use by mathematics learners and teachers. In the study described in this paper, prospective teachers (who are, in a sense, both teachers and learners) were interviewed about fractions. The objective of the work is to describe the type and range of gestures utilized by the students during these interviews, in a sense, to begin to develop a “natural history” of gesture in this particular social and mathematical context. A further goal is to investigate a hypothesis put forward by McNeill in regards to mathematical gestures, specifically, that within a given mathematical domain, gestures may have a shared, semi-conventional structure (McNeill, 1992).
McNeill analyzed a videotape of two mathematicians in conversation, and determined that, unlike other spontaneous gestures, the hand movements that corresponded to various mathematical concepts were similar between the two speakers. In addition, the same gestures were used for the same concepts at different times in the conversation, and a given gesture appeared to correspond to a single concept (rather than representing a complex of concepts or events). For example, when discussing the concept of a mathematical dual, both mathematicians used a gesture in which the hand was rotated between two distinctive positions. The gestures accompanying the word “limit” are described by McNeill as “straight-line trajectories followed by ‘end-marking’ (a tensed stop)” (ibid., p. 166). McNeill stated that similarly distinctive gestures were employed for concepts such as quotients, factoring, and compactness. His characterization of mathematical gesture was that it occupies a space between idiosyncratic, spontaneous gesture and true gesture languages, such as sign language, which share the characteristics of spoken language.
The participants in the research were fourteen prospective elementary school teachers, sophomores at a small liberal arts college. All were women, approximately 20 years of age. The students were enrolled in the first semester of a required two-semester sequence of mathematics courses taught by the author; the content covered in the first semester included number systems and algebra. The participants were volunteers, and received extra credit points for participating. Each pair of participants was interviewed twice by the author, once before and once after the topic of fractions was addressed in class. The interviews lasted approximately 30 minutes, and were videotaped. The participants spent between fifteen to twenty minutes of each interview working together to solve a set of problems involving addition, subtraction, multiplication, division and comparing fractions. In the remaining time, the students answered the following questions, posed orally by the interviewer:
How were you first introduced to the idea of fractions?
Do you remember anything that was particularly difficult about learning
fractions? What about adding, subtracting, multiplying or dividing
Have you ever used fractions in everyday life, or in other classes?
How would you introduce fractions to children?
How would you define a fraction to children?
The current analysis focuses on gestures displayed while answering these questions since, with one exception, the only gestures evident during the problem solving consisted of pointing to the written work. The goals of the analysis were to examine and categorize the variety of gestures associated with the students’ discourse about fractions, and to test the hypothesis that there are distinctive gestures for mathematical objects and processes.
The data analyzed in this report were taken from the six half-hour interviews carried out before the course instruction on fractions (the analysis of the remaining six interviews is in progress). A total of 86 gestures were analyzed. Gestures (also referred to by some researchers as gesticulations) are defined by McNeill as “idiosyncratic spontaneous movements of the hands and arms while accompanying speech” (McNeill, 1992, p. 37). The gestures were classified using a scheme established by McNeill and other researchers (McNeill, 1992, p. 76). This scheme has five basic categories; the terms and definitions below are from McNeill (1992):
Iconic gestures: “bear a close formal relationship to the semantic content of speech” (p. 12)
Metaphoric gestures: “the pictorial content presents an abstract idea rather than a concrete object or event” (p.14)
Beats: “indexes the word or phrase it accompanies as being significant...for its discourse-pragmatic content” (p. 15)
Cohesives: “serve to tie together thematically related but temporally separated parts of the discourse” (p. 16)
Deictics: “pointing movement [that] select a part of the gesture space” (p. 80).
All of the 86 gestures could, in a general sense, be classified into one of three of the above categories: iconic, metaphoric, or deictic. However, the particular discourse context seemed to call for an additional category or subcategory. As noted above, iconics, which refer to concrete objects or events, are distinguished from metaphorics, which refer to “an abstract idea.” In analyzing the gestures used by the students while talking about fractions and fraction operations, there were certain gestures that occurred in association with speech about entities that had both concrete and abstract characteristics, namely, written mathematical algorithms or procedures. When describing how they learned about operations with fractions, a number of students exhibited gestures that I came to think of as “algorithms in the air.” That is, they used their fingers or hands to indicate spatial positions that would correspond to the relative position of different aspects of a written algorithm. For example, in describing learning how to add fractions, one student stated, “I remember learning that you put one under the other.” The still image corresponding to this phrase is shown in Figure 1 (the underlined portion of the text indicates the “stroke,” or most active, phase of the gesture).
Figure 1: “I remember learning that you put one under the other...”
Similarly, when discussing multiplication of fractions, another student said, “I just remember crossing”. These word were accompanied by a gestured “X” drawn in the air with one finger (although not the focus of the current analysis, it is to be noted that the student’s words indicate a certain confusion between multiplication of fractions and taking the cross-product of two ratios).
These gestures were iconic in the sense that they referred to an actual experience of the students, and to a written inscription for a mathematical algorithm. However, the algorithm itself is a symbolic procedure drawn from the abstract world of mathematics. Thus, I have chosen to divide McNeill’s category of iconics into two subcategories: iconic-physical and iconic-symbolic.Iconic-symbolic refers to gestures that refer to written symbolic or graphical inscriptions, and/or to the procedures associated with these inscriptions. Iconic-physical corresponds to McNeill’s category of iconics, in which the referent of the gesture is something concrete or physical.
A number of the instances of iconic-physical gestures arose in response to the questions about how the students were introduced to fractions, or how they would introduce them to children. Their responses often referred to concrete hands-on manipulatives, or everyday objects such as food (“pie” was a favorite). An example of such a gesture, with the associated speech, is given in Figure 2.
Figure 2: “I think we did, like, just a stick or a rod…”
The student’s gesture in Figure 2 can be seen as indicating the length or extension of a physical rod; she went on to talk about how she remembered “dividing it in half,” with an associated cutting gesture (left hand perpendicular to the table). This was a typical iconic gesture associated with describing fractions in terms of “splitting,” “cutting,” and “dividing;” there were 11 instances of this type of gesture. Other iconic-physical gestures corresponded to objects associated with materials used to teach fractions such as “pie chart,” “pie pieces,” “boxes,” and “worksheets.”
A wide variety of terms and phrases were associated with metaphoric gestures, as might be expected with gestures corresponding to mathematical abstractions. These terms included “explanations,” “statistics,” formulas,” “the same as,” and “subtraction,” to mention only a few. Figure 3 shows an example of one such gesture. It came about while the student was describing an application of fractions in a high school class, in an activity called “Fantasy Football.” The gesture pictured came about two seconds into a sequence that extended over a period of about 15 seconds, when the student described how the outcome of the game depended on the statistics of the “players” (presumably expressed as fractions).
Figure 3: “And according to their statistics and depending
on like whether or not your team was going to win”
One interpretation of this gesture is that the left hand corresponds to a written table of statistics, with the right hand pointing to it.
An interesting set of metaphoric gestures was found in two students talking about a situation involving an “extra” amount. In one case, the student was discussing cooking, and adding “an extra teaspoon.” In the associated gesture, the student tapped the table, moving from left to right. In the second case, the student was discussing converting an improper fraction to a mixed number, and said, “If it was more than what the bottom was, it would become, like, one and – you’d have to figure, there was a lot of subtraction...” The stroke, or most active, phase of this gesture was a chopping gesture on the edge of the table, also moving from left to right, with a slight pause on the “one.” The similarity of the two gestures suggests the possibility that the left-to-right iteration represents the idea of “adding on” (a hypothesis that, clearly, would need to be confirmed through additional research).
There were only three apparently deictic gestures among the 86 analyzed, although none was the prototypical pointing with a finger or hand. Instead, one gesture consisted of waving a hand in a circle in the air, while referring to “the other four people.” In the second case, the student spoke about dividing something “between two different people;” the associated gesture concluded with her two hands separated at chest height. The final deictic gesture occurred at the end of a sequence of three gestures, as a student was describing an initial confusion she had about fractions. The associated speech included the phrase, “the whole concept of how you can, it can split and split, and this can still be the same as this.” This phrase was associated with three gestures. The first was an iconic-physical “chopping” motion, corresponding to the phrase “split and split.” In the second gesture, associated with the phrase “the same as”, the student’s hands formed open, downward facing, C-shapes, which she quickly alternated in lifting and dropping onto the table. This gesture was classified as metaphorical, in that the sameness or equality of two things was indicated by the same left and right hand shapes, and by the alternation of the same motion.
Finally, as shown in Figure 4, the second “this” in the sentence was accompanied by a “placing” gesture toward the right, indicating the location, in gesture space, of one of the two equivalent objects (fractions).
Table 1 below indicates the distribution of gestures across the four types found among the data, as well as the percentage of each type within the total set of gestures.
Table 1: Distribution of “Fraction” Gestures
As can be seen, the total number of iconic gestures, whether referring to physical or symbolic entities, is somewhat less than the number of metaphoric gestures. Iconic-physical gestures form the majority of the iconic type, but about 8% of all the gestures were iconic references to symbolic algorithms. As noted above, within the experimental context of interviews about fractions, there were few deictic gestures.
An interesting comparison can be made with gestures collected in a different experimental context. Much of the analysis of presented by McNeill is based on gestures collected during the narratives of university students who were asked to describe a cartoon they had just viewed (McNeill, 1992). The distribution of these gestures is shown in Table 2 (after McNeill, 1992, p. 93).
Table 2: Distribution of “Cartoon” Gestures
As might be expected, within a mathematical context, a much higher proportion of the gestures was metaphoric. Within the context of telling the story of what happened in a rather complicated cartoon, there were more iconics, as well as more beat gestures that help to mark the progress of a narrative.
The “Mathematical Gesture” Hypothesis
As noted above, McNeill has proposed that mathematicians have distinctive gestures for mathematical terms, and that these gestures are “somewhere on the road to a gesture language, but not all the way there” (McNeill, p. 164). That is, these gestures are similar across speakers and “have a more or less constant form,” yet are “apparently unconscious, not combining with other gestures, and ...not composed of parts” (ibid., p. 164). One of the purposes of the current research was to investigate whether people speaking about fractions exhibit distinctive gestures of this type for this concept.
In order to investigate this hypothesis, three kinds of phrases related to the concept of fraction were compiled. One type consisted of responses to the question asking for a definition of fraction suitable for children. The second consisted of phrases that named a specific fraction (e.g., “one eighth”). And the third were uses of the actual term “fraction” (the latter were generally in response to the questions about situations that used fractions or difficulties with learning fractions).
Table 3 shows the speech and associated gestures within each of these categories. The first column gives the initials of the speaker, the second, the spoken phrase, with the words associated with the stroke phase underlined. The third column gives an abbreviated description of the stroke phase of the gesture, and the last, the categorization of the gesture. (Abbreviations for the gesture descriptions are: RH, LH, BH= Right hand, Left hand, Both hands; C-, L- and S-shapes=ASL hand shapes).
One observation is that most of the students did not use any gestures in giving their definition of a fraction. In fact, of the five gestures in this category, four were from a sequence displayed by the same student, and the fifth from a second student. Although three of the gestures in the first sequence utilized L-shapes for one or both hands in some way, the gestures were not otherwise similar, nor did they resemble the other two gestures in this category. At least in the context of defining the mathematical term “fraction”, it is not possible to say that these data support the hypothesis about distinctive mathematical gestures.
In the second category, when students are talking about a specific fraction, the hypothesis received more support. In all five cases, the gestures involved either a static “cut” perpendicular to the tabletop, or a chopping motion. Finally, when the actual word “fraction” was used, there were no apparent similarities among the gestures (however, there were very few instances, and in the third gesture, the stroke phase did not correspond to the word itself).
Clearly, additional research is needed to more fully explore the hypothesis of distinctive mathematical gestures in this context. It seems that an iconic physical gesture of cutting is associated with speech describing specific fractions, but that a variety of gestures (even by the same speaker) are associated with either the explicit term “fraction,” or the attempt to define a fraction. There are a number of possible reasons for the lack of support for the hypothesis. In the first place, the participants in the study would probably not be considered “mathematicians.” They were college students, some of whom were not very confident in their understanding of mathematics. It is possible that McNeill’s hypothesis applied only or primarily to individuals with more mathematical expertise, who are accustomed to engaging in discussions about mathematics. That is, it may be a hypothesis not about mathematical gestures, but about mathematicians’ gestures. It may also be that the particular interview questions were not effective in eliciting gestures related to fractions. It would be interesting to examine further situations in which students worked with and talked about fractions, perhaps involving collaborative problem solving without the use of pencil and paper. Such a situation might not only result in more use of gesture, but might lead the participants to converge, consciously or unconsciously, on similar gestures for the same mathematical concepts. It is also possible that there were not enough instances of “fraction gestures” to fully test the hypothesis; the analysis of the remaining six interviews may provide more evidence on the question.
The purpose of the research reported here was to investigate the types of gestures utilized by college undergraduates in talking about a particular mathematical topic, that of fractions. The undergraduates were, in general, not strong mathematics students; as prospective elementary school teachers, however, the topic was important for them to understand. It is hoped that spontaneous, unconscious gestures can serve as a window into students’ understanding of particular mathematical concepts and procedures. The current analysis determined that the gestures fell into four categories: iconic-physical, iconic-symbolic, metaphoric, and deictic. Future research will continue to explore gestures related to fractions as well as other mathematical topics, and will undertake a deeper analysis of metaphorical gestures in situations involving mathematical talk. A second goal of the study was to look for evidence for the hypothesis that gestures for mathematical concepts have distinctive, semi-conventional forms. Although the evidence for this hypothesis from the current study was mixed, this hypothesis will continue to be investigated, drawing both on additional gestures from this experiment, as well as data collected in new situations. The exploration of the meaning and function of gesture in mathematical thinking, teaching, and learning has only recently begun, and it is hoped that the “natural history” of gestures related to talk about fractions will constitute a contribution to this new area of inquiry.
The research reported in this paper was supported by the Lasallian Scholars grant program of St. Mary’s College. I would like to express my appreciation to St. Mary’s Committee on Teaching and Scholarship for this support, and most particularly, to my Mathematics 101 students for their participation in the research.
Alibali, M. & diRusso, A. (1999) The function of gesture in learning to count: More than keeping track. Cognitive Development, 14. 37-56.
Corballis, M. (March-April,1999). The gestural origins of language. American Scientist, 87. 138-145.
Crowder, E. M. (1996). Gestures at work in sense-making science talk. The Journal of the Learning Sciences, 5, 173-208.
Goldin-Meadow, S., Kim, S. & Singer, M. (1999). What the teacher's hands tell the student's mind about math. Journal of Educational Psychology, 91(4).720-730.
Graham, T. (1999). The role of gesture in children’s learning to count. Journal of Experimental Child Psychology, 74. 333-355.
McNeill, D. (1992) Hand and mind: What gestures reveal about thought. Chicago: Chicago University Press.
McNeill, D. (ed.) (2000). Language and gesture. NY: Cambridge University Press.
Moschkovich, J. (1996). Moving up and getting steeper: Negotiating shared descriptions of linear graphs. The Journal of the Learning Sciences, 5(3). 239-278.
Nemirovsky, R., & Noble, T. (1997). Mathematical visualization and the place where we live. Educational Studies of Mathematics, 33(2), 99-131.
Nemirovsky, R., Tierney, C., & Wright, T. (1998). Body Motion and Graphing. Cognition and Instruction, 16(2), 119-172.
Reynolds, F. & Reeve, R. (2002). Gesture in collaborative mathematics problem-solving. Journal of Mathematical Behavior, 20. 447-460.
Roth, W.-M. (2001). Gestures: Their role in teaching and learning. Review of Educational Research, 71, 365-392.
Roth, W.-M., & Welzel, M. (2001). From activity to gestures and scientific language. Journal of Research in Science Teaching, 38. 103-136.
Varela, F. J. (1999). Ethical know-how: Action, wisdom, and cognition. Stanford, CA: Stanford University Press.
Varela, F., Thompson, E. & Rosch, E. (1991). The embodied mind: Cognitive science and human experience. Cambridge, MA: MIT Press.