# DAVID K.

PUGALEE

A COMPARISON OF VERBAL AND WRITTEN DESCRIPTIONS OF STUDENTS’ PROBLEM SOLVING PROCESSES

ABSTRACT. The purpose of this exploratory study was to investigate the impact of writing during mathematical problem solving. The study involved an analysis of ninth grade algebra students’ written and verbal descriptions of their mathematical problem solving processes. Through this comparison, a better understanding of the connection between problem solving and writing is realized. The written and verbal data show a relationship between the number of problem solving strategies tried by students and their success. The majority of problem solving behaviors involve execution actions such as carrying out goals and performing calculations. Students who construct global plans are more successful problem solvers. Students engage in veriﬁcation behaviors at various stages of problem solving though the majority of students do not verify their ﬁnal answers. While both oral and written descriptions serve as a tool for understanding students’ thinking processes, a comparison of the two modes of reporting, using a metacognitive framework as the lens of analysis, reveals some important variations. Students who wrote descriptions of their thinking were signiﬁcantly more successful in the problem solving tasks (p < 0.05) than students who verbalized their thinking. Differences in metacognitive behaviors also support the premise that writing can be an effective tool in supporting metacognitive behaviors. KEY WORDS: communication, discourse, metacognition, problem solving, writing

1. I NTRODUCTION

Recommendations emphasizing communication as an essential element of mathematics teaching and learning have been central tenets of standards for more than a decade (NCTM, 1989; NCTM, 2000). Students who have opportunities to engage in mathematical communication including speaking, reading, writing, and listening receive a dual beneﬁt of communicating to learn mathematics and learning to communicate mathematically (NCTM, 2000). Recent interest in writing as a communicative tool posits that this form of communication has the potential to promote mathematical understanding (Pugalee, 2000; Sierpinska, 1998; Morgan, 1998; Brown, 1997). Despite this interest, the relationship between writing and mathematics has been largely neglected in educational research with most literature describing writing in mathematics in general terms with little analysis of the text themselves (Morgan, 1998).

Educational Studies in Mathematics 55: 27–47, 2004. © 2004 Kluwer Academic Publishers. Printed in the Netherlands.

28

DAVID K. PUGALEE

In this study, research involving written texts will be extended through a comparison of written and verbal (talk-aloud) protocols of students’ mathematical problem solving. More speciﬁcally, this comparison will (a) provide a context for understanding basic differences in the oral and written communication of students working individually to solve mathematical problems and (b) provide a characterization of these processes.

2. BACKGROUND

Writing in mathematics has been identiﬁed as integral in sustaining the development of reasoning, communication, and connections (Connolly and Vilardi, 1989; Countryman, 1992; Sierpinska, 1998). The ability to articulate one’s ideas is seen as a benchmark of deep understanding, requiring reﬂection to identify and describe critical elements and concepts (Carpenter and Lehrer, 1999). This level of cognitive engagement substantiates writing as a generative act in the process of constructing meaning (Applebee, 1981; Emig, 1977; Flower, 1989; Miller and England, 1989; Morgan, 1998; Brown, 1997). Though the literature asserts the power of writing as a learning tool advancing the writing across the disciplines movement, it has not gained wide acceptance in mathematics classrooms. Vygotsky (1987) viewed writing as involving deliberate analytical action on the part of the writer requiring the writer to maximally compact inner speech so that it is fully understandable; thus, written words require a deliberate structuring of a web of meaning. He also viewed writing as important in forming associations between current and new knowledge, helping the writer tie down ideas in order to make connections between prior and new concepts. Meier and Rishel (1998) hold that writing, speaking and thinking are entwined such that narrative writing and speaking assists students in better understanding mathematical ideas. Written words provide a vehicle for students to communicate with self and others. These written forms support deeper mathematical understanding developed through students’ critical thinking (Masingila and Prus-Wisniowska, 1996). Research also shows that writing supports metacognition (Pugalee, 2001; Powell, 1997; Artzt and Armour-Thomas, 1992; Carr and Biddlecomb, 1998). In these studies, writing is posited as providing a level of reﬂection that promotes students’ attending to their thinking about mathematical processes. This awareness and self-regulation appears to play an important role in students’ selection of appropriate information and strategies while solving a wide range of mathematical problems. Metacognition includes such behaviors as predicting, planning, revising, selecting, classifying and checking (Allen, 1991) allowing individuals to be successful in

COMPARISON OF PROBLEM SOLVING PROCESSES

29

problem solving situations by contributing to their ability to identify and work strategically (Davidson and Sternberg, 1998). These behaviors also allow them to make connections between their problem solving work and their knowledge of mathematical content and procedures (Schurter, 2002). Garofalo and Lester (1985) identiﬁed a metacognitive framework consisting of four categories of activities or behaviors involved in performing a mathematical task: orientation, organization, execution, and veriﬁcation. Each phase includes several metacognitive behaviors related to problem solving. Orientation includes comprehension strategies, analysis of information and conditions, assessment of familiarity with a task, initial and subsequent representation, and assessment of problem difﬁculty and chance of success. Organization includes identiﬁcation of goals and subgoals, global planning, and the local planning necessary to carry out global plans. Execution includes performance of local actions, monitoring progress of local and global plans, and trade-off decisions. Veriﬁcation includes evaluation of decisions and results of executed plans. The researchers determined that these four categories of behaviors impacted performance on a wide variety of mathematical tasks. This framework provides a useful context for analysis of students’ written and oral protocols generated while engaged in mathematical problem solving. Think-aloud protocols are generally accepted as a means of assessing the mental processes of an individual. These protocols require the subject to ‘think aloud’ concerning his/her thoughts as s/he works. Such information is valued as a tool in process tracing research. Smagorinsky (1989) argues that internal thought processes of subjects are not altered when engaging in verbalizations that do not require the reporting of information not normally attended to while performing the task. He points to studies that show no differences in the speciﬁc steps taken by groups who either solved a problem silently or while talking aloud. Such verbalizations should not be confused with various genres of classroom discourse that involve a complex amalgamation of mathematical, social, and experiential factors providing cognitive advantages when students work together or interact with teachers (see O’Connor, 1998; Gravemeijer, Cobb, Bowers, and Whitenack, 2000). Writing has been viewed as “thinking aloud on paper” (Rose, 1989). These process-tracing methods (Flower and Hayes, 1983) provide more information than input-output methods, they yield rich data, and provide a means of observing important processes difﬁcult to identify using other methods. Verbal and written protocols are used in this study as a tool in comparing students’ mathematical problem solving processes.

30

DAVID K. PUGALEE

3. DATA AND ANALYSIS

3.1. Sample Twenty subjects enrolled in an introductory high school algebra class participated in this study. These ninth grade subjects included eleven females and nine males. Seven were African American and thirteen were Caucasian. The subjects were assigned to one of two groups by alternating from a ranked list based on a composite score from language arts and mathematics results obtained from the previous year’s state mandated tests in these two subjects. This process was utilized so that both groups would be comparable in mathematical and verbal abilities. Since the groups would be alternating in providing written and verbal descriptions of their problem solving processes, this procedure controlled for potential differences in the language and mathematics abilities of the two groups. 3.2. Problem selection The researcher and a university mathematics educator identiﬁed twenty problems from curricular materials, including print resources and textbooks, as appropriate for the age and academic level of the students. Academic level was considered so that the problems would likely provide students with an entry point for obtaining a solution. Additionally, these open-ended problems were deemed likely to produce descriptive problem solving processes. A panel of ten experts in mathematics education, the majority of them classroom teachers with an average 12.5 years of experience, then rated the problems for the level of difﬁculty. The ratings reﬂected the experts’ judgments about the relative success of students in introductory algebra to arrive at a correct solution: low level problems likely to be answered correctly by the majority of students, medium level problems likely to be answered correctly by less that three-fourths of students, and high level problems likely to be answered correctly by less than one-third of students. The panel also indicated the strategy which they felt was most likely to be used by the students. Six problems were then chosen so that two problems from each of three levels of difﬁculty (low, moderate, high) were included. This selection process also considered the strategies the panel thought would most likely be used by the students. This range of difﬁculty level and predicted strategy use was intended to provide opportunities to observe students’ responses in diverse problem situations. A task analysis of the six problems provides information about the problem structure giving additional indicators of problem complexity and variation (see Kulm, 1979; Barnett, 1979; Webb, 1979).

COMPARISON OF PROBLEM SOLVING PROCESSES

31

The task analysis of these six selected problems includes descriptions of syntax, content, context, and structure variables. Syntax, content, and context variables are discussed in the following paragraphs. Structure variables will be limited to a discussion of the problem-solving strategies intrinsic to the problem. This discussion occurs as part of the analysis of error and strategy types later in this paper. Syntax variables describe arrangement of and relationship among words and symbols in a problem including problem length, grammatical complexity, and data sequence. Syntactic variables can be used as indicators of the time and difﬁculty involved in language processing. Problem length may be measured by considering the total number of words per problem and the average number of words per sentence. For the six problems, the number of words ranged from 14 for problem 3 to 50 for problem 2. Average sentence length is considered as an indicator of grammatical complexity. Two problems had average sentence lengths of less than ten words per sentence (problem 2 with 8.3; problem 3 with 7.0). The other four problems had average sentence lengths of more than ten words (problem 1 with 11.3; problem 4 with 13.3; problem 5 with 14.0, and problem 6 with 18.3). Location of the question stem in the problem is another feature in considering syntax. All six problems contain the question stem at the end of the problem. Data sequence, or the arrangement of information in the problem used in the solution, is also a surface factor of a problem. For all the problems, with exception of number 2, the data in the problem was presented in the order needed to solve the task. Question 2 requires subtraction from the total distance which is given at the end of the problem. Content and context variables refer to the meanings conveyed by a problem. Content variables extend beyond mathematical topic to include linguistic content variables such as the use of mathematical words or phrases. The six problems contain some mathematical terms that might impact the meaning. Problem 1 requires distinction between digit and numeral. Problem 2 and 6 use distance units such as miles for problem 2 and meters (and meters per second) for problem 6. Problem 3 uses the term triangular number. Problem 4 requires understanding of average and relation terms such as lowest and least. Problem 5 uses ‘adding’ as a mathematical operation. Context involves nonmathematical meanings conveyed in the problem. Problems 2 and 5 involve the use of diagrams that are essential in solving the problem. This context provides an additional component in interpreting the problem situation. None of the problems contain extraneous information or numbers, and all numbers given in the problems are necessary to obtain correct solutions. Whether the problem is a real world application

32

DAVID K. PUGALEE

is also a context variable. All problems, except 2 and 5, present real-world contexts. An analysis of problem characteristics such as those described through consideration of content, context, and structure variables allows for the formulation of inferences about the complexity of the tasks and sources of possible variation in mathematical problem solving. This focus on those variables that are intrinsic to the problem itself offers support for the selection of the problems based on varying levels of linguistic complexity and anticipated strategy employment. The rationale for this process is based on the premise that these elements interact to determine the complexity of the problem situation and that such problem situations should be cognitively engaging while also at a level that would not be overly demanding for the student to carrying out, monitor, and describe solution processes. 3.3. Procedure Students were involved in a two-week enrichment period where they engaged in journal writing, focusing on describing their thought processes while solving mathematics problems. At the beginning of ten class sessions, students were given a mathematics problem to solve. Students were instructed to write every thought that came to mind while solving the problems. Students were given approximately ten minutes for the exercises, though students who needed more time were allowed to complete their work. The writing samples were collected each day and read by the teacher who provided comments and questions related to the depth of the details provided in the written descriptions of the students’ problem solving processes. Students were encouraged to expand and elaborate on steps where descriptive information was vague or nonexistent. In general, the focus was to encourage students to extend both the quantity and quality of the descriptions of their mathematical thinking: providing details about processes, justifying and reasoning about the steps taken in the problems. The problems along with the teacher’s feedback were returned during the next class. Students were encouraged to read the comments, build on strengths during subsequent writing sessions while also addressing the teacher’s questions and comments intended to extend the students’ descriptions. Since students routinely discussed problem solving approaches and strategies in the classroom, this emphasis on writing exposed students to the use of writing as a mode to describe their mathematical thinking. The intensive writing focus was intended to facilitate students’ ability to describe their problem solving processes orally as well as in writing. Additionally, all students were given two trial sessions providing oral descriptions of their problem solving processes while being videotaped. The

COMPARISON OF PROBLEM SOLVING PROCESSES

33

purpose of these sessions was to help students feel at ease with a camera recording their oral descriptions and to provide some basic comments related to how their verbal descriptions could be extended. Data was collected over a six-day period with students divided into groups as described. Students alternated in providing oral descriptions or written descriptions as they worked the problems. For example, students in Group 1 solved one of the difﬁcult problems and provided oral descriptions of their problem solving processes while Group 2 provided written descriptions for the problem. The groups then switched the type of descriptions provided for the second problem, also rated as difﬁcult. This process continued for the two medium level and two easy level problems. For each problem there was a total of twenty student responses, ten from the group providing verbal descriptions and ten from the group providing written descriptions. Each student gave both an oral and written description of a problem with a low, medium, and high level of difﬁculty. Students who provided written descriptions on a given day worked the problems individually in the classroom while those students who provided oral descriptions were supervised in the library. This prevented students from having any prior knowledge of the prompt for that day. For all problems, students were provided paper, calculators, and pencils without erasers to discourage erasing of work. In the writing sessions, students were presented with the problem and read the following statement:

You have been given a mathematics problem to solve. Please record any work for the problem on the paper provided just as you did in the practice sessions. Remember to write everything which comes to mind during the solving of the problem.

If any students were observed not writing for ﬁfteen seconds, they were prompted, “Please write what you are thinking.” The problems were collected at the end of the session and labeled using codes for identiﬁcation purposes so that all responses from an individual student would be identiﬁable along with the student’s race and gender, but protecting the personal identity of the student. Subjects in the think-aloud groups were taken one at a time from the library to an adjoining media room where their problem solving sessions were videotaped. The student was given the problem and materials. The following instructions were read by the researcher:

You have been given a mathematics problem to solve. You may do any necessary computations and work on the paper provided. While you are solving the problem, please think out loud by telling everything that comes to mind while you are solving the problem.

34

DAVID K. PUGALEE

The student was videotaped while solving the problem with the camera focusing on the student’s work. If the student was silent for ﬁfteen seconds, s/he was prompted, “Please tell me what you are thinking.” When the student ﬁnished, s/he was sent back to class with care taken that s/he did not communicate with any students who were waiting to complete their videotaped problem solving session. All work was collected and coded for identiﬁcation. 3.4. Analysis of the data The videotaped sessions were transcribed and categorized by student and problem. The data also included any samples of work generated by the student during the problem solving session. These data were analyzed at two levels: identiﬁcation of error patterns and strategies employed by the students, and categorization of data components using qualitative methodologies to identify similarities and differences in the students’ responses. Additionally, the frequency of correct and incorrect ﬁnal answers for each of the problems was recorded. The researcher and a trained rater reviewed all problems to determine the error patterns and the strategies used by the students. The rater was a college graduate with extensive experience working with a private company specializing in scoring tests and inventories. For practice, the rater worked through ﬁve sample problems, under the direction of the researcher, obtained from the practice sessions preceding the collection of data. The analysis involved a content analysis of students’ work focusing on conceptual and procedural errors (Porter and Masingila, 2000). Three categories were utilized: procedural, computational, and algebraic. Computational errors were a subset of procedural errors separated for descriptive purposes. Procedural errors primarily consisted of using an incorrect mathematical operation in executing a plan. Algebraic errors included conceptual and algorithmic problems in selecting an appropriate model, representing information, and application of algebraic principles. The researcher and rater agreed on 91.2% of the ratings. The rater also used the strategies identiﬁed by the problem selection panel as likely to be used by students in solving the mathematics problems. The categories included using a diagram, table, list, or other visual; guess and check; working backwards; searching for a pattern; logical reasoning; and other. The researcher and rater agreed at a rate of approximately 98% in categorizing the primary strategy used in a problem. The second level of analysis employed a qualitative process of identifying pieces of data based on their homogeneity and heterogeneity (Patton, 2001). Determining which pieces of data were similar led to categories

COMPARISON OF PROBLEM SOLVING PROCESSES

35

of data where the differences in the categories were separate and evident while also demonstrating categories where the data interlocked in a meaningful way (Patton, 2001). This process of grouping the data into themes provided a link to patterns that could be induced from the categories (Gay and Airasian, 2003). Five categories emerged with four corresponding to the metacognitive framework used by Garofalo and Lester (1985) to describe mathematical problem solving processes: orientation, organization, execution, and veriﬁcation. The ﬁfth category consisted of responses that represented either students’ affective statements or ﬁller phrases. This initial data analysis of the transcribed verbal and written responses from the problem solving sessions provided data components in the four broad categories already mentioned. Data in each of these four categories were further classiﬁed into sub-categories using Garofalo and Lester’s (1985) metacognitive framework. The ﬁfth category which included ﬁller statements such as uhhh, ok, ummm and similar terms was not included in this subsequent analysis. It might be noted, however, that students giving verbal descriptions of their problem solving processes used more such ﬁllers, 62 instances versus 41 occurrences in the written data. The categorized data pieces varied in length from two sentences to sentence fragments of a few words. The pieces were color-coded based on the type of metacognitive behavior represented. Totals for the categories and sub-categories were tabulated. A university-level mathematics educator reviewed a random sample of 30% of the students’ work categorizing data segments using the same qualitative methods. This process included categorizing data items into one of the ﬁve broader categories, discussing discrepancies and reaching a consensus on this initial classiﬁcation, further classifying the data into subcategories from the metacognitive framework, discussing discrepancies in this subsequent categorization and reaching a ﬁnal consensus. An equal number of problems in this random sample came from the writing group and the ‘think-aloud’ group. Initial agreement was at 88% and all discrepancies were satisfactorily resolved through reanalysis and discussion.

4. F INDINGS AND DISCUSSION

4.1. Strategies and error types The strategies used by students did not vary greatly between those who provided written or verbal descriptions of their problem solving processes. For four of the six problems, more than 90 percent of the students used the same strategy for that particular problem. For two of the problems, 1

36

TABLE I Frequencies of problem solving strategies Diagram, table, list, other visual Problem 1 Problem 2 Problem 3 Problem 4 Problem 5 Problem 6 Totals 0 – Verbal 1 – Written 13 – Verbal 12 – Written none 1 – Verbal 2 – Written none none none none 9- Verbal 10 – Written none 2 – Verbal 1 – Written none 13 -Verbal 14 – Written none 1 – Verbal 2 – Written none 7 – Verbal 5 – Written 2 – Verbal 2 – Written 4 – Verbal 4 – Written none none none 2 – Verbal 3 – Written none Guess and check Work backwards Search for pattern Logical reasoning Other

1 – Verbal 2 – Written none 2 – Verbal 0 – Written none none none 3 -Verbal 2 – Written

DAVID K. PUGALEE

9 – Verbal 10 – Written 8 – Verbal 9 – Written 3 – Verbal 0 – Written 20 – Verbal 19 – Written

2 – Verbal 1 – Written 9 – Verbal 9 – Written 1 – Verbal 1 – Written 1 – Verbal 0 – Written 1 – Verbal 1 – Written 4 – Verbal 3 – Written 18 -Verbal 15 – Written

COMPARISON OF PROBLEM SOLVING PROCESSES

37

TABLE II Frequency of error types by group Algebraic Think-Aloud Written 4 4 Computational 10 7 Procedural∗∗ 30 19 None∗∗ 20 32

∗∗ Test of signiﬁcance between proportions, signiﬁcant at p < .05.

and 6, the percentage of students using the same strategy was 78 and 70. These two problems also had the lowest rate of success. The strategy used the least by the students was ‘working backwards’ which was only used for problem number 2. The raters selected working backwards as the least likely strategy to be used. The other strategies ranged from twenty-ﬁve, approximately 19% of total, for using diagrams, tables, lists, and other visuals to thirty-nine, approximately 30%, for guess and check. In general, strategies actually used by students were consistent with the raters’ predictions about strategy use with some differences in the rankings between predicted and actual strategy used. The raters selected making diagrams / tables / other visuals, guess and check, and logical reasoning (in that order) as the three most likely strategies to be used by students. Students actually used guess and check most frequently followed by logical reasoning and diagrams / tables / other visuals. Table I shows the primary strategy employed for each problem across the two reporting methods: think-aloud and writing. A two proportion z test was computed for each strategy to determine if there were signiﬁcant differences in the strategies employed based on whether the think-aloud or written reporting method was used. There were no signiﬁcant differences in the strategies used: the resulting one tailed p values were all greater than .28. An analysis of the types of errors made by the students indicated that the majority of errors was procedural (66.2% of all errors) followed by computation (23%) and algebraic (10.8%). Table II shows total frequency across all problems by type of error and reporting method. A test for differences between proportions revealed a signiﬁcant difference in procedural errors based on students’ process reporting method. Students written descriptions resulted in signiﬁcantly fewer procedural errors when compared to thinkaloud descriptions (z = 2.222, p < 0.05) (see Bruning and Kintz, 1987). There were no statistical differences between the two reporting methods on algebraic and computation errors. All problems were scored for the correct ﬁnal answer. Those students who wrote about their problem solving processes produced 32 correct solutions out of the total of 60 solutions,

38

DAVID K. PUGALEE

TABLE III Frequency of metacognitive behaviors Think-aloud process ORIENTATION Reading/Rereading Initial/Subsequent Representation Analysis of Information & Conditions Assessment of Difﬁculty Total Orientation∗∗ ORGANIZATION Identiﬁcation Goals/Subgoals Making/Implementing Global Plans Data Organization Total Organization EXECUTION Performing Local Goals Monitoring Goals Calculations Redirecting Total Execution∗∗ VERIFICATION Evaluating Decisions Checking Calculations Total Veriﬁcation∗∗

∗∗ Signiﬁcant at p < 0.05.

Written process

12 9 9 11 38 11 60 6 77 37 14 75 1 127 42 2 44

25 10 20 16 71 22 35 15 72 64 31 73 4 172 18 3 21

whereas the think aloud students produced 20 correct solutions. A test for signiﬁcance between two proportions was conducted resulting in a z score of 2.2126, signiﬁcant at the .05 level. 4.1.1. Describing the data using the metacognitive framework The four major categories of the framework include sub-categories that further delineate the metacognitive behaviors indicative of each of the four phases. Each of these is described in the following paragraphs using representative data to illustrate the types of student responses that are indicative of each of these metacognitive behaviors. This descriptive process also includes a comparison of the frequencies of these behaviors depending on the type of reporting method (verbal or written) employed when the student was engaged in solving the problem. Table III shows the frequencies for the sub-categories and four phases for the two reporting methods. Due to the small frequencies in some of the cells for the sub-categories, a test for

COMPARISON OF PROBLEM SOLVING PROCESSES

39

signiﬁcance between proportions was only conducted for each of the four phases: orientation, organization, execution, and veriﬁcation. 4.2. Orientation phase The orientation phase represents initial attempts of the student to become familiar with the problem situation. Metacognitive behaviors associated with this category include reading/rereading, initial and subsequent representations, analysis of information and conditions, and assessment of problem difﬁculty. The total number of orientation behaviors was signiﬁcantly higher for students providing written descriptions (z = 4.470, p < 0.05). Reading and rereading behaviors were more evident in students who described their problem solving processing through writing, indicating that the problem was reread more than twice as often as the ‘think-aloud’ group. The initial reading of the problem was not coded. The data indicates that students placed high importance on rereading. A student in the thinkaloud group stated, “Well, ﬁrst of all I do not understand the problem so I will reread it.” Another student in the writing group disclosed, “I read this problem three times.” Analysis of information and conditions statements was more prevalent for the writing group. Comments from both groups for the digit problem (number 1) show similarity in the content of such statements. A student using ‘think-aloud’ added, “For 1 through 9 that would be single digits” while another student wrote, “There is 1 of every number including zero every 10 numbers.” Such statements suggest that metacognitive behaviors of this type press students toward an initial understanding of how information in the problem can be applied in developing a solution plan. Initial/subsequent representations continued the pattern of being more than twice as prevalent among the written responses. For both groups, initial and subsequent representations involved restating information from the problem. “The apartments are numbered from 1 through 99” was vocalized by one student. Another student wrote, “I said if she falls 54 meters per second without the parashoot and 6 meters per second with it.” Such statements illustrate students’ attending to information in the problem. Assessing problem difﬁculty was evident at relatively equivalent levels for both groups. Behaviors in this category also represented the fewest number of responses in the orientation category. There were few statements about the level of difﬁculty of the problem. Statements directly related to problem difﬁculty were restricted to the two problems with the lowest success rate, problems one and six. Statements across both groups were comparable including short comments about the problem being “hard”.

40

DAVID K. PUGALEE

Behaviors in these two subcategories, analysis of information and conditions and assessment of difﬁculty, seem to be indicative of students sorting through the information and focusing on how to initially represent key information from the problem. 4.2.1. Organization phase The organization phase includes metacognitive behaviors such as identiﬁcation of goals and subgoals, making and implementing global plans, and data organization. In general such behaviors assist the student in understanding how information in the problem relates to the problem-solving task including the formulation of goals and plans. There was no signiﬁcant difference in the total number of responses between the two reporting methods for the organization phase (z = .579, p > .28). For identiﬁcation of goals and subgoals, protocols from the writing group were twice as frequent as those from the think-aloud group. Goals for the apartment numbers problem included such statements as “How many numbers does she have to buy,” “So, I can count every number,” and “You go up a number each time in the pattern.” Statements in this subcategory denoted an overall plan that would lead to a successful solution for the problem. The subcategory of making and implementing global plans involved more speciﬁc information than identiﬁcation of goals and subgoals. Behaviors in this category reﬂected students’ attention to more speciﬁc ideas about how to carry out the broader goals that reﬂected how to best approach solving the problem. This category shows a higher frequency of statements for the think-aloud group. Further investigation shows that some of these verbalized statements reﬂected the restatement of plans. For example, a student for problem ﬁve stated that she was “going to select random numbers to equal 14.” Later, she states that she is “lost” and reiterates that she is going to “try another number.” There were four such occurrences in the verbal protocols of similar restatements while there were no observations noted for the writing group. The robust performance of a student verbalizing his solution to number six also contributed to a high frequency in this category as he changed his local plan several times in an attempt to solve the problem.

Let’s say she falls 54 meters a second before her chute opens for 10 seconds and she falls 90 seconds after her chute opens. . . . I’m going to try something else. I am going to try 20 seconds before her chute opens. . . . I’m going to try 19. She dropped 19 seconds before her chute opened. . . . So, I’m going to do 16. . . . I was going to see if you could say he was like 17.5 seconds. . . . I’m going to try since I need to go up a little bit I’m going to try 163 /4 before her chute opens and 821 /4 seconds after chute open. . .

COMPARISON OF PROBLEM SOLVING PROCESSES

41

This student’s performance accounted for approximately ten percent of the coded segments for making/implementing global plans by think-aloud students. The extent to which this student evaluated and reevaluated his plan was not matched by any other student in either group. Repetitious statements and this robust performance appear to account for the difference in frequencies between the two groups. Responses labeled as data organization were more evident in the students’ written responses. Data organization actions included listing information about the number of digits or numbers for problem one and making dots for the triangular numbers in problem three. Though differing in quantity, these processes were similar regardless of the reporting method. Two responses (for problem two) from the writing group included diagrams with labels showing the distances between the cities. There were no diagrams constructed by any of the students who used think-aloud to describe their problem solving processes. 4.2.2. Execution phase The execution phase includes metacognitive behaviors such as performing local goals, doing calculations, monitoring goals and redirecting plans. This category accounted for the highest number of coded data statements. While the writing group generated more statements classiﬁed as performing local goals, monitoring goals, and redirecting, the think-aloud group had more instances of performing calculations. There was a signiﬁcant difference in the proportion of execution behaviors with approximately 35% more for the written descriptions (z = 3.680, p < 0.05). Performance of local goals and calculations are intrinsically related in the students’ problem solving descriptions; therefore, they are discussed together. For example, a student verbalized, “So, I’ll try 7 + 7. Then 7 + 16. 16 is in the lower left-hand circle. That is 23.” This movement back and forth between executing local goals and performing calculations is further elucidated in the following student’s response to the digit problem:

So there are 9 sets of 10 up to 90 + 9 more numbers to 99. So I multiply 1 × 10 sets of 1s and that give me 10 ones plus 10 more counting the numbers in the teens. That gives me 20 ones. That process is repeated for the following numbers 1–9. 0 is an exception. There are only 10 0s because it only used for numbers 0, 10, 20, 30, 40, 50, 60, 70, 80, 90. So she should buy 20 of each digit except zero where she should buy only 10 of those.

Problem solving actions classiﬁed as redirecting plans result from an effective monitoring of goals. There were few redirecting behaviors for students using either process. Several such examples are found in the written descriptions for problem 1. In one of these occurrences, the student had begun counting the number of digits needed for the apartment numbering

42

DAVID K. PUGALEE

by considering all ones and then all twos. He realizes the output is the same and changes his approach to ﬁnding a pattern. “By looking at the numbers – all the numbers, they each need twenty digits but zero needs nine digits.” The other example of redirecting for problem 1 also involved a student moving beyond a counting method for the apartment numbers to that of ﬁnding a pattern. Students who wrote descriptions while solving the problems produced more than twice as many examples of monitoring goals than students who engaged in think-aloud. One student wrote of his approach for ﬁnding the average in problem 4, “Then what I did was pick some number and average till I got the lowest he could make.” Monitoring goals was also evident in more global statements such as the following comment from a student using think-aloud. “By using the strategy guess and check, I found my answer and the answer is 96.” The student displays conﬁdence that the selected strategy was viable in producing an acceptable response. Such instances demonstrate the metacognitive nature of monitoring behaviors. While it is difﬁcult to infer the level of monitoring required in the types of actions described by students using either process, such statements show the importance of goals in directing students’ thinking about their problem solving behaviors. 4.2.3. Veriﬁcation phase The veriﬁcation phase consists of metacognitive behaviors such as evaluating decisions and results. Because of the non-linear nature of problem solving, these behaviors were not necessarily found at the conclusion of the problem solving sessions. Some veriﬁcation processes are by nature part of the process of monitoring global and local plans and were categorized with the related behaviors in other parts of the framework. The behaviors in this category deal with those segments demonstrating that students were checking ﬁnal answers and computations. Veriﬁcation behaviors accounted for the smallest number of data cells among the four phases. There was a signiﬁcant difference between the two methods with twice as many veriﬁcation behaviors for students using the think-aloud description method (z = 4.034, p < 0.05) In many cases, students simply indicated that they had engaged in such a process. “I checked to make sure that I was right. It was right, so I am ﬁnished with this problem” and “I have checked my problem very carefully and my conclusion is right.” There were few instances of verifying computations. Veriﬁcation of computations included using subtraction to check for addition, labeling work as “mistake” and recalculating, and checkmarks beside computation results on papers from both groups. Statements such

COMPARISON OF PROBLEM SOLVING PROCESSES

43

as “Might go a little bit higher” and “In order to check you would have to draw everything out” represent cases where students’ perceptions about their solutions appeared to indicate some dissonance but did not result in any follow-up actions that would have changed their incorrect solutions. The frequency of veriﬁcation behaviors was not related to success with the problem solving task.

5. C ONCLUSIONS AND IMPLICATIONS

This study involved analysis of written and think-aloud descriptions of the problem solving processes of ninth grade algebra students as they solved mathematical problems. Qualitative analysis showed that both reporting methods provide a useful lens for studying the problem solving behaviors of students. A comparison of the data from the two modes (writing and think-aloud) reveals some notable differences. Before focusing on the comparison, there are some pertinent conclusions related to students’ problem solving behaviors regardless of the method used to describe their thinking. First, there was a relationship between the number of different strategies attempted by students for a particular problem and their success in solving that problem. Problems with a high rate of success showed similar strategy use among the students; likewise, there was evident variability in strategies for those problems with low success rates. Second, execution behaviors (performing local goals, monitoring goals, performing calculations, and redirecting) comprised the largest number of problem solving actions. Third, students who constructed global plans were more likely to be successful at the problem solving tasks whether those plans were stated or implied in the descriptions of their problem solving processes. Fourth, students assessed the difﬁculty at the beginning and end of the problem solving process suggesting that this behavior is both formative and summative. Fifth, students did not generally verify the accuracy of their ﬁnal answers. Students who wrote about their problem solving processes produced correct solutions at a statistically higher rate than when using think-aloud processes. Students who wrote descriptions of their processes produced signiﬁcantly more orientation and execution statements than students who verbalized their responses. While there was no signiﬁcant difference between verbalized and written organization statements, closer inspection revealed that this difference is largely accounted for in repetitious statements and one student’s robust verbalization for one problem as he worked through multiple redirects in attempting to reach a solution. In two of the three subcategories, identiﬁcation of goals and data organization, written

44

DAVID K. PUGALEE

responses were at least twice as frequent as verbal responses. While there was a signiﬁcant difference favoring the verbalization mode for veriﬁcation behaviors, the greater frequency did not have an impact on students’ success with the problem. While students may have stated that they veriﬁed their answer, there is little evidence that such behaviors occurred and few examples of students redirecting their work. These ﬁndings provide support for the premise that the interaction of students with metacognitive processes along with an understanding of mathematical concepts distinguishes them as successful problem solvers (Schurter, 2002). This study demonstrates that writing can be a tool for supporting a metacognitive framework and that this process is more effective than the use of think-aloud processes. Further research is necessary to investigate those factors that provide for successful implementation of a writing program designed to develop metacognitive behaviors. Additional research is also necessary to determine how and to what extent verbalized processes might be used in positively affecting students’ problem solving performance.

A PPENDIX A 1. Mrs. Maples buys numerals to put on the doors of each apartment in a 99-unit apartment building. The apartments are numbered 1 through 99. How many of each digit should Mrs. Maples buy? 2. Claire drove from Philadelphia to Brooklyn, then to Manhattan, then to Scarsdale. She returned over the same route. From Philadelphia to Brooklyn is 93 miles. From Brooklyn to Manhattan is 11 miles. The trip from Scarsdale to Philadelphia was 131 miles. What is the distance between Manhattan and Scarsdale? 3. These are the ﬁrst four triangular numbers. What is the tenth triangular number?

4. Kevin’s scores on 4 tests were 86, 72, 94, and 77. He wants his average on ﬁve tests to be at least 85. What is the lowest score he can get on the ﬁfth test to achieve this average?

COMPARISON OF PROBLEM SOLVING PROCESSES

45

5. The numbers in the big circles are found by adding the numbers in the two smaller circles on each side. Find the numbers for the small circles.

6. Skydivers fall at 54 meters per second before their chutes open. They fall at 6 meters per second after their chutes open. If a skydiver jumped from a plane 1400 meters high and reached the ground in 100 seconds, how high was she when she opened her chute?

R EFERENCES

Allen, B.R.: 1991, A Study of Metacognitive Skill as Inﬂuenced by Expressive Writing in College Introductory Algebra Classes, Unpublished doctoral dissertation, Louisiana State University and Agricultural And Mechanical College. Applebee, A.N.: 1981’, Writing in the Secondary School, National Council of Teachers of English, Urbana, IL. Artz, A.F. and Armour-Thomas, E.: 1992, ‘Development of a cognitive-metacognitive framework for protocol analysis of mathematical problem solving in small groups’, Cognition and Instruction 9(2), 137–175. Barnett, J.: 1979, ‘The study of syntax variables’, in G.A. Goldin and C.E. McClintock (eds.), Task Variables in Mathematical Problem solving, ERIC Clearinghouse for Science, Mathematics, and Environmental Education, Columbus, OH, pp. 23–68. (ERIC Document Reproduction Service No. ED178366) Retrieved July 22, 2003, from the ERIC database. Brown, T.: 1997, Mathematics Education and Language: Interpreting Hermeneutics and Post-Structuralism, Kluwer Academic Publishers, Dordrecht, The Netherlands. Bruning, J.L. and Kintz, B.L.: 1987, Computational Handbook of Statistics, Scott, Foresman and Company, Glenview, IL. Carpenter, T.P. and Lehrer, R.: 1999, ‘Teaching and learning mathematics with understanding’, in E. Fennema and T. Romberg (eds.), Mathematics Classrooms that Promote Understanding, Lawrence Erlbaum Associates, Mahweh, NJ, pp. 19–32. Carr, M. and Biddlecombe, B.: 1998, ‘Metacognition in mathematics: From a constructivist perspective’, in D.J. Hacker, J. Dunlosky and A.C. Graesser (eds.), Metacognition in Educational Theory and Practice, Lawrence Erlbaum Associates, Mahweh, NJ. Davidson, J.E. and Sternberg, R.J.: 1998, ‘Verbalization and problem solving’, in D.J. Hacker, J. Dunlosky and A.C. Graesser (eds.), Metacognition in Educational Theory and Practice, Lawrence Erlbaum Associates, Mahweh, NJ, pp. 47–68. Emig, J.: 1977, ‘Writing as a mode of learning’, College Composition and Communication 28, 122–127.

46

DAVID K. PUGALEE

Flower, L.: 1989, ‘Taking thought: The role of conscious processing in the making of meaning’, in E.P. Maimon, B.F. Nodine and F.W. O’Connor (eds.), Thinking, Reasoning and Writing, Longman, New York, pp. 185–212. Flower, L. and Hayes, J.R.: 1983, ‘Uncovering cognitive processes in writing: An introduction to protocol analysis’, in P. Mosenthal, L. Tamor and S.A. Walmsley (eds.), Research on Writing, Longman, New York. Gay, L.R. and Airasian, P.: 2003, Educational Research: Competencies for Analysis and Applications, Pearson Education, Inc., Upper Saddle River, NJ. Garofalo, J. and Lester, F.: 1985, ‘Metacognition, cognitive monitoring and mathematical performance’, Journal for Research in Mathematics Education 16(3), 163–176. Gravemeijer, K., Cobb, P., Bowers, J. and Whitenack, J.: 2000, ‘Symbolizing, modeling, and instructional design’, in P. Cobb, E. Yackel and K. McClain (eds.), Symbolizing and Communicating in Mathematics Classrooms, Lawrence Erlbaum Associates, Mahwah, NJ, pp. 225–273. Kulm, G.: 1979, ‘The classiﬁcation of problem-solving research variables’, in G.A. Goldin and C.E. McClintock (eds.), Task Variables in Mathematical Problem Solving, ERIC Clearinghouse for Science, Mathematics and Environmental Education, Columbus, OH, pp. 1–22. (ERIC Document Reproduction Service No. ED178366) Retrieved July 22, 2003, from the ERIC database. Masingila, J.O. and Prus-Wisniowska, E.: 1996, ‘Developing and assessing mathematical understanding in calculus through writing’, in P.C. Elliott and M.J. Kenney (eds.), Communication in Mathematics, K-12 and Beyond, National Council of Teachers of Mathematics, Reston, VA, pp. 95–104. Meir, J. and Rishel, T.: 1998, Writing in the Teaching and Learning of Mathematics, The Mathematical Association of America, Washington, DC. Miller, L.D. and England, D.A.: 1989, ‘Writing to learn algebra’, School Science and Mathematics 89(4), 299–312. Morgan, C.: 1998, Writing Mathematically: The Discourse of Investigation, Falmer Press, London. National Council of Teachers of Mathematics: 2000, Principles and Standards for School Mathematics, author, Reston, VA. National Council of Teachers of Mathematics: 1989, Curriculum and Evaluation Standards for School Mathematics, author, Reston, VA. O’Connor, M.C.: 1998, ‘Language socialization in the mathematics classroom: Discourse practices and mathematical thinking’, in M. Lampert and M.L. Blunk (eds.), Talking Mathematics in School: Studies of Teaching and Learning, Cambridge University Press, Cambridge, UK, pp. 17–55. Patton, M.Q.: 2001, Qualitative Research and Evaluation Methods, Sage, Thousand Oaks, CA. Porter, M.K. and Masinglila, J.O.: 2000, ‘Examining the effects of writing on conceptual and procedural knowledge in calculus’, Educational Studies in Mathematics 42(2), 165– 77. Powell, A.B.: 1997, ‘Capturing, examining and responding to mathematical thinking through writing’, The Clearing House 71(1), 21–25. Pugalee, D.K.: 2001, ‘Writing, mathematics and metacognition: Looking for connections through students’ work in mathematical problem solving’, School Science and Mathematics 101(5), 236–245.

COMPARISON OF PROBLEM SOLVING PROCESSES

47

Rose, B.J.: 1989, ‘Writing and math: Theory and practice’, in P. Connolly and T. Vilardi (eds.), Writing to Learn Mathematics and Science, Teachers College Press, New York, pp. 15–30. Schurter, W.A.: 2002, ‘Comprehension monitoring: An aid to mathematical problem solving’, Journal of Developmental Education 26(2), 22–33. Sierpinska, A.: 1998, ‘Three epistemologies, three views of classroom communication: Constructivism, sociocultural approaches, interactionism’, in H. Steinbring, M.G. Bartolini Bussi and A. Sierpinska (eds.), Language and Communication in the Mathematics Classroom, National Council of Teachers of Mathematics, Reston, VA, pp. 30–62. Smagorinsky, P.: 1989, ‘The reliability and validity of protocol analysis’, Written Communication 6(4), 463–479. Vygotsky, L.S.: 1987, ‘Thinking and speech’, in R.W. Rieber and A.S. Carton (eds.), The Collected Works of L.S. Vygotsky, Plenum Press, New York, pp. 39–243. Webb, N.: 1979, ‘Content and context variables in problem tasks’, in G.A. Goldin and C.E. McClintock (eds.), Task Variables in Mathematical Problem Solving, ERIC Clearinghouse for Science, Mathematics and Environmental Education, Columbus, OH, pp. 69–102. (ERIC Document Reproduction Service No. ED178366) Retrieved July 22, 2003, from the ERIC database.

University of North Carolina Charlotte, 9201 University City Blvd. Charlotte, NC 28223-0001, Telephone 704-593-1283, Fax 704-687-3410, E-mail dkpugale@email.uncc.edu