This action might not be possible to undo. Are you sure you want to continue?

1007/s11409-008-9026-0

Multi-method assessment of metacognitive skills in elementary school children: how you test is what you get

Annemie Desoete

Received: 14 September 2006 / Accepted: 16 June 2008 / Published online: 8 July 2008 # Springer Science + Business Media, LLC 2008

Abstract Third grade elementary school children solved tests on mathematical reasoning and numerical facility. Metacognitive skillfulness was assessed through think aloud protocols, prospective and retrospective child ratings, teacher questionnaires, calibration measures and EPA2000. In our dataset metacognition has a lot in common with intelligence, but planning measured with teacher ratings plays a role above and beyond IQ. Moreover, we found that skills are generally related, but that it is more appropriate to assess them separately. In addition, results show the value of an experienced teacher as actual measure of metacognitive planning skills. Our dataset suggests convergent validity for prospective and retrospective child ratings, but no significant relationship with the other metacognitive measures. Metacognitive skillfulness combined with intelligence accounts for between 52.9% and 76.5% of the mathematics performances. The choice of diagnostic instruments highly determines the predicted percentage. Consequences for the assessment of metacognitive skills are discussed. Keywords Metacognition . Assessment . Teacher ratings . Think aloud protocols . Prospective questionnaire . Retrospective questionnaire . Child

Introduction This study is devoted to the multi-method assessment of metacognition in elementary school children. The relationship between mathematical problem solving and metacognitive skills is described in young children. In the study it is investigated if prospective (e.g.,

A. Desoete (*) Department of experimental clinical and health psychology, Ghent University, Henri Dunantlaan 2, 9000 Ghent, Belgium e-mail: anne.desoete@Ugent.be A. Desoete Arteveldehogeschool, Ghent, Belgium

Desoete Vermunt 1996).g.g. 2002) techniques or teacher ratings can be used for elementary school children. 126–5= …). units in mental mathematics). division of the hundreds. Planning Planning skills make children think in advance of how. when. Metacognition was found to be instrumental in challenging tasks in mathematics. proper learning characteristics and the available time. Planning involves in a classroom context analyzing exercises (e. retrieving relevant domain-specific knowledge and skills (e. The four metacognitive skills investigated Prediction One of the metacognitive skills is prediction.. Moreover. Metacognitive knowledge can be described as the knowledge.g.. 126:5= …) from the easy ones (e. most authors agree that the construct can be differentiated into a knowledge and skills component (Lucangeli et al.190 A. in order to be able to concentrate on and persist more in the high-effort tasks. retrospective (e. Monitoring Monitoring skills can be described as the self-regulated control of used cognitive skills during the actual performance. In addition prediction makes children relate certain problems to other problems.g. tens. (e.g. children estimate or predict the difficulty of a task and use that prediction metacognitively to regulate their engagement related to outcome and efficacy expectation..g.. in order to identify problems and to modify plans. division of the hundreds. and deeper understanding of one’s own cognitive processes and products (Flavell 1976). prediction refers to activities aimed at differentiating difficult exercises (e. monitoring and evaluation skills (e. Metacognitive knowledge and skills Since Flavell introduced the concept of metacognition in 1976.. Artzt and Armour-Thomas 1992) and combined (e. tens. units in mental mathematics). Sperling et al. on-line (e. Artelt 2000).g. develop intuition about the prerequisites required for doing a task and distinguish between apparent and real difficulties in mathematical problem solving (Lucangeli et al. not overtaxing the capacity and skills of children and in relatively new skills that are being acquired (e. . The ability to predict enables children to foresee task difficulties in the classroom and makes them work slowly on difficult tasks and more quickly on easier tasks. 1995).g.. Prediction can be described as the skill enabling children to think about the learning objectives. and why to act in order to obtain their purpose through a sequence of subgoals leading to the main problem goal. Metacognitive skills can be seen as the voluntary control people have over their own cognitive processes (Brown 1987). 1998). A substantial amount of data has been accumulated on four metacognitive skills important for mathematics: prediction. planning.g.g. ‘it is a division exercise in a number-problem format’). how to do divisions) and sequencing problem solving steps (e.g. In mathematics.. 1998).. Carr and Jessup 1995). awareness. Lucangeli et al.

intelligence and metacognition (Veenman et al. a lot of diagnostic tools are designed to assess metacognition.g. Specifically children reflect on the outcome and the understanding of the problem and the appropriateness of the plan.. 2003. making use of the awareness of previously knowledge and selecting appropriate study behavior (Montague 1998). in which intelligence and metacognition are considered as two independent factors in the prediction of learning outcome (Swanson 1990). A global comparison is made of whether the evaluation after the task corresponds to the actual performance on the task. In prospective methods.. Evaluation (and calibration) A last metacognitive skill. Metacognitive assessment Nowadays.). The mainstream of those tools are addressed to assess the metacognition prospective or retrospective to specific arithmetical performances.g. intelligence and mathematics remains very unclear (Desoete 2007. Retrospective techniques. Vermunt 1996). such as self-report questionnaires and hypothetical interview. where intelligence and metacognition have a lot in common but metacognitive skills are more relevant than intelligence in predicting learning outcome in initial and complex learning situations (Demetriou et al. responsible for mathematics learning outcome (Sternberg 2001). Elshout-Mohr et al. . Metacognition and intelligence The relationship between metacognition. 1992). Proficient students are assumed to select appropriate skills and adjust behavior to changing task demands. whereby children look at what they did and whether or not this led to a desired result. ‘I ask myself questions to make sure I know the material I have been studying’) is representative of their behaviour (e.g.. 2005. Desoete et al. the evaluation skill. the execution of the solution method as well as on the adequacy of the answer within the context of the problem (Vermeer 1997). 1998). 2006). Veenman and his colleagues compared three models on the relationship between mathematics learning outcome. 2004. Evaluation makes children in the classroom evaluate their performance and compare task performance with people and use the final result in locating the error in the solution process (Lucangeli et al. in which metacognitive skills are considered as manifestations of intelligence and as part of the cognitive repertoire.. can be defined as the reflections that take place after an event has transpired (Brown 1987). Lin and Zabrucky 1998). both questionnaires and interviews have also been applied to assess metacognition (e.g. ‘Calibration’ can be defined in terms of whether the evaluated value corresponds with the occurrence of that value on the criterion test (e. ‘is this plan working?’ ‘should I use paper and pencil to solve the division?’ and so on. students have to indicate on a Likert-type of scale to what extent a statement (e. The first model is the ‘intelligence model’. Artelt 2000). To see the true contribution of metacognition to mathematical problem solving IQ should be used as covariate. A second model is the ‘independency model’.Multi-method assessment of metacognitive skills in elementary school… 191 Monitoring is related in a classroom context to questions such as ‘am I following my plan?’. A third model is a combined model.

. Third. Furthermore teacher’s perception of students’ use of skills was found to be an important predictor of academic performances in children with learning disabilities (Meltzer et al. although there are studies showing that children can be strategic form early age (Perry 1998. younger than 11–12 years. 2005). reviews indicate that teachers’ judgments can serve as worthy assessments of students’ achievement-related behaviors triangulated with data gathered by other protocols (Winne and Perry 2000).. Artzt and ArmourThomas 1992. Multi-method techniques seem indicated to get a good picture of metacognitive skills (Veenman 2005. The second aim of this study is to investigate whether concurrent and prospective and retrospective child-questionnaires differ in the assessment of metacognitive skills in young children. Research comparing types of measures of metacognitive skillfulness in children younger than 12 years of age is relatively limited. Veenman and his colleagues seem sceptic and point to the lack of accuracy and the limited explained variance of learning outcomes of prospective and retrospective assessment methods. the present study aims to add to the body of knowledge concerning the relationship between on-line and off-line metacognitive skills and mathematical problem solving in young children. several problems emerge in the assessment of metacognition making study outcomes difficult to compare (e. Whitebread et al. concurrent assessment. Although some researchers question the trustworthiness of teacher questionnaire data. Veenman et al. such as think-aloud protocols can take place. it is investigated in this paper if a teacher questionnaire can be used to get a good picture of metacognitive skills in elementary school children. In thinking-aloud protocol analysis participants are instructed to merely verbalize their thoughts during task performance. Huet and Marine 1998. the present study aims to add some insight to the value of teacher ratings on metacognitive skills of elementary school children. On the other hand. Despite all the emphasis on metacognition. Therefore we focus on the teacher ratings of metacognitive skills in young children. such as selfreport questionnaires. In addition to prospective and retrospective techniques.g. such as thinkaloud protocols were found to be accurate but time-consuming techniques to assess metacognitive skills. Because multi-method assessment is extensively time-consuming. concurrent-assessment techniques. On the one hand. 2006). To investigate the role of metacognition above and beyond intelligence. all analyses will be run with IQ as covariate. It is hypothesized that prospective and retrospective off-line questionnaire will have higher correlations than off-line and on-line or combined assessment techniques. It is hypothesized that the teacher questionnaire will correlate with the other assessment techniques. According to previous literature the prediction of mathematics by means of on-line techniques will be better than the prediction by off-line (prospective and retrospective) techniques. Pressley 2000). In the current study we narrow our research to three major aims. First. 1998). Aims of the present study The short overview clearly shows that there is scant research concerning the assessment of metacognition.192 A. Desoete An obvious problem with retrospective assessment questionnaires is the risk of memory distortions due to the time lag between the actual performance of problem solving and the verbal reports afterwards.

24) on WISC-III. Ethnic breakdown of the sample was 25% Asian. the mean performance IQ was 101. 10% African and 65% Caucasian. Dowker (2005) differentiated between two domains: ‘mathematical reasoning’ and ‘numerical facility’. Twenty children in their third year of this elementary school participated in this study.59 months (SD=3.33). The mean full-scale IQ of these children was 101.Multi-method assessment of metacognitive skills in elementary school… 193 Method Participants Since it is time consuming to assess children with different metacognitive instruments. The female teacher had been teaching third graders for 20 years in the same school. brain damage. The Arithmetics Number Facts Test (TTR) is often used to measure numerical facility in Belgium. Teacher consent was requested and given.27 months).63 (SD= 12.90). Studies with a longitudinal design and larger sample are currently being prepared. At the time of testing. the participants had a mean age of 99. native Dutch-speaking children. For the implications of the sample size and typicality of the teacher we refer to the discussion section. Measures Teacher ratings. All participants were fluent Dutch-speakers without histories of extreme hyperactivity. different mathematic tests and metacognitive tests were used in this study (see Table 1). insufficient instruction or serious emotional or behavioural disturbance. sensory impairment.60 (SD=8. Mathematics tests Initial mathematics can be seen as a broad domain of various computational skills. the mean verbal IQ was 102. Participants were 13 girls and seven boys. following regular elementary education in Flanders for more than 2 years. we started this study within a cross-sectional design in one school in an urban big town in Flanders. Informed consent from a parent of each participant was obtained before starting this study. The Kortrijk Arithmetic Test (KRT-R) is often used to measure mathematical reasoning. Table 1 Assessment instruments compared Mathematics KRT-R TTR Prospective test Retrospective test CDR Teacher Rating Thinking Aloud EPA2000 X X X X X X (X) X X X X X X X X X X X X X X X X X X Prediction Planning Monitoring Evaluation Calibration . In this study the school was purposefully selected out of ten schools on behalf of the long experience of the teacher and the verbal fluency of the children. a chronic medical condition.03 (SD=7.

129+878=_) and number knowledge tasks (e.g. KRT-R and TTR. on-line and combined techniques. In this study the Prospective Assessment of Children (PAC or prospective test) and Retrospective Assessment of Children (RAC or retrospective test) were used as off-line ratings for children. Metacognition was prospectively assessed one day before the real experiment. The Evaluation and Prediction Assessment (EPA2000) was used as combined (prospective and retrospective) assessment. retrospective test.059 children (Ghesquière and Ruijssenaars 1994). For the subscales Cronbach’s α were 0.75 (nine items). add three tens to 61 and you get _).. The given answers all referred to the constructs in question. The psychometric value of the KRT-R has been demonstrated on a sample of 3. Children were videotaped during word problem solving and fulfilling the prospective test. Children have to indicate before solving any mathematical problem on a seven-point Likert-type of scale to what extent a statement (e. The Arithmetic Number Facts Test (Tempo Test Rekenen.g. Metacognitive tests Metacognition can be assessed with off-line (prospective and retrospective).0. 5×9=_). The teacher was interviewed after the completion of the teacher rating. Teacher ratings were used as off-line rating for teachers.74 (ten items) . planning. Desoete et al. think aloud protocols..g. planning. All metacognitive and mathematics instruments were tested in a previous studies in order to determine the usefulness for this age group and for the sensitivity in measuring individual differences (e. Analyses showed that children. EPA2000.194 A. CDR. 0. 0. Observers reviewed the videotape of their performance and were asked why they performed that way and what they thought while performing the task. 2001) for this research line.. Desoete The Kortrijk Arithmetic Test Revision (Kortrijkse Rekentest Revision.99). This means that the study was conducted on a relatively low-achieving school. monitoring and evaluation skills. Children have to indicate on a seven-point Likert-type of scale to what extent a statement . This means that the children performed average on numerical fact retrieval.76 (SD=31. Thinking-aloud protocol analysis (TAP or think-aloud protocols) were used as on-line technique. the children in this study achieved a standardized mean percentile score of 55. 1992) is a numerical facility test which requires children in grade 1 to solve as many number fact problems as possible within 5 minutes (e.55 (four items). 2006) is a Belgian test on arithmetic reasoning which requires children to solve mental arithmetic (e.246 Dutch-speaking children from grade 1 till 6.. In the study the standardized total percentile based on Flanders norms was used..70 (two items) for prediction. monitoring and evaluation respectively. teachers and observers/ coders could handle the instruments well.81 (26 items). de Vos.g. ‘I control exercises I make’) is representative of their behaviour during mathematical problem solving (1=never. On the KRT-R the children in this study achieved a standardized mean percentile score of 39. 2001.59). On the TTR. 7=always). The Retrospective Assessment of Children is the same 25 item rating scale questionnaire for children on metacognitive prediction. KRT-R. The psychometric value has been demonstrated for Flanders on a sample of 10.g.37 (SD=5. Off-line techniques The Prospective Assessment of Children is a child-questionnaire adapted from the MSA (Desoete et al. Participants solved the Cognitive Developmental Arithmetics Test (CDR) as an off-line calibration rating for children. Baudonck et al. is a 25 item rating scale questionnaire for children on metacognitive prediction. TTR. planning. In this study Cronbach’s alpha for the scale was 0. monitoring and evaluation skills. 2006).

‘I controlled exercises I made’) was representative of their mathematical problem solving behaviour on the past task (1=never.. here 60) and the macro-evaluation or performance calibration score (e. The psychologist practiced this rating procedure in advance on several other participants not included in the sample until she felt confident that an adequate level of rating fluency was reached.g.89 (25 items). the child never (1)/always (7) knows in advance whether an exercise will be easy or difficult). 0. The larger ‘misses’ (ten is more then eight) are misses. –10). Only the activities where all four persons agreed upon the concept and operationalisation. planning. monitoring and evaluation respectively. Gutmann’s split-half and Spearman–Brown’s coefficients were 0.. all variables were normally distributed and test–retest correlations were 0..91 and 0. The absolute difference between the mathematics performance (e.g.g.64 (two items) for prediction. respectively. Desoete and Roeyers 2006) is a 90-item test developed for the assessment of cognitive and calibration skills of young children.g. Based on a review of metacognition literature in this age group metacognitive activities were inventoried by three psychology students and by the main researcher. In this study Cronbach’s α of 0.g.. monitoring (six items) and evaluation (three items) skills (e..01) were found. 0. In the case children fall silent.g. The score children attribute to their work (e.81 (nine items).g. On-line technique Thinking-aloud protocol analysis was applied during three word problem solving tasks.Multi-method assessment of metacognitive skills in elementary school… 195 (e. 0. Test–retest correlations of 0. In this study Cronbach’s alpha for the total scale was. planning. planning (four items).99 and 1. Metacognitive ratings were assessed by the psychologist concurrent with the participant’s ongoing process of solving math problems. The Teacher Rating. 7=always). ‘I think I will obtain 70/90 on this test’). Furthermore. which was created for this research line. The teacher questionnaire was tested in previous studies in order to determine its construct validity (Desoete et al.g. Children in this study achieved a standardized mean percentile score of 52. 70) is the calibration score (e.g.97. children have to gauge confidence in the correctness of the given answers (e.. 60/90 on the test). here 10).g. Psychometric value has been demonstrated on a sample of 483 Dutch-speaking children in Flanders. Furthermore teachers scored the mathematical and reading performances as well as the intelligence of children [e.ex. is a 20 item rating scale teacher-questionnaire on metacognitive prediction (seven items). +8) or negative (f.. Children were merely instructed to verbalize their thoughts during three word problem task performance. 0.01) and inter-rater reliabilities varying between 0. here 10). To examine the psychometric characteristics of the developed metacognitive parameter..52 (four items). here −10) is changed in an absolute difference score (e. 70) is the difference score (e.89.. All protocols ..90 for prediction.40/80 (SD=10.77 (ten items).72. The difference between the mathematics performance (e.g. the assessor urged them ‘to keep on thinking aloud’. The number of correct answers is the mathematics performance score (e. For the subscales Cronbach’s α were 0. For the teacher rating subscores Cronbach’s α were 0. were included in the study. here 70) is the macro-evaluation or performance calibration score.70 and 0. In addition. students were observed and videotaped during and interviewed after the test. monitoring and evaluation respectively..04). This thinking-aloud prompts children to think aloud during the whole problem solving.ex.81 (p <0. here 60) and the macro-evaluation or performance calibration score (e.. 2001).00 (p <0.g.g.85 (p <0. regardless of whether they are positive (f. very low compared to peers (1)/ very good compared to peers (7)]. In addition..0005) in a previous study (Desoete and Roeyers 2006). The Cognitive Developmental Arithmetics Test (CDR. 0.98 was found for the test score (20 items).

Desoete were transcribed verbatim and analyzed according to this metacognitive coding scheme on the presence these 40 activities derived from grounded analysis in previous studies in this age-group (e. Metacognitive predictions or evaluations were awarded two points whenever they correspond to the child’s actual performance on the task (predicting or evaluating 1 and doing the exercise wrong and rating 4 and doing the exercise correctly). checking calculation. checking the results. In the measurement of prediction skillfulness.75 for prediction. For each problem a metacognition score was calculated on the 39 activities. whether they will be successful in this task. reading he task again to comprehend better. selecting relevant steps. writing down what is already known. underlining important words.88 was found for the total protocol analyses. taking time designing an action plan. which led to hardly any modifications in scores. whereas a score of 1 was given if the activity was present. acting according to plan. 3 sure I am correct. reflecting. averaged later. 4 absolutely sure I am correct).82 and 0. orderly sequencing steps. 2000) is a computerized procedure for assessing mathematics. other behaviour that points in the direction of planning). 2 sure I am wrong. 17 monitoring items (adhering to plan.. For the subscales Cronbach’s α were 0.60. selecting the calculation needed. Areas of non-agreement (between the psychologist. Desoete et al. 2001. All scores were also independently coded and controlled by the author. Other answers did not gain any points. making correct use of unities and tens. monitoring and evaluation respectively. reflecting on a clear exact and precise answer. other behaviour that points in the direction of evaluation). reflecting on what went well and how the tasks were solved.g. checking the answer with the estimated outcome. other behaviour pointing in the direction of prediction). The three scores (prediction. mathematics and evaluation) were unrelated. prediction and evaluation. selecting relevant materials. It was also possible to give half a point if the activity was initiated but not completed. Children could give four ratings (1 absolutely sure I am wrong. monitoring problem-solving process. rating 1 or 3 received 1 point whenever they correspond. a total metacognition score was calculated over the three problems (with a Cronbach’s alpha of 0. Children had to evaluate after solving the different mathematical problem-solving tasks on the same four-point rating scale. One prediction-item was deleted form further analyses because there was not enough variance in the scores of the children on this parameter. correct in calculation. relating to other problems. 2002): 11 prediction items (reading the problem oriented on comprehension. two colleagues and the author) were discussed with reference to the definitions of the skills and were resolved through mutual consent.79. children were asked to look at exercises without solving them and to predict on a four-point rating scale. 0. orderly note-taking of problem solving steps. six planning items (selecting relevant data. other behaviour that points in the direction of monitoring) and six evaluation items (summarizing the answer and reflecting on the answer. estimating possible outcome. Finally. making a drawing. putting information together. 0. writing down what was asked for. The psychologist and two colleagues checked the ratings afterwards by replaying the tapes. drawing a conclusion referring to the task. resulting in three scores. De Clercq et al.196 A. not forgetting steps. selecting relevant information. Cronbach’s α of 0. relating to future problems. For . systematic activities.89). making notes relating to the problem. referring tot problem statement in the answer. planning. Predicting and evaluating. In line with Veenman and Spaans (2005) a zero was given if the activity was absent. Combined technique The Evaluation and Prediction Assessment (EPA2000. taking note of the precise answer. as they are considered to represent a lack of off-line metacognitive skillfulness.

The psychologist only urged them to continue thinking aloud whenever they fell silent with a standard prompt ‘Please keep on thinking aloud’. participants were provided no calculator but some blank sheets of paper for making notes. a psychologist. 2006). whatsoever. in theory a child could obtain maximum scores for prediction. where they completed the KRT-R (Baudonck et al.92. in a quiet room outside the classroom setting. During the mathematics task.Multi-method assessment of metacognitive skills in elementary school… 197 instance. The test-protocol was not included in the analyses of this study. The training took place 2 weeks before the start of the assessment. To be sure that children were able to make consistent decisions between being ‘sure I am wrong’ and ‘absolutely sure I am wrong’ or the reverse ‘sure I am right’ and ‘absolutely sure I am right’. No help or feedback. retrospective and on-line skill measures (research hypothesis 2). They all could answer convincingly. received practical and theoretical training in the assessment and interpretation of mathematics. prediction (with a max score of 160 points). . The average prediction score was 110. The examiner. Results Relationship between the teacher rating and other skillfulness measures To investigate the relationship between the teacher ratings and the assessment by other instruments (research hypothesis 1) and to establish the relationship between the prospective. for about 2 h in total. TTR (de Vos 1992) and the CDR (Desoete and Roeyers 2006).87) whereas the average evaluation score was 115. a zero score for mathematics and a medium score for evaluation. 2002). The intelligence of all subjects was assessed 3 months after the metacognitive and mathematics tasks. In order to guarantee reliability of the assessment. on two different days. was given by the psychologist.91. Cronbach’s alphas of 0. The prospective and retrospective task was performed at the beginning and at the end of a mathematics test (KRT-R or TTR).98). and evaluation (with a max score of 160 points) respectively.63/160 (SD=14. This protocol was analyzed and corrected by the main researcher of the study.89 were found for mathematics. For every instrument. Procedure All subjects were assessed individually.00/80 (SD=7. 0.73). In this study children’s mathematical skills on EPA2000 were 59. ongoing supervision and training was provided during the assessment of the first five children. children reviewed the videotape of their performance and were asked why they choose ‘sure’ or ‘very sure’. leading to correlations influenced by this effect. the psychologist had to test one child and score the protocol in advance. All participants were instructed to think aloud while solving the three math problems.95/160 (SD= 16. Children got no training on how to provide verbalization. a counterbalanced design was used. and metacognition. To prevent an unintended learning effect with the repeated metacognitive measures. The psychometric validity and reliability has been demonstrated on a sample of 550 Dutchspeaking third-graders (Desoete et al. They were capable to draw the fine line between ‘sure’ and ‘very sure’ and ‘very sure’ being correct referred to a higher level of prediction skillfulness. Pearson correlations were computed between the measures (see Table 2). Sheets for note taking were removed during the KRT-R and EPA2000. The regular teacher completed a teacher survey in the same period. and 0. instructions given to the children in relation to their think alouds and scoring rules were explained. Systematic.

01 .39** / / / / 0.67** 0.08 −0.20 −0.36** 0.46** 0.42**.39** −0.18 0.24 −0.10 0.14 0. −0.92** 0. **p <.22 0.35** −0.41** 0. Th Think. Retrosp.21 0.44** / / / / −0. prospective.42** Teacher teacher rating. Planning Retrosp. Evaluation Calibration/eval.05 / / / 0.09.19 −0.67** 0.aloud planning Th.35 0.25 0.05 0.24 0.25 / 0.20 0.10 0.55** −0.198 Table 2 Correlations among the metacognitive skills between the instruments Off-line Retrospective Off-line Calibration On-line Concurrent Combined EPA 2000 Intelligence Off-line Prospective Off-line tests Teacher prediction Teacher planning Teacher monitoring Teacher evaluation Prosp. eval.53** / / 0.20 0.19 −0.92** / / / / 0.25 0.73** 0.20 0. Evaluation Retrosp.42** / / 0. monitoring Retrosp. Desoete *p <. Planning Prosp.10 0.51** / / / 0.19 −0.56** −0.47** 0.73** 0.19 −0. On-line tests Th.14 / / 0.10 0. Monitoring Prosp.33 −0.08 0. retrospective.44** 0.40** 0.05 0.41** 0.22* −0.02 0.05 / / / / 0.48** 0.42** 0.10 −0.24 0.aloud Evaluation EPA prediction EPA evaluation −0.78** 0. Prosp. evaluation A.34 −0. Prediction Prosp.25* −0.25 0.33 / / −0. pred.78** 0.02 0.25 / / / −0.24 −0. Prediction Retrosp.15 0. prediction.02 / / 0.13 −0.54** −0.48** 0.55** −0.aloud monitoring Th.62** −0.aloud Prediction Th.16 0.13 −0.

Since all variables were normally distributed and did meet the assumptions for multiple regression. r =0.464% of the common variance with an eigenvalue of 2. p =0. planning. CDR and EPA2000 r =0. The second criterion as to the adequacy of a one component solution to our data set was that a one component solution accounted for 54.41. r =0.174. p =0. Planning and monitoring assessed with think aloud protocols correlated with the teacher opinion on the planning (resp. For example row 2 of Table 2 revealed that teacher ratings of prediction (TR prediction) correlated −0. Teacher ratings on evaluation skills correlated significantly with calibration. a principal components analysis and several regression analyses were conducted.42.551.Multi-method assessment of metacognitive skills in elementary school… 199 In the left hand column all types of ratings are included.41. p =0. Row 3 of Table 2 revealed that teacher ratings of planning (TR planning) correlated −0.34 with ‘planning’ retrospectively assessed. The first criterion was that there was only one component with an eigenvalue higher than l (Kaizer normalization).196 and TTR and EPA2000 r =0. In the top row of the chart the type of assessment is given compared with type of ratings presented in the column.19 with ‘prediction’ prospectively assessed.462.051) respectively. planning.310. p <0.668 (p =0. KRT-R and CDR r =0.39. r =0. Given the high intercorrelations between the mathematics subtest scores (KRT-R and TTR r =0.448.387) the internal structure of the mathematical data was first analyzed with a principal components analysis.01) and the results on EPA 2000 (r =0. retaining enough variance for an adequate fit but not so many that parsimony was lost. monitoring and evaluation r =0.01.210. 3.442 (p =0.01).001). Teacher ratings on planning and monitoring skills correlated significantly with the results on the think aloud protocols. monitoring and evaluation skills for mathematical problem solving To answer research question 3 and so establish to what extent the metacognitive skills were associated with mathematics performance in third grade.53 with ‘prediction’ assessed with EPA 2000. Component 2.611. r =0.005. p <0. Importance of prediction. 0. Think aloud protocols on prediction did not correlate significantly with other prediction measures.15 with ‘prediction’ concurrently assessed with think aloud protocol and 0.702 and 0.732 (p =0. p <0. regression analyses were conducted in the sample to evaluate how well the .55.19 with ‘prediction’ retrospectively assessed. to account for all the variance.01). p <0. the prospective child questionnaire (r =0.269 respectively) and were found not as important from a variance perspective. p <0.850. The correlations between prospective and retrospective assessment techniques were for prediction. think aloud protocols and with EPA2000. This analysis was carried out to develop a small set of components empirically summarizing the correlations among the variables. Evaluation assessed with think aloud protocols correlated significantly with the teacher rating (r =0. but not with the prospective or retrospective child questionnaire results on these skills.0005) and r =0.41 with ‘planning’ concurrently assessed with think aloud protocol and there was no measure of planning included in EPA 2000.007. and 4 had an eigenvalue of 0. Teacher ratings on prediction skills correlated significantly with EPA 2000 prediction skills. This initial number of four could be reduced to one.0005). KRT-R and EPA2000 r = 0. p =0. This one component solution was based upon two criteria. CDR and TTR r =0. Moreover teacher ratings of planning correlated 0. p =0.778 (p =0.01) skills of pupils. Four components were needed to account for all the variance in our dataset. Teacher ratings of prediction also correlated −0.20 with ‘planning’ prospectively assessed. Teacher ratings of planning also correlated −0.01.179. Moreover teacher ratings of prediction correlated 0. p =0.

a regression analysis was performed on the mathematical component as outcome variable with intelligence and the sub scores simultaneously as predictor variables. 19)=6. β = −0.206. Prediction (B = −0. planning Table 3 Prediction of mathematics component by teacher ratings Mathematics component Metacognitive skills teacher ratings Constant Intelligence Prediction Planning Monitoring Evaluation *p ≤ 0. Adjusted R2 was 0.502 0.003) in the expected way.731 4.001* 0. There was a trend indicating that think aloud protocol assessing monitoring skills (B =0.092.000 0.573.565 0.093) distributed to the variance in mathematics learning. Adjusted R2 was 0.188 β t −5. 19)=13. p = 0. p =0.180 −0.901. Again only intelligence was beneficial in the expected way for the variance in mathematics performance of third grade children. Especially intelligence and teacher ratings of planning were beneficial for the variance in mathematics performances of third grade children. The metacognitive skills and intelligence were included simultaneously as predictor variables.003. p =0.259. monitoring and evaluation off-line measured by teacher ratings was significantly related to mathematics component.091 −0.902 −0.248.608. t =1. and F(5.200 A.689 −1.608 P 0. Off-line measures: prospective and retrospective child questionnaires A multiple regression analyses pointed out that the linear combination of the intelligence and the metacognitive skills measured by prospective child ratings was also significantly related to mathematics performances.385.529 and F(5. On-line measures To establish to what extent the mathematics performance was associated with metacognitive skills measured by think aloud protocols. Due to limited power the analyses were not combined in one analysis. A third multiple regression analyses pointed out that the linear combination of intelligence and the metacognitive skills measured by retrospective child ratings was also significantly related to mathematics performances.521 3.399 0. Off-line measure: teacher ratings The linear combination of intelligence assessed and prediction.545. Intelligence was a significant predictor (B =0.05 Unstandardised coefficients −9.682.975 . F(5. t =3.130 0. F(5.412 −0. planning.638 −0.207. Adjusted R2 was 0.610 0. p =0. β=0.002. Only intelligence was beneficial in the expected way for the variance in mathematics performance of third grade children (see Table 4).151 1.040 0.356.006. β =0.077 0.002* 0.007.093. p =0. Adjusted R2 was 0. Desoete metacognitive skills measured by different assessment techniques predicted the mathematics component in grade 3. t = −1.066 0. p =0. p =0.765 (see Table 3).800. 19)=5.569. 19)=6.247).

001 −0. However comparisons revealed .592.040 β t p 201 0.375) assessed by think aloud protocols in third grade children were no significant predictors for mathematics performances of these children.320 −0. p =0. whereas prediction (B =0.917.442. t =3.046 −0. The correlations highly depended on the technique used to assess metacognitive skills.449.087 −0.078 −0. Differences between below-average.816 0.027. p =0. except between evaluation and prediction or planning prospectively measured or assessed with think aloud protocols.081. p =0.373 0.056 0. A MANCOVA was performed on these metacognitive subskills and mathematics-performance groups (based on the component matrix) in order to test if below-average. 17)=3.202.083 0. Combined measures To investigate to what extent the mathematics component score was associated with intelligence and with the metacognitive prediction and evaluation skills of the EPA2000.602 (B = −0.843) had no significant additional predictive value for the mathematics component score.531 −0.058 −0.072 −0. 19)=10.027* 0.186 −0. β = −0.313) and evaluation (B =0. There was a high intercorrelation amongst the teacher rating and EPA2000 sub scores.004.072 −0.149.373 −0. p =0.639 −0. β = −0. Intelligence (B =0.121 0.531 2.866 −0.534 0.061.067 β T −1. Correlations among metacognitive skills The correlations between the metacognitive skills in the divergent instruments were computed (see Table 5).127 0.545.788.601 and [F(3.167) and evaluation (B = −0.066 −0.237 −0.061.073]. Positive and significant relations were found between most skills.303 −0.018 −0. Preliminary comparisons revealed that the children in the three conditions did not differed significantly in TIQ [F(2. average and above-average participants differed on metacognitive skills. p =0. a regression analyses was conducted on the mathematical component as outcome variable and intelligence as well as the EPA2000 metacognitive sub scores simultaneously as predictor variables.584 0.949 0.159 0.002) was beneficial. p =0. β =0.465 −0.186.002 −0. Adjusted R2 was 0.005* 0.005 −0. average and above-average performers Average ratings across the metacognitive subskills were computed.969 3.028 −0. β = 0.090 −2.069 0.013 −0.Multi-method assessment of metacognitive skills in elementary school… Table 4 Prediction of mathematics component from prospective and retrospective questionnaires Mathematics component Prospective questionnaire Unstandardised coefficients Constant Intelligence Prediction Planning Monitoring Evaluation *p ≤ 0. t = −1.950 0.414. t =0.921 p Retrospective questionnaire Unstandardised coefficients −5. t = −0.047.843 −1.930 0. t =1. p =0.000].05 −5.414 0.217. β =0.

69* 0. For M and SD we refer to Table 6.27]. different methods to assess metacognition have been used (Tobias and Everson 2002).92* 0.3% by EPA2000.13 0. It is investigated if prospective. teacher ratings. power=0.9% by the think aloud protocols and for 15. Overall. for 36.12 / / 0. 11)=1. As predicted the teacher ratings on prediction skills correlated positively with the combined assessment by EPA2000 but not with the child questionnaire. p =0.89* On-line Thinking aloud Combined EPA2000 Thinking aloud = think aloud protocol.95* / 0. Prospective test = prospective assessment of children by questionnaire.09 −0. for 51% by the retrospective questionnaire.62* / 0. As expected the teacher questionnaire on evaluations skills also correlated positively with the concurrent and combined assessment techniques. We also focused on teacher ratings to investigate if such a questionnaire can have some value added in the assessment of metacognitive skills of young children. Retrospective test = retrospective assessment of children by questionnaire *p <0. The MANCOVA with summed EPA2000 results. The performance group was predicted for 40.172.91* 0.51* / / 0.63] and no significant effect for intelligence [F(6. the results clearly confirm the value of ratings of an experienced teacher as actual measures of metacognitive skills in elementary school children.202 Table 5 Correlations among the metacognitive skills within the instruments Planning Off-line Prospective test Monitoring A.427. retrospective scale and Thinking Aloud scale as dependent variables and intelligence as covariate revealed.26 0.9% by the teacher ratings. This study is devoted to the multi-method assessment of metacognition in elementary school children. on-line and combined (EPA2000) techniques can be used for elementary school children.1% by calibration results.08 0. for 29.4% by the prospective questionnaire.96* 0.53* 0. p =0.01 that above-average participants outperformed (although not significant) the two other groups on full-scale intelligence.34 / 0.98* 0. power=0.086. Desoete Evaluation Retrospective test Teacher Rating Prediction Planning Monitoring Prediction Planning Monitoring Prediction Planning Monitoring Prediction Planning Monitoring Prediction 0.02 0.58* 0. teacher ratings correlated significantly with prospective child .29 0. perhaps due to a very limited power no significant differences on the multivariate level for the performance group [F(12. Discussion Since Flavell introduced the concept 30 years ago. 22)=1.575.67* 48 0. Moreover.84* 0. for 32.12 / 0. retrospective. calibration. prospective scale.55* / / 0.90* / / 0.

The next research question focused on the extent to which metacognitive skills were associated with mathematics performance in third grade.201)b 134. Concerning the usefulness of the skills in mathematical performances.259) (21.061 F(2.94) 61.148** F(2. Our findings might suggest that metacognitive ‘prediction’ and ‘evaluation’ skills add noting in prediction performance above that accounted for by IQ in almost every measure and test.45 99.948 2. CDR mathematics test also used for calibration.16) 3.Multi-method assessment of metacognitive skills in elementary school… Table 6 Results of children with different mathematic skills in grade 3 Below-average performers M (SD.817) (9. consistent with Veenman’s literature review no significant correlations were demonstrated between the child questionnaires and the other techniques. This is in line with previous findings (Desoete et al. The study revealed that the choice of diagnostic instruments highly determined the predicted percentage.40 (8.619)a (3. In our dataset especially intelligence predicted mathematics. N =8) 99.75 54.556** 2.20 67.60 (17.003)a (7.37 (4.05 questionnaires and with measured intelligence. In our study there seemed to be a fairly consistent mindset that was not much influenced by student’s actual performances.29 (6. planning.62 51. Most targeted metacognitive variables did not predict scores on the mathematics component after IQ was covaried out of the picture.449)b (26.00 13.558)c 10. These results are in line with the ‘intelligence model’ in which metacognitive skills are considered as manifestations of intelligence.516** 4. test (min 0–max 175) Retrosp. in .75 28.289)b Above-average performers M (SD.37 83.164) (5. 2003) that metacognitive skills need to be taught explicitly in order to develop.71 38.80 17.457) (26. teacher rating.85 58.598** 3.16) 214.14 (14.20 1.86 (3. Teachter Rat. test (min 0–max 175) Teacher Rat.14 47.584** 4. However.494) (12.847)a 73.057* 5.25 (9.120)b Average performers M (SD.072* 6.60 101.test prospective child rating.536)a (6.178)b (18.13 111.428(17. Think aloud think aloud protocol *p ≤ 1. As expected. Prosp. 2001) this is a relevant finding.014)b (6.892)a (30. N =5) 107. Retrosp.05 TTR numerical facility.370)a (26.42 217. In addition.184)b 3.278* 0. (min 0–max 140) Calibration (min 0–max 90) Think aloud (min 0–max 40) 98.test retrospective child rating.492) (5. our dataset suggests convergent validity for the prospective and retrospective ratings of prediction. (min 0–max 80) Metacognitive skills EPA2000(min 0–max 320) Prosp. Furthermore especially the rating of an experienced teacher on planning was associated with mathematics performances.071) (16.14 (16. monitoring and evaluation skills.78) 65.043) (24.616* abc: different indexes refer to significant between-group differences with a significance level of 0. KRT-R mathematical reasoning.62 107. especially planning measured by teacher questionnaires could be regarded as the best estimates of variance.60 67.123)b (13.00 258. resulting in similarity of ratings before and after mathematical problem solving.96) 82.62 (11.17) 3.40 104.346)b (4.80 60.12 (13.125 8. Another relevant issue concerns the relationship assessment techniques of metacognitive skills.702) (7.43 (13.246)b (9. They cannot be assumed to develop from freely experiencing mathematics.180) 14. Since in elementary school children a combination of prediction and evaluation skills explains a substantial amount of variance in mathematics (Desoete et al. **p ≤ 0. N =7) Intelligence Mathematic skills TTR (min 1–max 100) KRT-R (min 1–max 100) CDR (min 0–max 90) EPA math.636)a 139.727)a 203 F(2. EPA2000 mathematics test also used for prediction and evaluation.30)a (14.

A. However. (2000). Teacher questionnaires seem to give additional valuable information on the planning skills of third grade children.204 A. (2006). Hillsdale. E.. References Artelt.. Kortrijk: CAR Overleie. a chronic medical condition. It might be also interesting to look how other teachers can reach such a level of performance. Metacognition. (1992). Dewulf. C. Brown. Additional research should focus on these factors. A.. & Desoete. First it should be acknowledged that sample size is a limitation of the present study. self-regulation.. 14. Obviously sample size is not a problem for significant correlations or regressions. Finally. motivation and understanding (pp. Baudonck. doi:10. F. Kluwe (Eds. or a teacher who has not been in the same school for 20 years might lead to other results. brain damage. 9. expertise. In addition metacognitive skills may be age-dependent and still maturing. A. Additional research with larger groups of children is indicated. In F. a teacher who teaches different grades. B. Debusschere. H. the number of other possible causes of low mathematical functioning (language problems. Second. Such research is recently being planned. Reflecting on the results of the present study there is evidence that how you test is what you get.. V. Our dataset certainly seems to indicate that in studies on metacognition related to mathematical problem solving IQ should be used as covariate. familiarity and knowledge related to the students and their metacognition might be placed at the exemplary end that runs from novice to expert. In Think aloud protocols a high relationship was found between monitoring and prediction and planning. E. Artzt. have to be noted as limitations of this research. the results of this study should be interpreted with care since the analyses are based on a single teacher. It might be so that a less experienced teacher. a risk of type 2or β-mistakes (concluding from the cohort that there were no differences although in reality there were differences in the population) can not be excluded. M. (1987). sensory impairment. retrospectively. concurrently and assessed with EPA 2000 and with a calibration approach. We suggest that researchers who are interested in skillfulness in young children use multiple-method designs. A. when analyses have insufficient power and were not significant. executive control.). These results should be interpreted with care. . Wie prädiktiv sind retrospektive Selbstberichte über den Gebrauch von Lernstrategien für strategisches Lernen? German Journal of Educational Psychology. & Armour-Thomas. We found that skills are generally related. causing a limitation in the random sampling. Samyn. Such studies are recently being planned.1207/s1532690xci0902_3. and other more mysterious mechanisms. Desoete line with the ‘combined model’ metacognitive ‘planning’ measured with teacher ratings played a role above and beyond IQ. Metacognition. hyperactivity. & R. planning. 137– 175. but that it is more appropriate to assess them separately. Weinert. 72–84. De Kortrijkse Rekentest Revision KRT-R. Moreover.. serious emotional or behavioural disturbance) was restricted to a minimum in this study. monitoring and evaluation skills rated by the teachers and between the prediction and evaluation skills assessed by EPA2000. Vercaemst. There was a positive correlation between evaluation assessed prospectively. 65–116). Additional research is needed with less experienced and perceptive teachers in observing and rating their students. The evaluation skill seems relatively independent in prospective child ratings and think-aloud protocols. insufficient instruction. NJ: Erlbaum. [The Kortrijk Arithmetic Test Revision KRT-R]. since there are some limitations to the present study. Her ability. including teacher questionnaires. These restrictions. Cognition and Instruction. Development of a cognitive-metacognitive framework for protocol analysis of mathematical problem solving in small groups. our dataset revealed very high intercorrelations between prediction. F.

& Computers. 2de en 3de graad. M. R.1998.. H. 711–726.. 262–296). Padova. Lin. Journal of Educational Psychology. A self-report inventory for metacognition related to academic tasks. Metacognitive skills in Belgian third grade children (age 8 to 9) with and without mathematical learning disabilities. A. M. 23. In T. Mastropieri (Eds. Demetriou. Perry. neuroscience and education. doi:10. Scruggs. (2006). M. (2003).). Ghesquière. Greenwich: JAI. Learning styles. Learning Disabilities Research & Practice. Roeyers. J. L’ Annee Psychologique. T. Houser. In A. Lucangeli. Huet. Reliability & validity proceedings of the 7th Annual ELSIN Conference Ghent University. 1(2). D. (1998). P.. Gent: Academia Press Scientific Publishers. Issues in the measurement of metacognition (pp. & A. 95(1). Vlaamse normen voor studietoetsen Rekenen en technisch lezen lager onderwijs. R. EPA2000: A multilingual. A.. Montague. NJ: Erlbaum. Sperling.1016/1041–6080(95)90012–8. (2000). A.. Meijer. C. 247–260). 32. Individual differences in arithmetic.). (1998). Behavior Research Methods. & Roeyers... & W. L’ Annee Psychologique.A. abilities.O. Assessment of metacognition II. Italy.. & Platsidou. C. Assessment of metacognition I—Independent measures. (2007).. & Huylebroeck. A. 5(3). A. & M. Lucangeli.1.1177/002221940103400505. (1995).1091.1006/ ceps. (1998). 11–21. Metacognition and Learning. and developing expertise: what makes an expert student? In H. (2000).95. Advances in learning and behavioral disabilities (pp. (1998). De Vos. & Cornoldi. Implications for psychology. In T. In L. Roeyers. Evaluating and improving the mathematics teaching-learning process through metacognition? Electronic Journal of Research in Educational Psychology.1998. Can off-line metacognition enhance mathematical problem solving? Journal of Educational Psychology. (2002). 119– 135.. D. Cognitive and metacognitive predictors of arithmetics strategy use. L.90. H. A. 437–451. Herenthals: Vlaamse Vereniging voor Logopedisten (VVL). 219– 285). A. Structural systems in developing cognition. Rekentests voor 1ste. Desoete. & Roeyers. Resnick (Ed..2001. H.S. L.. Belgium (pp. & J. 98. E. & Ruijssenaars.. Sternberg. & Zabrucky. (1992). (1998). A.. 34. Research on metacognition in special education. (2002). C.. Flavell. Gombeir. & M. 235–247.. Hartman (Ed.. Meltzer. Development of grounded theories of complex cognitive processing: Exhaustive withinand between study analyses of thinking-aloud data.).. Schraw. Specific and general transfer effects following metamemory training. Shayer. Contemporary Educational Psychology. Metacognition and learning disabilities in mathematics. 7. Desoete. doi:10. Tempo-Test-Rekenen. R. The measurement of individual metacognitive differences in mathematical problem solving. 151). (1976)...).). A. doi:10. Journal of Learning Disabilities. 27 . Metacognition. D. Hillsdale. Lincoln. De Clercq.. Demetriou. Miller. K. & De Clercq. 10. A. programmable computer assessment of off-line metacognition in children with mathematical-learning disabilities. Concurrent measures. M. A. Psychology Press: Hove. F.1007/s11409–006–8152–9. Perceptions of academic strategies and competence in students with learning disabilities. 98. 304–311. J. Instruments. 231–235). Desoete. M. Advances in learning and behavioural disabilities (p. H. H. N.. M.-C. 93–102). E.4. Metacognition in Learning and Instruction (pp. The nature of intelligence (pp. 51 – 79. (1998). 705–730. Measures of children’s knowledge and regulation of cognition. 90...0972. Dordrecht: Kluwer Academic Press. Roeyers. Berkhout: Nijmegen.. (2001). Young children’s self-regulated learning and context that support it. Roeyers. Pressley. science. (2005). C. A. Contemporary Educational Psychology. Elshout-Mohr.3406/psy. Efklides (Eds. Neo-Piagetian theories of cognitive development: Implications and applications for education (pp. W. N. B. (2006). 26–30 August 2003. Howard. A. A. & Murphy. doi:10. & Huet.Multi-method assessment of metacognitive skills in elementary school… 205 Carr. N. Metacognition and mathematical problem solving in grade 3. 435–449. C. M. (1994).715. doi:10. UK. San Diego: Academic. & Marine. Gustafsson. and education. L. van Daalen-Kapteijns. Metacognitive aspects of problem solving..E. In M. C. A.U. 727–742. Desoete. & De Clercq. Impara (Eds. H.. A. J. Scruggs. D. 345–391. (1995). 31. 715–729. (1998).B. & Tellarini. M. C. L. Paper presented at the 10th Conference of the European Association for Research on Learning and Instruction (EARLI).. M. Cornoldi. Mastropieri (Eds. H. Desoete. C. Calibration of comprehension: Research and implications for education and instruction. NE: Buros Institute of Mental Measurements. A. Desoete.188.28566. M. & Perlman.L. D. (2001). Dowker. doi:10. Efklides.1006/ ceps. 79–103). Desoete. Smith (Eds. Marine. Leuven: K. Journal of Learning Disabilities. London: Routledge. & Buysse. & Jessup.. 188–200. Cognitieve Deelvaardigheden Rekenen (CDR). M. J. doi:10.. Galderisi. Roditi. & Meeus. B. Learning and Individual Differences. A. E.1037/0022–0663. Valcke.1037/0022–0663.). . (2003)..). doi:10. In G. B. A.. (1992).

Metacognition in young children: Evidence form a naturalistic study of 3–5 year olds.). J.learninstruc. Tobias. Metacognitive. (1990). (1997). Veenman. H.. & Beisheuzen.206 A. Kok.. doi:10. Boekaerts.lindif. & M. V. The relation between intellectual and metacognitive skills in early adolescence. & Everson. & Blote. J. M. A. (2002). Metacognition and learning. P. 82. L. Learning and Individual Differences. doi:10. M. The assessment of metacognitive skills: What can be learned from multi-method designs? In C. E. 14. Winne. . D. D. Vermunt. Measuring self-regulated learning. Veenman. 159–176. Zeidner (Eds. J.1016/j. Vermeer. E. P. H. (1996). Relation between intellectual and metacognitive skills: Age and task differences. P. J. 77–99). Metacognition and Learning.. doi:10. H. Nicosia. In M. J. (2000). M. (2005). 306–314. Learning and Instruction. M. 193–211... Paper presented at the 11th Biennial European Association for Research on Learning and Instruction (EARLI) Confrerence.1007/s11251–004–2274–8.1007/BF00129106.). H. Cyprus (August). P. (2004).004.. A. S. V. M. Mehta.. D. New York: College Entrance Board.2004.. 15. H. P. (2006). Veenman. J. N.2003. M.001. J. Desoete Swanson. (2005). Higher Education. doi:10. H.. & Pasternak. 531–566). Moschner (Eds. Journal of Educational Psychology. B.1016/j.. Handbook of self-regulation (pp. & B. Veenman. Coltman. V. M. Münster: Waxmann. Conceptual and methodological considerations. V. 3–14.. A.10. (2005). Whitebread. Van Hout-Wolters. M. Anderson. 1. S. R.12. doi:10. Pintrich. J.(2005). Knowing what you know and what you don’t: Further research on metacognitive monitoring. San Diego: Academic. College board research report 2002–3. 33. Lernstrategien und Metakognition: Implikationen für Forschung und Praxis (pp. W. Wilhelm.. The relation between intellectual and metacognitive skills from a developmental perspective. P. & Spaans. Sixth-grade students’ mathematical problem-solving behaviour. 31. Artelt. V. UFB: Leiden University. Influence of metacognitive knowledge and aptitude on problem solving. H. Instructional Science. & Perry. Motivational variables and gender differences. & Afflerbach.1007/ s11409–006–6893–0. Veenman. 89–109. cognitive and affective aspects of learning styles and strategies: A phenomenographic analysis. T.. 25–50.

meta

meta

- Prediction
- Teachers Guide to Holt Winters1 (1)
- tmp2CE1.tmp
- Bearing Case Study
- assessing the default risk discrete titme survival analyis approach
- H_264
- Lesson Plan- Complete (Andrews)
- On Line Stability Comprehensive Assessment and Preliminary Decision Technology Study
- Leonhardt - When Crowd Isn't Wise
- Reading Strategies Lesson Plan
- Cmsc 798 Research Report
- Predictive Algorithms
- Element 1_science Process Skill
- Fallacy of planning
- AkberJkt Henry ring - The Problem of Prediction
- 3mcmoneagle
- Predictions of Social Learning
- eps logit
- Language Assessment
- Stock Market Prediction System With Modular Neural Networks
- Naveen (AR Model)
- zocalo-sdforum
- Traffic modeling and Prediction
- Full Text
- Human Mobility
- PrdctnLrnngGms
- Report
- 04.3.2-EvolProg
- Multiple Intelligences Chart
- Part 5 - Prediction Error Methods
- 031

Are you sure?

This action might not be possible to undo. Are you sure you want to continue?

We've moved you to where you read on your other device.

Get the full title to continue

Get the full title to continue reading from where you left off, or restart the preview.

scribd