This article was downloaded by: [Stockholm University Library]

On: 24 August 2015, At: 23:48

Publisher: Routledge
Informa Ltd Registered in England and Wales Registered Number: 1072954 Registered
office: 5 Howick Place, London, SW1P 1WG

International Journal of Lifelong

Education
Publication details, including instructions for authors and
subscription information:
http://www.tandfonline.com/loi/tled20

Development of learning to learn skills

in primary school
a

Mari-Pauliina Vainikainen , Sascha Wstenberg , Sirkku

a

Kupiainen , Risto Hotulainen & Jarkko Hautamki

a

University of Helsinki, Finland

University of Luxembourg, Luxembourg

Published online: 20 Aug 2015.

Click for updates

To cite this article: Mari-Pauliina Vainikainen, Sascha Wstenberg, Sirkku Kupiainen, Risto
Hotulainen & Jarkko Hautamki (2015): Development of learning to learn skills in primary school,
International Journal of Lifelong Education, DOI: 10.1080/02601370.2015.1060025
To link to this article: http://dx.doi.org/10.1080/02601370.2015.1060025

PLEASE SCROLL DOWN FOR ARTICLE

Taylor & Francis makes every effort to ensure the accuracy of all the information (the
Content) contained in the publications on our platform. However, Taylor & Francis,
our agents, and our licensors make no representations or warranties whatsoever as to
the accuracy, completeness, or suitability for any purpose of the Content. Any opinions
and views expressed in this publication are the opinions and views of the authors,
and are not the views of or endorsed by Taylor & Francis. The accuracy of the Content
should not be relied upon and should be independently verified with primary sources
of information. Taylor and Francis shall not be liable for any losses, actions, claims,
proceedings, demands, costs, expenses, damages, and other liabilities whatsoever or
howsoever caused arising directly or indirectly in connection with, in relation to or arising
out of the use of the Content.
This article may be used for research, teaching, and private study purposes. Any
substantial or systematic reproduction, redistribution, reselling, loan, sub-licensing,
systematic supply, or distribution in any form to anyone is expressly forbidden. Terms &

Downloaded by [Stockholm University Library] at 23:48 24 August 2015

Conditions of access and use can be found at http://www.tandfonline.com/page/termsand-conditions

INT. J. OF LIFELONG EDUCATION, 2015

http://dx.doi.org/10.1080/02601370.2015.1060025

Downloaded by [Stockholm University Library] at 23:48 24 August 2015

Development of learning to learn skills in

primary school
MARI-PAULIINA VAINIKAINENa,
STENBERGb, SIRKKU KUPIAINENa,
SASCHA WU
KIa
RISTO HOTULAINENa and JARKKO HAUTAMA
b
a
University of Helsinki, Finland; University of
Luxembourg, Luxembourg

In Finland, schools effectiveness in fostering the development of transversal skills is

evaluated through large-scale learning to learn (LTL) assessments. This article presents
how LTL skillsgeneral cognitive competences and learning-related motivational beliefs
develop during primary school and how they predict pupils CPS skills at the end of
sixth grade. The six-year follow-up of 608 pupils shows that cognitive competences
demonstrated in the beginning of the first grade in a learning preparedness test predict
both later cognitive LTL competences and CPS, but their development is not fully
determined by earlier individual differences in learning preparedness in the first grade.
Motivational beliefs begin to be related to cognitive LTL performance gradually from age
10 on, and they may have a slightly stronger effect on CPS than on cognitive LTL
performance. It is concluded that the development of CPS is partly depending on pupils
initial learning preparedness and the development of their LTL skills.
Keywords: learning to learn; development of cognitive competences; development of
learning-related motivational beliefs; complex problem solving

The strategies for evaluating educational effectiveness and equity vary from
country to country. The importance of the more general outcomes of education
the so called cross-curricular or transversal skillsis often also acknowledged
but there is a clear lack of indicators for them. The most influential effort of
assessing these prerequisites of lifelong learning, the Organisation for Economic
Co-operation and Developments (OECD) Programme for International Student
Mari-Pauliina Vainikainen has a PhD in special education with a thesis about the
development of primary school pupils learning to learn skills and the factors influencing their performance in educational assessment studies. She is a licensed psychologist
who has worked as the project manager of the Centre for Educational Assessment at
the University of Helsinki since 2007 and researcher since 2002. She coordinates all the
large-scale municipal and nationally representative assessment studies conducted at the
Centre, and she also participated in the national implementation of Programme for
International Student Assessment (PISA) 2006. In PISA 2015, she acts as the vice-NPM
for Finland, having the responsibility of the core domain of collaborative problem solving. In 20072012, she coordinated the work of the Scientific Board of Psychology in
Finland, preparing evidence-informed recommendations for developing practices. She is
the editor-in-chief of Psykologia (The Finnish Journal of Psychology). Correspondence:
Centre for Educational Assessment, P.O. Box 9, 00014 University of Helsinki, Finland.
Email: mari-pauliina.vainikainen@helsinki.fi

2015 Taylor & Francis

Downloaded by [Stockholm University Library] at 23:48 24 August 2015

MARI-PAULIINA VAINIKAINEN ET AL.

Assessment (PISA), has besides measuring the application of knowledge

acquired at school to real-life situations targeted also complex problem solving
(CPS), which was measured as a transversal domain-general skill in PISA 2012
(CPS; OECD, 2013). However, there are considerable limitations in how the
results of international comparative assessments can be utilized as feedback
when monitoring development or developing practices in individual schools due
to strongly regulated sampling and timing of the assessments, and the lack of
longitudinal data. Therefore, national programmes for assessing transversal skills
are needed to complement the international studies. This article gives an introduction to the Finnish model for assessing learning to learn (LTL) skills, the
domain-general cognitive competences and learning-related motivational beliefs,
and presents how the development of LTL during the first six years of the nineyear comprehensive school predicts pupils CPS skills (see Greiff et al., 2013 for
an overview on CPS) at the end of sixth grade. It is expected that both the
cognitive and the motivational domains of LTL that have developed during earlier school years are needed when solving complex problems, which become
increasingly important during later school years.

Sascha Wustenberg received his PhD (with distinction) at the University of Heidelberg,
Germany, in early 2013. His dissertation with the title Nature and Validity of Complex
Problem Solving considerably advanced research on CPS. His main interest lies in educational measurement of problem solving competency and he brings along several years of
experience in assessing non-routine transversal skills. Sascha Wustenberg developed computer-based CPS tasks that were included in the assessment of Interactive Problem Solving
in the arguably most influential educational large-scale assessment worldwide, the PISA,
in its 2012 cycle.Correspondence: University of Luxembourg, Luxembourg. Email:
sascha.wuestenberg@uni.lu
Sirkku Kupiainen is a special adviser MS (architecture), PhD in education under work. She
is a researcher at the Centre for Educational Assessment, University of Helsinki, since
1996. Her research centres on learning to learn and related transversal skills, and on the
Finnish education system. She has been a member of the Finnish Education Evaluation
Council and of the national Finnish consortium for PISA 2006 and PISA 2015 and a coordinator for the Finnish part of the European Commission Learning to Learn Pilot Project
in 20082009, including the writing of the final report. Correspondence: Centre for
Educational Assessment, P.O. Box 9, 00014 University of Helsinki, Finland. Email:
sirkku.kupiainen@helsinki.fi
Risto Hotulainen is an associate professor, and has PhD in special education. He is the
director of the Centre for Educational Assessment at the University of Helsinki, and has
studied school readiness and its association with self-concept, school-achievement, educational and career paths. Currently, he is involved in the Finnish THINK SCIENCE MATH
project which produces and assesses teaching materials targeted to promote development
of thinking and math skills with children aged 68 years. Correspondence: Centre for
Educational Assessment, P.O. Box 9, 00014 University of Helsinki, Finland. Email:
risto.hotulainen@helsinki.fi
Jarkko Hautamaki, professor Emeritus, PhD in Psychology, the founding member and
director (19962013) of Centre for Educational Assessment. He is the former professor
of special education (19902013), University of Helsinki, and honorary professor of Psychological Faculty of Moscow State University. He is a full member of the Finnish Society
of Science and Letters, and Academy of Social and Educational Sciences, Moscow. He is
the president (19992014) of Finnish Psychological Society, former member of Finnish
Education Evaluation Council and National Board of Education, as well as expert tasks
within The Office of The President of Finland, The Parliament, Ministry of Education
and Culture, and European Union. Correspondence: Centre for Educational Assessment,
P.O. Box 9, 00014 University of Helsinki, Finland. Email: jarkko.hautamaki@helsinki.fi

DEVELOPMENT OF LTL SKILLS

Downloaded by [Stockholm University Library] at 23:48 24 August 2015

Complex problem solving and learning to learn

Working life in the twenty-first century has changed notably as the increasing
computerization leads to an increasing demand of labour input in non-routine
tasks compared to routine tasks (Autor, Levy, & Murnane, 2003). That is, labour
force not only has to apply existing knowledge when dealing with problems, but
to adapt to changing work environments and technologies requiring lifelong
learning. CPS is regarded as a non-routine skill measuring the acquisition and
application of new knowledge (cf. Greiff et al., 2013) to overcome barriers
between a given state and a goal state (Mayer & Wittrock, 2006). Assessed by
computer-based interactive tasks clearly removed from the curricular subjects,
CPS can be considered as being closer to the pupils future practical working
situations than the more school-like and curriculum-bound LTL tasks. The
Finnish LTL test measures childrens capacity to solve problems and apply their
knowledge in novel situations, but the originally paper-based taskseven when
transferred to a computer-based platformdo not address the skill of actively
generating and applying new knowledge in an interactive and dynamically
changing problem situation. It has been shown that CPS explains supervisor
ratings of overall job performance in a variety of occupations (e.g. Danner,
Hagemann, Schankin, Hager, & Funke, 2011), and it also explains variation of
later academic achievement beyond mere reasoning skill (Wustenberg, Greiff, &
Funke, 2012).Therefore, CPS is used in the present study to evaluate the validity
of the LTL model in predicting childrens later readiness to meet the requirements of modern working life.
The origins of the Finnish LTL model
In Finland, school inspection was ceased in the late 1980s to be replaced 10
years later with national sample-based assessment of curricular outcomes in key
subjects as part of a strategy for the monitoring of three higher level domains
the efficiency, effectiveness and economy of educationand LTL was defined as
one of the measurable indicators of effectiveness (National Board of Education,
1999). At the time, the topic was intensively discussed at the European level and
LTL was later defined as one of the key competences for lifelong learning
(Recommendation 2006/962/EC of the European Parliament and of the Council of 18 December 2006).
LTL is defined in the Finnish model as comprising cognitive competences
and motivational beliefs which support their effective use (Hautamaki et al.,
2002; Hautamaki, Hautamaki, & Kupiainen, 2010; Hautamaki & Kupiainen,
2014). The model was created in 1996 and developed further during the following years along with nationally representative large-scale assessment studies (see
Hautamaki, Kupiainen, Marjanen, Vainikainen, & Hotulainen, 2013). The scales
also formed a substantial part of the European LTL instrument piloted in eight
countries in 2008 (Hoskins & Fredriksson, 2008; Kupiainen, Hautamaki, &
Rantanen, 2008). Recently, the focus has been shifting toward the use of assessment for developing classroom practices to enhance students preparedness for
lifelong learning. To evaluate the predictive validity of the LTL tests and to gain
a deeper understanding of the development of LTL skills as a result of

Downloaded by [Stockholm University Library] at 23:48 24 August 2015

MARI-PAULIINA VAINIKAINEN ET AL.

schooling, large-scale longitudinal studies have been implemented (e.g.

Marjanen, Vainikainen, Kupiainen, Hotulainen, & Hautamaki, 2014). The present article reports of one of such studies, started with the first graders of one
large municipality in 2007 and to be continued until the end of compulsory
education in 2016. In the study, pupils preparedness for lifelong learning has
been assessed besides the LTL test by computer-based CPS tasks (Greiff et al.,
2013), which were also used as a measure of CPS in PISA (OECD, 2013). This
allows evaluating the validity of the LTL test in predicting pupils capacity to
actively generate and apply new knowledge in interactive, dynamically changing
problem situations, which is a requirement of the working life in the twenty-first
century. Moreover, the study allows evaluating some of the prerequisites of CPS,
which cannot be measured easily as such with very young children due to its
complex non-routine nature.
The theoretical origins of the LTL modelthe understanding of learning as
a measurable outcome of general modifiable cognitive competences (Adey,
Csapo, Demetriou, Hautamaki, & Shayer, 2007) and of beliefs which guide and
support their uselie in Snows views on aptitude, development and education
(1996). The cognitive competences are measured by tasks which are related to
curricular contents but require the application of higher order thinking skills
instead of just repeating subject-specific knowledge and skills. Using the concepts of Demetriou, Spanoudis, and Mouyi (2011) theory on the architecture,
development and education of the human mind, the functioning of the inference
system and problem solving is measured by tasks set in the contexts of categorical,
quantitative, spatial, causal and verbal reasoning which are, according to the theory, the underlying specialized structural systems. The tasks used in the present
study can be roughly categorized as measuring general thinking and reasoning,
reading comprehension and mathematical thinking.
The present study also takes into account childrens initial preparedness for
learning when they come to school at the age of seven, and schooling has not
yet produced any effects on their LTL or CPS skills. This allows looking at the
influence of prior cognitive competence on later acquirement of these transversal skills, and discussing the limitations schools face when trying to enhance
these skills through education. The learning preparedness test, using the concepts of Demetriou and colleagues (2011), measures the functioning of the representational capacityworking memory and executive controlas well as
analogical reasoning, which is central for the inference system at this age.
One important component of Demetrious et al. theory (2011) is the consciousness system which refers to the monitoring processes (i.e. metacognition) for
ensuring anawareness of the goal, to evaluative functions for comparing the present state with the goal, and to control functions for correcting actions. It is
related to learning strategies, reflection, self-evaluation and self-awareness, which
are partially covered by the motivational belief questionnaires of the LTL scales
measuring control expectancies, causality and agency beliefs, achievement goals,
academic self-concepts and engagement. The effects of motivational beliefs on
achievement have been studied extensively over the years (Eccles & Wigfield,
2002), and they do explain variation in school achievement even though their
explanatory power has not been very strong in the few studies in which the effects
of prior competences have been controlled for (e.g. Gagne & St Pe`re, 2001).
When studying the development of LTL or CPS skills, the role of motivational

DEVELOPMENT OF LTL SKILLS

Downloaded by [Stockholm University Library] at 23:48 24 August 2015

beliefs needs to be taken into account because they can influence childrens
behaviour in learning and assessment situations. For instance, detrimental motivational beliefs have been shown to decrease time investment in assessment tasks
which in turn leads to poorer results (Kupiainen, Vainikainen, Marjanen, &
Hautamaki, 2014). As the participants of the present study are younger compared
with much of earlier research, it also needs to be evaluated when children have
sufficient capacity for self-evaluation for their self-reported beliefs to have any
connection with their competences (Harter, 1999). Based on earlier studies
(Demetriou & Kazi, 2006), this is expected to happen when pupils are 11- or
12-year olds, approximately during the fifth year of Finnish primary education.
Research questions and hypotheses
The purpose of this study was to examine to what extent the development of
pupils LTL skills, followed from the beginning of the fourth grade to the end
of the sixth grade, depends on their initial learning preparednesscognitive
competence and teacher-evaluated reading skillswhen they entered formal
education, and how well the LTL model succeeds in predicting CPS skills. The
research questions and hypotheses of this study were:
Q1:
H1:

Q2:
H2:
Q3:
H3:

Q4:
H4:

How do pupils initial cognitive competences and reading skills in the beginning of
the first grade predict their performance in the cognitive LTL tasks in the fourth
and the sixth grade?
Initial cognitive competences and reading skills are relatively strong predictors of
later performance in the cognitive LTL tasks. The prediction is not complete, however, as education should have produced an additional effect (Adey et al., 2007) and
there are other factors influencing pupils performance in the assessment situation
(Vainikainen, 2014).
Do childrens initial cognitive competences and reading skills have a direct effect on
their sixth-grade performance or is the effect completely mediated by their fourth
grade performance?
The direct effect of initial competences on the sixth-grade performance is weak when
pupils fourth-grade performance in the same tasks has been taken into account.
Can learning-related motivational beliefs be measured with the LTL beliefs questionnaires already at the age of 10? At which age do pupils expressed motivational
beliefs begin to be related to cognitive performance?
The motivational scales are measurement invariant across grades four and six but
they only begin to explain cognitive performance beyond cognitive predictors during
the follow-up period (Demetriou & Kazi, 2006; Kenney-Benson, Pomerantz, Ryan, &
Patrick, 2006). In general, the effects are expected to be small due to the age of the
pupils (Harter, 1999).
Does the LTL modelthe cognitive and the motivational domains, which are
predicted by initial cognitive competences and reading skillspredict CPS?
The LTL model is a relatively strong predictor of CPS: Both pupils initial cognitive
competences and their later cognitive LTL competence explain variance of CPS, and
motivational beliefs add to explaining how the pupils perform in the tasks.

Methods
Participants
In autumn 2007, 16 schools were randomly selected from the schools of an urban
municipality in Southern Finland using an equal-probability method to ensure

Downloaded by [Stockholm University Library] at 23:48 24 August 2015

MARI-PAULIINA VAINIKAINEN ET AL.

representativeness with regard to socio-economic status. All 40 first-grade classes

of the schools were instructed by the Education Department of the City to participate in the study, making the number of participating pupils 744. For a detailed
description of the sample, see Lonnqvist, Verkasalo, and Vainikainen (2011).
In the beginning of the fourth year of the study, four new schools were
included in the sample as many pupils had transferred to them. The whole age
cohorts of the 20 schools participated in the LTL assessment in the beginning
of the fourth grade in 2010 and at the end of the sixth grade in 2013. Of the
original 744 sampled pupils, 608 (51% girls) were still in these 20 schools in the
sixth grade, and at least partial data were available for them at all time points.
Only the data of these pupils were used in the present study. The mean age of
the pupils at the time of the first data collection was 7.31 years (SD = .31) and at
the time of the last data collection 12.81 years (SD = .33).
Measures
First-grade assessment of initial cognitive competences. A group test for learning
preparedness (Hautamaki et al., 2001) was administered to the first-grade pupils
shortly after school start. The paper-and-pencil tasks were presented to the
pupils by classroom teachers as part of normal school work. To avoid the effects
of exhaustion, only one task was presented each day. The test, which is
described in detail in Lonnqvist, Vainikainen, and Verkasalo (2012), comprised
five non-verbal cognitive tasks of which three are used in this study (one task
was too easy and one too difficult to produce enough variation between pupils).
The tasks and their reliabilities are presented in table 1. The reliabilities were
just acceptable. The teachers also evaluated each pupils initial reading skills on
a seven-point scale (1 = Not at all to 7 = Fluently. Note: in Finland after the
non-compulsory pre-school year children are expected to know letters but not to
read yet).
In the test of initial cognitive competences, analogical reasoning skills were
measured by a task adapted from a Dutch geometric analogies test (Hosenfeld,
van den Boom, & Resing, 1997). The pupils were presented with a pair of two
geometric figures, and the task was to choose a corresponding pair to another
figure from among five alternatives. The task consisted of seven analogies, which
were scored dichotomously as correct or incorrect.
In the visuospatial memory task (Wilson, Scott, & Power, 1987), the pupils
were presented with six grids with some of the squares painted black. The pupils
were then asked to reproduce the figure they just saw in an empty grid of the
same size. Each grid was scored dichotomously as correct or incorrect.
Pupils ability to follow instructions and to understand rules was assessed by a
task in which they were to draw a path on an empty 12 * 5 grid according to the
teachers step-by-step dictation. Halfway through the grid, the teacher stopped
dictating and the pupils had to continue the path following the rule formed by
the drawn path. The four grids were scored dichotomously as correct or
incorrect.
Cognitive LTL tasks in the fourth and sixth grade. Also the fourth- and the
sixth-grade LTL assessments were conducted by classroom teachers according to

DEVELOPMENT OF LTL SKILLS

Table 1. Reliabilities of the scales for different grades (Cronbachs )

Downloaded by [Stockholm University Library] at 23:48 24 August 2015

Scale
First grade learning preparedness
test
Analogical reasoning
Visuo-spatial memory
Following instructions
Motivational beliefs
Learning orientation
Achievement orientation
Agency: Effort
Learning to learn testa
Arithmetical Operations
Mental Arithmetic
Reasoning skills (Bottles task)
Reading comprehension
Complex problem solving
CPS Knowledge acquisition
CPS Knowledge application

Number of
items

7
6
4
3
2
3
37
4
5
8
20
9
9

1st
grade

4th
grade

6th
grade

.68
.56
.60
.85
.66
.77
.75

.80
.79
.79
.83

.79
.66

The test score was calculated as a weighted mean of the three domains so that reading
comprehension, mathematical thinking and reasoning skills got an equal weight.

written instructions. The pupils worked on a LTL booklet with cognitive tasks
and beliefs questionnaires. In the fourth grade, the pupils were allocated four
45-min sessions for the assessment, and in the sixth grade one 90-min session
without a break. Of the LTL assessment battery, only items that were identical
for both age groups were used in this study (for more information, see
Vainikainen, 2014). The tasks and the reliabilities of the test for both age groups
are presented in table 1. The reliabilities were acceptable.
Mathematical thinking skills were measured with nine items. Five of these were
adapted from the Mental Arithmetics task of the Wechsler Adult Intelligence
Scale-Revised (Wechsler, 1981). The teacher read aloud a mathematical problem
(e.g. If you buy two bus tickets and one ticket costs three euros fifty cents, how
much money do you get back if you give ten euros?), and the pupils wrote down
the answer in their test booklets. Four items were adapted from the Arithmetical
Operations task (Demetriou, Pachaury, Metallidou, & Kazi, 1996). These arithmetical equations feature done to four hidden operators (e.g. (5 a 3) b 4 = 6. In
this task letter a/b stands for: addition/subtraction/multiplication/division?).
The items were first coded dichotomously as correct or incorrect. Then a mean
score was calculated separately for the five Mental Arithmetics items and for the
four Arithmetical Operators items. The final score for mathematical thinking was
calculated as a mean of the two mean scores.
Reasoning skills were measured by the classical water-level task (Piaget &
Inhelder, 1956). A picture of eight empty bottles was presented to the pupils.
One of the bottles was standing, and the rest of them were inclined by 45, 90,
135, 320, 270, 225 and 180, respectively. The task was to draw the line indicating the water level and mark the area filled with water when the bottle was
half full. The bottles were coded dichotomously as correct or incorrect, and
then a mean score was calculated.

Downloaded by [Stockholm University Library] at 23:48 24 August 2015

MARI-PAULIINA VAINIKAINEN ET AL.

Reading comprehension was assessed by two tasks based on expository texts.

In the hierarchy-rating task, which was based on the theoretical framework of
Kintsch and van Dijk (1978), the pupils were asked to read a one-page text and
to then assess 16 statements based on the text as to whether they present a good
description of the text as a whole, important information regarding the content
of the text, or refer to less important details in the text (Lyytinen & Lehto,
1998). The other task was a shorter text with four multiple-response items for
measuring pupils ability to understand and interpret complex sentences. The
items were coded dichotomously as correct or incorrect, and a mean score was
calculated for all 20 items together. Thus, the longer hierarchy-rating task
received a substantially larger weight in the mean score. The final LTL test
scores for grades four and six were calculated by averaging once more the three
task-type mean scores of each grade.
Complex problem solving. CPS was assessed by nine computer-based MicroDYN
tasks (Greiff & Funke, 2006; Greiff et al., 2013). The 45-min CPS test was completed in school computer labs approximately one week after the LTL test. In
MicroDYN, input variables were related to output variables but relations were
opaque for pupils and values of output variables might change dynamically without pupils interventions. Pupils task was to actively manipulate variables and to
plot their assumptions about their relations in a causal model diagram (knowledge acquisition). After this, the pupils were asked to reach certain target values
of the output variables (knowledge application). Credit was given separately for
knowledge acquisition and application, yielding in sum nine indicators per
dimension, one for each item. Mean scores were calculated for both dimensions.
Learning-related motivational beliefs. Motivational beliefs were measured by two
scales based on achievement goal theory (e.g. Harackiewicz, Barron, Pintrich,
Elliot, & Thrash, 2002) and one scale based on agency beliefs theory (e.g. Little,
Lopez, Oettingen, & Baltes, 2001). The numbers of items per scale are presented
in table 1. In the analyses, all the eight items were used to form a latent factor of
motivational beliefs. The scales were selected from almost 30 beliefs scales of the
LTL test as in earlier LTL studies (e.g. Hautamaki et al., 2013), they have shown to
be the most directly connected to pupils school achievement. From achievement
goal theory, Mastery Intrinsic Orientation (e.g. An important goal for me at school is
to learn new things.) and Mastery Extrinsic Orientation (e.g. Getting good grades at
school is important to me.) were used (see Tuominen-Soini, Salmela-Aro, &
Niemivirta, 2008), and of agency beliefs the construct Agency: Effort (e.g. I work
hard to do well at school.). All the items were answered on a seven-point
Likert-scale (1 = Not true at all7 = Very true). The reliabilities are presented in
table 1. In further analyses, the scale average scores were used instead of specifying
measurement models for each scale separately.
Statistical methods
Descriptive statistics were calculated with SPSS18. Structural equation modelling
(SEM) was applied in AMOS21. Since the deviation from normality of all

Downloaded by [Stockholm University Library] at 23:48 24 August 2015

DEVELOPMENT OF LTL SKILLS

variables was within the recommended limits (Kline, 2005), maximum likelihood
estimation was used. The models were considered having a good fit with CFI
and TLI > .95, and RMSEA < .06, and an acceptable fit with CFI and TLI > .90,
and RMSEA < .08. 2-values are also reported but due to the sample size and the
large number of variables in the models, significant p-values were to be
expected. Before testing the full models with beliefs that were measured at two
time points, measurement invariance tests were applied to ensure that the items
measured the same constructs at both occasions. Measurement invariance was
tested according to the procedure of Byrne and Stewart (2006) by constraining
stepwise the factor loadings (weak factorial invariance) and then intercepts
(strong factorial invariance) equal across time points, and comparing the
change of the fit indices to the baseline model at each stage. Even if latent
means were not compared in the present study, strong factorial invariance was
seen as a criterion for an acceptable model (cf. Byrne & Stewart, 2006).

Results
The descriptive statistics for all the variables are presented in table 2.
H1: In the first hypothesis it was assumed that pupils later performance in
the cognitive LTL tasks would be relatively strongly predicted by their first grade
competences. To test this assumption, a structural equation model was specified
using all the first-grade variables presented in table 2 as predictors of fourthand sixth-grade LTL performance. First, a measurement model was specified by
regressing the first-grade three-task sum scores to a common latent factor reflecting first-grade competences (factor loadings .47 .58, p < .001). Additionally,
teacher-evaluated reading skills at school start were added as a predictor of both
fourth- and sixth-grade LTL test scores, and the sixth-grade scores were additionally predicted by the fourth-grade scores. The model fit the data excellently
(CFI & TLI = 1.00; RMSEA = .00; 2 = 5.566, df = 6, p = .474).
The first-grade cognitive competences and reading skills correlated (r = .34,
p <.001), and the initial cognitive competences were a strong predictor of the
fourth-grade LTL test score ( = .63, p <.001). The path from initial reading
skills to the fourth-grade test performance, however, was not statistically significant ( = .07, p = .100). Initial cognitive competences predicted the sixth-grade
LTL test score ( = .46, p < .001), and initial reading skills were of more
importance than in explaining the fourth-grade score ( = .16, p < .001). The
sixth-grade performance was also moderately predicted by the fourth-grade
performance ( = .21, p < .001). Altogether, 43% of the variance of the fourthgrade performance, and 48% of the variance of the sixth-grade performance
were explained by the model. It was concluded that H1 was fully supported.
H2: The second hypothesis was that initial cognitive competences would not
have a strong direct effect on the sixth-grade performance when the fourthgrade performance is included in the model. The path coefficients described for
H1 show that the direct effect of first-grade cognitive competences on the sixthgrade test score was quite strong and the indirect effect weak ( = .13) as and
the sixth-grade test score was only moderately predicted by the fourth-grade test
score. Thus, it was concluded that H2 was not supported.

10

MARI-PAULIINA VAINIKAINEN ET AL.

Table 2. Descriptive statistics for the variables used in the models

Downloaded by [Stockholm University Library] at 23:48 24 August 2015

Variable
Reading skills first grade
Analogical reasoning first grade
Visuo-spatial memory first grade
Following instructions first grade
Learning orientation fourth|sixth
grade
Achievement orientation fourth|sixth
grade
Agency: effort fourth|sixth grade
LTL test score fourth|sixth grade
CPS knowledge acquisition sixth grade
CPS knowledge application sixth
grade

Min

Max

SD
2.39
1.90
1.50
1.18
1.24|1.12

472 1.00
608 .00
608 .00
608 .00
581|567 1.00

7.00
7.00
6.00
4.00
7.00

3.77
3.96
3.96
2.84
5.84|5.40

581|565 1.00

7.00

6.03|5.62 1.16|1.23

580|567 1.00
7.00
5.75|5.24 1.13|1.14
608|573 .07|.06 .93|.98 0.41|0.58 .17|.18
415 .00
1.00
.35
.24
403 .00
1.00
.36
.22

Notes: The values for fourth|sixth grade are separated with a vertical bar.
N = Number of responses, Min = minimum value, Max = maximum value, M = Mean,
SD = Standard deviation.

H3: The third hypothesis was that learning-related motivational beliefs could
be measured already in the fourth grade using the same scales as at later age,
and that they would gradually begin to explain cognitive performance even
when initial cognitive competence is taken into account. For testing measurement invariance to assess the validity of the scales, a measurement model for
motivational beliefs was first specified by regressing the belief-scale means of
Mastery Intrinsic and Extrinsic Orientation, and Agency: Effort to a latent factor
for fourth and sixth grade separately (factor loadings .77.88, p < .001). A
higher order factor was used as the aim was to test a hypothesis regarding the
influence of learning-enhancing beliefs on performance on a general level, not
to study the structure of these beliefs as such. Then, the fourth-grade latent factor was used in predicting the sixth-grade factor, and measurement invariance
across the two time points was tested by constraining the factor loadings and
then the intercepts equal. The changes in fit indices are presented in table 3. As
the differences to the baseline model were minimal, it was concluded that the
beliefs model was measurement invariant across the two grade levels and, thus,
could be used as a predictor of cognitive LTL performance.
Next, the motivational beliefs model was added to the first model for testing
the effects of initial cognitive competences on later performance. The fit indices
of this model are presented in table 3. The fourth- and the sixth-grade motivational beliefs were not affected by initial cognitive competences or reading skills,
so these paths were removed from the model. The beliefs did not predict students fourth-grade test performance but there was a weak connection between
the sixth-grade beliefs and the sixth-grade performance ( = .08, p < .050),
which increased the share of explained variance of the sixth-grade test score
from 48% to 49%. It was concluded that H3 was supported only partially
regarding the sixth-grade results.
H4: The fourth hypothesis was that CPS would be predicted by the other
variables in the model. First, to study the relationships between CPS, initial
cognitive competences and performance in the cognitive LTL tasks, bivariate
correlations were calculated for all cognitive tasks and for the two CPS

11

DEVELOPMENT OF LTL SKILLS

Table 3. Measurement Invariance of the fourth- and sixth-grade beliefs, and of

the whole models for testing hypotheses three and four

Downloaded by [Stockholm University Library] at 23:48 24 August 2015

Model
Motivational beliefs in fourth and sixth grade
Baseline model
Factor loadings constrained equal
Measurement intercepts constrained equal
The whole model without CPS
Baseline model
Factor loadings constrained equal
Measurement intercepts constrained equal
The model for predicting CPS
Baseline model
Factor loadings constrained equal
Measurement intercepts constrained equal

df

CFI

TLI

RMSEA

40.856
52.842
58.636

8
10
12

.980
.974
.971

.946
.944
.949

.082
.084
.080

<.001
<.001
<.001

87.365
99.429
105.510

49
51
53

.983
.978
.977

.973
.967
.965

.036
.040
.040

.001
<.001
<.001

112.450
124.986
131.007

68
70
72

.982
.978
.976

.973
.967
.966

.033
.036
.037

.001
<.001
<.001

Notes: CFI = Comparative fit index, TLI = TuckerLewis index, RMSEA = Root mean
square error of approximation.

dimensions. The LTL test seemed to correlate stronger with knowledge acquisition than with knowledge application (r = .43/.47, p < .001 and r = .35/.28,
p < .001, respectively, for the fourth-/sixth-grade test scores). Thus, CPS was
clearly related to LTL. To study the relations further, the scores for knowledge
acquisition and application were regressed to a latent CPS factor (factor loadings
.85 and .60 for knowledge acquisition and application, p < .001), which was then
added to the model specified when testing H3. The final model is presented in
figure 1 and its fit indices in table 3.
Figure 1 with the final model shows that CPS was predicted by initial
cognitive competences ( = .41, p < .001), fourth-grade LTL performance
( = .16, p < .050), sixth-grade LTL performance ( = .17, p < .050), and sixthgrade beliefs ( = .14, p < .010). Contrary to the expectations, the relations
between the fourth- and sixth-grade LTL test performance and CPS were weaker
than CPSs relation with initial cognitive competences. Thus, CPS was clearly
related to LTL but the changes which had happened in the cognitive LTL
competences by the end of sixth grade beyond the initial cognitive competences
did not add very much to explaining the variance in CPS, at least when LTL was
modelled as manifest test scores instead of latent factors. Nevertheless, altogether 46% of the variance of CPS could be explained with the model and it
was concluded that H4 was supported.
Discussion
The aim of this longitudinal study was to understand how primary school pupils
LTL skills develop during primary education and how LTL predicts CPS. The
development of LTL was studied using the Finnish LTL scales and CPS using
the MicroDYN approach. A structural equation model was specified in which
pupils performance in the cognitive LTL tasks in the fourth and sixth grade,
and their CPS skills at the end of sixth grade were predicted by their cognitive
competence and teacher-evaluated reading skills in the beginning of the first

Downloaded by [Stockholm University Library] at 23:48 24 August 2015

12

MARI-PAULIINA VAINIKAINEN ET AL.

Figure 1. The final model for predicting performance in cognitive LTL tasks
and complex problem solving. The only non-significant path from initial reading
skills to fourth-grade LTL test is marked with (ns). Numbers in brackets indicate
the variance accounted for
grade, and by their learning-related motivational beliefs measured at grades four
and six. The modelling confirmed that there are individual differences in
pupils cognitive competences in the beginning of the first grade, and that the
initial competences predict pupils later performance in regard to both LTL and
CPS. This is a general and important result, which is true for general ability or
intelligence (Deary, Strand, Smith, & Fernandes, 2007; Demetriou et al., 2011;
Rohde & Thompson, 2007) as well as for more specific studies of development
of mathematical thinking in school (Aunola, Leskinen, Lerkkanen, & Nurmi,
2004; Koponen, Aunola, Ahonen, & Nurmi, 2007). The results fit well together
with the findings of earlier studies regarding the development of cognitive
competences during primary schooldevelopmental cognitive constructs contribute to learning and the roles of different factors may vary by age (Demetriou
et al., 1996, 2014).
Even though the measured initial competences explained almost half of the
variance of later performance, there were clearly other factors influencing the
results, too. On the one hand, this leaves room for the interpretation that cognitive competences can be enhanced by means of education (Adey et al., 2007;
Snow, 1996), even though educational effects or school and class-level variation
could not be systematically modelled in the present study and the measures of
cognitive competences had some limitations. On the other hand, motivation,
interest and effort influence childrens performance in external low-stakes assessments (see Kupiainen et al., 2014; Vainikainen, 2014), which was the main
reason for including motivational beliefs also in the present study.
There was a moderate bivariate correlation between initial reading skills and
pupils fourth-grade performance, but the relation between these was not statistically significant in the modelling. It is possible that during the early school years
the effect of early reading skills gets imbedded in the more general cognitive
competences, and hence they do not provide additional value for the fourth-grade

Downloaded by [Stockholm University Library] at 23:48 24 August 2015

DEVELOPMENT OF LTL SKILLS

13

performance. The inaccurate measure of initial reading skills, the teacher evaluations, may also explain why the effect was not as strong as expected.
Also, contrary to the expectations, the effect of initial cognitive competences
on the sixth-grade performance was mainly direct with hardly any mediation
through the fourth-grade performance. That is, the LTL skills acquired during
the first four school years only slightly strengthened the LTL test performance
further by the end of the sixth grade, and the sixth-grade LTL test score was
mostly explained by pupils initial competences, in other words with already
existing differences when children came to school.
As there is a firm understanding that motivational beliefs affect learning and
educational outcomes (e.g. Demetriou et al., 2011; Eccles & Wigfield, 2002;
Harackiewicz et al., 2002; Kenney-Benson et al., 2006; Snow, 1996), the beliefs
measured at grades four and six were added next to the model. It was simultaneously studied how strongly the fourth-grade beliefs predicted the sixth-grade
beliefs and how these were related to the LTL test performance in the respective
grades. In earlier studies with children at this age (e.g. Kenney-Benson et al.,
2006), it has been found that motivational beliefs are more strongly related to
school grades than to external assessment tasks but weak relationships were
nevertheless expected to be found.
Motivational beliefs were not predicted by initial cognitive competence or
reading skills. Regardless of a small decline from the fourth to the sixth grade,
beliefs were in general very positive but in the fourth grade they did not explain
performance at all. This can be interpreted as a sign of 10-years-olds limited
self-awareness and capacity for self-evaluation (Demetriou & Kazi, 2006; Harter,
1999), which is most likely the reason for the limited literature on young pupils
self-reported beliefs and the reason for the non-existent relation between their
actual performance and self-evaluations. However, the fourth-grade motivational
beliefs predicted the sixth-grade motivational beliefs moderately and measurement invariance was met, so the same constructs were measured already in the
fourth grade. As demonstrated by Kupiainen et al. (2014), these constructs
become highly important in the context of LTL by the ninth grade when pupils
are 15-year olds and this is expected to be the case for CPS, as well, even though
thus far there has been very little research on that. In the present study, just as
in the studies of Demetriou and Kazi (2006), pupils self-evaluations could be
interpreted as becoming more accurate by the end of the sixth grade as then
there was a weak positive connection between motivational beliefs and the LTL
test score. Yet, beliefs did not add much to explaining the sixth-grade LTL test
score, and only 1% more variance was accounted for.
At the last stage, the LTL model was used to predict the sixth grade CPS skills
to evaluate the validity of the LTL model in predicting pupils performance in
computer-based problem solving tasks, which are further removed from the
school context and can thus be expected to be closer to the practical requirements of modern working life. The results showed that CPS skills were predicted
by the same factors that predicted performance in the sixth grade LTL test, that
is, initial cognitive competence and performance in the fourth grade LTL test.
This is in line with previous findings on that working memory (Schweizer,
Wustenberg, & Greiff, 2013) and reasoning (Sonnleitner, Keller, Martin, &
Brunner, 2013; Wustenberg et al., 2012) are related to CPS, but a considerable
amount of variance in CPS is not associated with these constructs.

Downloaded by [Stockholm University Library] at 23:48 24 August 2015

14

MARI-PAULIINA VAINIKAINEN ET AL.

Even though 46% of the variance of CPS was explained with the final model
depicted in figure 1, the fourth- and the sixth-grade LTL tests explained relatively little of its variance when pupils initial cognitive competence was taken
into account. That is, in SEM, the relations between LTL test performance and
CPS were much weaker than their bivariate correlations would have suggested.
This indicates that both task types require a common basis of cognitive competences but they also measure unique aspects of transversal skills. Of these, LTL
seems to be closer to school and curricular contents than CPS at least in lower
grades. Therefore, it is reasonable to assume that effects of schooling on CPS
may be more influential in later grades, in which self-regulated discovery of nontransparent problem situations is more relevant in curricula (e.g. in science
experiments). This is backed up by empirical results of Frischkorn, Greiff, and
Wustenberg (2014), who showed in a longitudinal study that development in
CPS performance takes place from grades 8 to 11 that can only partly predicted
by pupils general cognitive competence. Nevertheless, further research in this
area is clearly needed. Based on the present study, it could be concluded that
pupils LTL skills as they have developed during the primary school predict the
CPS skills. It could also be concluded that learning-related beliefs play a role in
how pupils perform in the computer-based CPS tasks. As the connection
between beliefs and cognitive performance was in this study as weak as it was
expected to be due to the age of the pupils, more research should be done with
older students about the role of beliefs in explaining CPS skills.
Limitations of the study
The biggest limitations of the study are related to the measures: the use of teacher evaluations for initial reading skills, the low reliability of the visuo-spatial
memory task and the problems related to presenting self-report questionnaires
to young children, which may also be the reason for the relatively high RMSEA
values for the measurement model of motivational beliefs. Another limitation of
this study was that, regarding the LTL tasks, there were no measures available
for pupils effort in the assessment. Hence, pupils time investment in the computer-based CPS tasks was not studied either even if the log data would have
allowed for that (cf. Kupiainen et al., 2014). It is expected that effort would partially explain the variation in performance (cf. Ainley, Hidi, & Berndorff, 2002).
Furthermore, the expected systematic school, class and peer group level effects
on the development of LTL cognitive competences and learning-related beliefs,
which were not addressed in this study, should be taken into account when
designing interventions for enhancing these skills.

Conclusions
The results of this study can be seen to support the understanding that the
development of pupils thinking and problem-solving skills at school is relatively
strongly related to but not determined by their competences acquired before
entering formal schooling. The present study shows that in addition to pupils
initial cognitive competences at school start, CPS skills, as demonstrated at the
end of the sixth grade, are related with the cognitive and the motivational

DEVELOPMENT OF LTL SKILLS

15

Downloaded by [Stockholm University Library] at 23:48 24 August 2015

domains of LTL developed during schooling. This indicates that when solving
complex problems close to future working life situations, pupils need also the
generalized thinking skills and learning-related motivational beliefs, which have
developed in different school subjects during formal education. The results suggest that school as a context of education has potential in fostering transversal
skills crucial for lifelong learning already during the primary years (cf. Adey
et al., 2007; Snow, 1996; see also Vainikainen, Hautamaki, Hotulainen, &
Kupiainen, 2015). Moreover, the systematic fostering of the development of
pupils general thinking and problem-solving skills during compulsory and later
education can help secure that every child gets their cognitive potential in full
use in the different learning situations of later life.

Disclosure statement
No potential conflict of interest was reported by the authors.

References
Adey, P., Csapo, B., Demetriou, A., Hautamaki, J., & Shayer, M. (2007). Can we be intelligent about
intelligence? Educational Research Review, 2, 7597. doi:10.1016/j.edurev.2007.05.001
Ainley, M., Hidi, S., & Berndorff, D. (2002). Interest, learning, and the psychological processes that
mediate their relationship. Journal of Educational Psychology, 94, 545561. doi:10.1037//00220663.94.3.545
Aunola, K., Leskinen, E., Lerkkanen, M. K., & Nurmi, J. E. (2004). Developmental dynamics of math
performance from preschool to grade 2. Journal of Educational Psychology, 96, 699713.
Autor, D. H., Levy, F., & Murnane, R. J. (2003). The skill content of recent technological change: An
empirical exploration. The Quarterly Journal of Economics, 118, 12791333.
Byrne, B. M., & Stewart, S. M. (2006). Teachers corner: The MACS approach to testing for multigroup invariance of a second-order structure: A walk through the process. Structural Equation
Modeling: A Multidisciplinary Journal, 13, 287321. doi:10.1207/s15328007sem1302_7
Danner, D., Hagemann, D., Schankin, A., Hager, M., & Funke, J. (2011). Beyond IQ: A latent state
trait analysis of general intelligence, dynamic decision making, and implicit learning. Intelligence, 39, 323334.
Deary, I. J., Strand, S., Smith, P., & Fernandes, C. (2007). Intelligence and educational achievement.
Intelligence, 35, 1321.
Demetriou, A., & Kazi, S. (2006). Self-awareness in g (with processing efficiency and reasoning).
Intelligence, 34, 297317.
Demetriou, A., Pachaury, A., Metallidou, Y., & Kazi, S. (1996). Universals and specificities in the
structure and development of quantitative-relational thought: A cross-cultural study in Greece
and India. International Journal of Behavioral Development, 19, 255290. doi:10.1080/
016502596385785
Demetriou, A., Spanoudis, G., & Mouyi, A. (2011). Educating the developing mind: Towards an overarching paradigm. Educational Psychology Review, 23, 601663. doi:10.1007/s10648-011-9178-3
Demetriou, A., Spanoudis, G., Shayer, M., van der Ven, S., Brydges, C. R., Kroesbergen, E.,
Swanson, H. L. (2014). Relation between speed, working memory, and intelligence from preschool to adulthood: Structural equation modelling of 14 studies. Intelligence, 46, 107121.
Eccles, J., & Wigfield, A. (2002). Motivational beliefs, values, and goals. Annual Review of Psychology,
53, 109132.
Frischkorn, G., Greiff, S., & Wustenberg, S. (2014). The development of complex problem solving in
adolescence: A latent growth curve analysis. Journal of Educational Psychology, 106, 10071020.
Retrieved from http://dx.doi.org/10.1037/a0037114
Gagne, F., & St Pe`re, F. (2001). When IQ is controlled does motivation still predict achievement?
Intelligence, 30, 71100. doi:10.1016/S0160-2896(01)00068-X
Greiff, D., & Funke, J. (2006). Measuring complex problem solving: The MicroDYN approach. In
F. Scheuermann & J. Bjornsson (Eds.), The transition to computer-based assessment. New approaches
to skills assessment and implications for large-scale testing (pp. 157163). Luxembourg: Office for
Official Publications of the European Communities.

Downloaded by [Stockholm University Library] at 23:48 24 August 2015

16

MARI-PAULIINA VAINIKAINEN ET AL.

Greiff, S., Wustenberg, S., Molnar, G., Fischer, A., Funke, J., & Csapo, B. (2013). Complex problem
solving in educational contextsSomething beyond g: Concept, assessment, measurement
invariance, and construct validity. Journal of Educational Psychology, 105, 364379. doi:10.1037/
a0031856
Harackiewicz, J. M., Barron, K. E., Pintrich, P. R., Elliot, A. J., & Thrash, T. M. (2002). Revision of
achievement goal theory: Necessary and illuminating. Journal of Educational Psychology, 94,
638645. doi:10.1037//0022-0663.94.3.638
Harter, S. (1999). The construction of self. A developmental perspective. New York, NY: Guilford Press.
Hautamaki, J., Arinen, P., Eronen, S., Hautamaki, A., Kupiainen, S., Lindblom, B., Scheinin, P.
(2002). Assessing learning-to-learn: A framework. Helsinki: National Board of Education, Evaluation 4/2002.
Hautamaki, J., Arinen, P., Hautamaki, A., Lehto, J., Lindblom, B., Kupiainen, S., Scheinin, P.
(2001). Ensiaskeleetoppimisen edellytykset: Luokanopettajille tarkoitetun seulan toimivuus Helsinkiaineiston perusteella [First stepsPrerequisities of learning: The functionality of the screening
test for class teachers based on Helsinki-data]. Helsinki: Publication series A17: 2001.
Hautamaki, A., Hautamaki, J., & Kupiainen, S. (2010). Assessment in schoolsLearning to learn. International Encyclopedia of Education, 2010, 268272.
Hautamaki, J., & Kupiainen, S. (2014). Learning to learn in Finland. Theory and policy, research
and practice. In R. D. Crick, C. Stringher, & KaiRen (Eds.), Learning to learn. International perspectives from theory and practice. Oxon: Routledge.
Hautamaki, J., Kupiainen, S., Marjanen, J., Vainikainen, M.-P., & Hotulainen, R. (2013). Oppimaanoppiminenperuskoulunpaattovaiheessa: Tilannevuonna 2012 jamuutosvuodesta 2001 [Learning to learn
at the end of basic education: The situation in 2012 and the change from 2001]. Helsinki:
Unigrafia. University of Helsinki, Department of Teacher Education, Research Reports 347.
Hosenfeld, B., van den Boom, D. C., & Resing, W. C. M. (1997). Constructing geometric analogies
for the longitudinal testing of elementary school children. Journal of Educational Measurement,
34, 367372.
Hoskins, B., & Fredriksson, U. (2008). Learning to learn: What is it and can it be measured? JRC Scientific and Technical Report. Luxembourg: European Commission, Centre for Research on Lifelong Learning. Office for Official Publications of the European Communities.
Kenney-Benson, G. A., Pomerantz, E. M., Ryan, A. M., & Patrick, H. (2006). Sex differences in math
performance: The role of childrens approach to schoolwork. Developmental Psychology, 42,
1126. doi:10.1037/0012-1649.42.1.11
Kintsch, W., & van Dijk, T. A. (1978). Toward a model of text comprehension and production.
Psychological Review, 85, 363394.
Kline, R. B. (2005). Principles and practice of structural equation modeling (2nd ed.). New York, NY: The
Guildford Press.
Koponen, T., Aunola, K., Ahonen, T., & Nurmi, J. E. (2007). Cognitive predictors of single-digit and
procedural calculation skills and their covariation with reading skill. Journal of Experimental
Child Psychology, 97, 220241.
Kupiainen, S., Hautamaki, J., & Rantanen, P. (2008). EU pre-pilot on learning to learn: Report on the compiled data. 2008-1190/001-001 TRA-TRINDC. Helsinki: Centre for Educational Assessment.
Kupiainen, S., Vainikainen, M. P., Marjanen, J., & Hautamaki, J. (2014). The role of time on task in
computer-based low-stakes assessment of cross-curricular skills. Journal of Educational Psychology,
106, 627638. doi:10.1037/a0035507
Little, T. D., Lopez, D. F., Oettingen, G., & Baltes, P. B. (2001). A comparative-longitudinal study of
action-control beliefs and school performance. International Journal of Behavioral Development,
25, 237245.
Lonnqvist, J.-E., Vainikainen, M.-P., & Verkasalo, M. (2012). Teacher and parent ratings of sevenyear-old childrens personality and psychometrically assessed cognitive ability. European Journal
of Personality, 26, 504514.
Lonnqvist, J.-E., Verkasalo, M., & Vainikainen, M.-P. (2011). Parentteacher agreement on 7-year-old
childrens personality. European Journal of Personality, 25, 306316.
Lyytinen, S., & Lehto, J. E. (1998). Hierarchy rating as a measure of text macroprocessing: Relationship with working memory and school achievement. Educational Psychology, 18, 157169.
Marjanen, J., Vainikainen, M. P., Kupiainen, S., Hotulainen, R., & Hautamaki, J. (2014). Oppimaan
oppiminen Vantaan peruskouluissa: Kolmas-, kuudes- ja yhdeksasluokkalaiset oppijoina vuosina2013 ja
2010 [Learning to learn in comprehensive schools in Vantaa: Third, sixth and ninth graders
as learners in 2013 and 2010]. Vantaa: City of Vantaa.
Mayer, R. E., & Wittrock, M. C. (2006). Problem solving. In P. A. Alexander & P. H. Winne (Eds.),
Handbook of educational psychology (pp. 287303). Mahwah, NJ: Lawrence Erlbaum.
National Board of Education. (1999). A Framework for evaluating educational outcomes in Finland.
National Board of Education, Evaluation 8/1999. Helsinki: National Board of Education.
OECD. (2013). PISA 2012 assessment and analytical framework: Mathematics, reading, science, problem
solving and financial literacy. Author. doi:10.1787/9789264190511-en

Downloaded by [Stockholm University Library] at 23:48 24 August 2015

DEVELOPMENT OF LTL SKILLS

17

Piaget, J., & Inhelder, B. (1956). The childs conception of space. London: Routledge & Kegan Paul.
Recommendation 2006/962/EC of the European Parliament and of the Council of 18 December
2006 on key competences for lifelong learning. Official Journal L 394 of 30 December 2006.
Rohde, T. E., & Thompson, L. A. (2007). Predicting academic achievement with cognitive ability.
Intelligence, 35, 8392.
Schweizer, F., Wustenberg, S., & Greiff, S. (2013). Validity of the MicroDYN approach: Complex
problem solving predicts school grades beyond working memory capacity. Learning & Individual Differences, 24, 4252.
Snow, R. E. (1996). Aptitude development and education. Psychology, Public Policy, and Law, 2, 536560.
Sonnleitner, P., Keller, U., Martin, R., & Brunner, M. (2013). Students complex problem-solving
abilities: Their structure and relations to reasoning ability and educational success. Intelligence,
41, 289305.
Tuominen-Soini, H., Salmela-Aro, K., & Niemivirta, M. (2008). Achievement goal orientations and
subjective well-being: A person-centred analysis. Learning and Instruction, 18, 251266.
doi:10.1016/j.learninstruc.2007.05.003
Vainikainen, M.-P. (2014). Finnish primary school pupils performance in learning to learn assessments: A
longitudinal perspective on educational equity. Helsinki: Unigrafia. University of Helsinki,
Department of Education Research Reports 360.
Vainikainen, M.-P., Hautamaki, J., Hotulainen, R., & Kupiainen, S. (2015). General and specific
thinking skills and schooling: Preparing the mind for new learning. Thinking Skills and
Creativity. doi:10.1016/j.tsc.2015.04.006
Wechsler, D. (1981). WAIS-R: Manual: Wechsler adult intelligence scaleRevised. New York, NY: Harcourt
Brace Jovanovich for Psychological Corp.
Wilson, J. T. L., Scott, J. H., & Power, K. G. (1987). Developmental differences in the span of visual
memory for pattern. British Journal of Developmental Psychology, 5, 249255.
Wustenberg, S., Greiff, S., & Funke, J. (2012). Complex problem solvingMore than reasoning?
Intelligence, 40, 114.