You are on page 1of 10

Learning and Individual Differences 31 (2014) 1120

Contents lists available at ScienceDirect

Learning and Individual Differences

journal homepage:

Incremental theories of intelligence predict multiple

document comprehension
Jason L.G. Braasch a,b, Ivar Brten a,, Helge I. Strms a, istein Anmarkrud a
University of Oslo, Norway
University of Memphis, USA

a r t i c l e i n f o a b s t r a c t

Article history: This study examined implicit theories of intelligence as predictors of multiple document comprehension in a
Received 12 April 2013 sample of 59 Norwegian upper-secondary school students. In four multiple regression analyses with multiple
Received in revised form 6 November 2013 document comprehension indicated by students' inclusion of scientic concepts in their essays, discrimination
Accepted 25 December 2013
between more and less useful documents given the reading task, consideration of document trustworthiness
as a basis for making those discriminations, and ability to draw inferences across documents as outcome mea-
Implicit theories of intelligence
sures, beliefs in intelligence as a malleable, increasable quality emerged as a unique positive predictor after con-
Multiple document comprehension trolling for word recognition, prior knowledge, and working memory. However, beliefs in intelligence as a xed,
Source evaluation unchangeable quantity did not emerge as a unique negative predictor. The ndings indicate that the benets of
Intertextual inferences endorsing an incremental theory of intelligence may be more pronounced than the costs of endorsing an entity
theory of intelligence in complex reading task contexts.
2013 Elsevier Inc. All rights reserved.

1. Introduction Stadtler & Bromme, 2008). Moreover, if readers do not strategically

infer relationships between concepts found within various documents,
Many reading researchers contend that the 21st century information their resultant understandings may involve a less coherent mixture of
age has been a kind of game changer for comprehension, bringing unconnected facts and concepts (Goldman, 2004).
about not only new affordances but also new concerns (Alexander, Research to date has highlighted that a number of cognitive variables
2012; Brand-Gruwel & Stadtler, 2011; Britt & Gabrys, 2002; Leu, Kinzer, may be associated with evaluation and integration processes (Banas &
Coiro, Castek, & Henry, 2013). On one hand, readers can rapidly, almost Sanchez, 2012; Brten, Anmarkrud, Brandmo, & Strms, 2014; Brten,
instantaneously, access a wide range of up-to-date information, partic- Ferguson, Anmarkrud, & Strms, 2013; Brten, Strms, & Salmern,
ularly when retrieving documents via Internet search engines. On the 2011). However, research on psychosocial factors is surprisingly scant.
other hand, such access requires additional competencies, especially in In the current article, we posited that readers' implicit beliefs about the
considering the ease of publishing in the current information age. In nature of intelligence whether they believe that intelligence is a xed
this sense, readers should be more vigilant about the nature of docu- and stable characteristic versus malleable and within their control to
ments as socially constructed artifacts (written by a particular author, exibly develop might predict the extent to which they will employ
in support of a particular agenda, for a particular publication venue, at strategies to evaluate source features and integrate content information
a particular point in time, and so forth) (Britt, Rouet, & Braasch, 2013). in a multiple document context independent of several cognitive indi-
In addition, comprehension often requires that readers integrate con- vidual difference factors.
tent information germane to their research question that is distributed Classic and recent empirical research ndings support that source
across multiple documents (Aferbach & Cho, 2009; Goldman, evaluation and content integration are both crucial components of mul-
Braasch, Wiley, Graesser, & Brodowinska, 2012). Taken together, tiple document comprehension (Brten, Strms, & Britt, 2009; Britt,
readers must restrict their processing towards reliable, higher quality Perfetti, Sandak, & Rouet, 1999; Goldman et al., 2012; Rouet, Britt,
information to successfully comprehend. It seems reasonable to assume Mason, & Perfetti, 1996; Wiley et al., 2009; Wineburg, 1991). Brten
that, if people do not employ critical reading strategies to differentiate et al. (2009) demonstrated that post-reading trust evaluations on reli-
the quality and reliability of the information they read, they may expe- able documents and considerations of source features in making these
rience an overload of information, or even inappropriately integrate trust decisions independently predicted multiple document compre-
both accurate and inaccurate information (Graesser et al., 2007; hension, after controlling for readers' prior topic knowledge. Recent
studies using verbal protocols also demonstrate that strategies focused
Corresponding author at: Department of Educational Research, University of Oslo, P.O.
on differentiating more versus less useful documents during reading
Box 1092, Blindern, N-0317 Oslo, Norway. Tel.: +47 22 85 52 82; fax: +47 22 85 42 50. and using trustworthiness criteria when doing so relate to better post-
E-mail address: (I. Brten). reading multiple document comprehension. For example, Anmarkrud,

1041-6080/$ see front matter 2013 Elsevier Inc. All rights reserved.
12 J.L.G. Braasch et al. / Learning and Individual Differences 31 (2014) 1120

Brten, and Strms (in press) demonstrated relationships between 1.2. Implicit theories of intelligence predict performance
evaluations of information sources produced during reading and argu-
mentation sophistication and source usage in post-reading essays. Sim- Seminal work by Dweck and colleagues (Blackwell, Trzesniewski, &
ilarly, Goldman et al. (2012) contrasted the kinds of processing that Dweck, 2007; Dweck, 1999; Dweck & Master, 2008; Hong, Chiu, Dweck,
better and poorer learners displayed during reading. Findings suggested Lin, & Wan, 1999) has related learners' implicit beliefs about the nature
that better learners were more likely to go beyond content analyses to of intelligence to their approaches and ultimate success in intellectually
evaluate the trustworthiness of the sources of documents compared demanding tasks (e.g., acquiring new knowledge and skills). Some
with poorer learners. Instead, poorer learners spent more time reading learners hold beliefs that intelligence is a xed, relatively stable charac-
unreliable documents and were more likely to include erroneous con- teristic or quantity that is out of their control. Learners who identify
cepts in post-reading essays. Thus, there is a growing body of evidence with these beliefs are often described as holding entity theories of in-
that source evaluation strategies support the construction of accurate telligence. Others believe that intelligence is malleable, a quality that is
understandings from multiple documents. within their control to exibly develop and change. Accordingly,
Integration of higher quality content information both within single learners who identify with these beliefs are often described as holding
documents and across multiple documents also appears to promote incremental theories of intelligence.
successful comprehension. In Brten and Strms (2011), college stu- A large body of research has established relationships between stu-
dent readers' identication with cross-text elaboration strategies on a dents' implicit theories of intelligence and their achievement in various
post-reading survey positively predicted their intertextual comprehen- learning contexts (Blackwell et al., 2007; Dupeyrat & Marin, 2005;
sion, a measure of the inferred connections made across the documents. Faria & Fontaine, 1997; Greene, Costa, Robertson, Pan, & Deekens,
Wolfe and Goldman (2005) showed strong relationships between ado- 2010; Henderson & Dweck, 1990; Stipek & Gralinski, 1996). For exam-
lescents' self-explanation strategies during reading the inferred con- ple, Henderson and Dweck (1990), after controlling for prior achieve-
nections they made both within and across multiple history texts ment, found that adolescents endorsing more of an incremental
and their subsequent reasoning performance about the texts' topic. theory of intelligence earned signicantly higher grades in the rst
Using longer, more diverse documents, Goldman et al. (2012) demon- year of junior high school than did those endorsing more of an entity
strated that more successful college student learners constructed accurate theory. Experimental research supports the relationships established
and coherent representations because they employed self-explanation through correlational work. That is, interventions focusing on the acqui-
strategies in response to information germane to understanding the sci- sition of incremental theories signicantly increased adolescents'
entic topic. Poorer learners displayed less evidence of self-explanation achievement test scores (Good, Aronson, & Inzlicht, 2003) and college
strategies overall, and appeared to give equal weight to more and less students' grades (Aronson, Fried, & Good, 2002) relative to controls.
reliable information when they did. Thus, inferring relationships Several empirical reports indicate that theories of intelligence orient
among concepts also appears to support learning from multiple differently towards strategic processes during learning, which contrib-
documents. utes to ultimate success (Blackwell et al., 2007; Dupeyrat & Marin,
2005; Hong et al., 1999; Rhodewalt, 1994; Robins & Pals, 2002).
1.1. Cognitive individual differences that predict multiple document Rhodewalt (1994) demonstrated that individuals holding entity theo-
comprehension ries were more likely to self-handicap, withdrawing effort or procras-
tinating within challenging learning contexts relative to those holding
Empirical research demonstrates that a number of cognitive vari- incremental theories. Hong et al. (1999) compared entity and incre-
ables contribute to the facets of multiple document comprehension mental theorists' responses to setbacks during learning. Incremental
outlined above. Research has investigated three cognitive variables in theorists were more likely to modify their strategies by taking remedial
particular word recognition, prior knowledge, and working memory action after having received negative feedback compared to entity the-
either as variables of interest or as control variables. For example, many orists. Similarly, Blackwell et al. (2007) demonstrated that junior high
empirical studies of multiple document comprehension control for school students holding incremental theories were more likely to re-
prior knowledge when judging whether additional variables explain spond to failures by exibly seeking out new learning and strategies
unique variance in comprehension performance. Even with additional compared to entity theorists and, as a result, increased their achieve-
variables included in a model, prior topic knowledge often remains a sig- ment. In the same vein, Robins and Pals (2002) showed that incremen-
nicant and substantial predictor of multiple document comprehension tal theorists displayed a greater evidence of mastery-oriented strategies
(Brten et al., 2009, 2014; Strms, Brten, & Samuelstuen, 2008). including effort escalation. In contrast, entity theorists blamed their fail-
Moreover, Brten et al. (2013) investigated the contribution of individual ure on low ability, and gave up or perseverated on disadvantageous
differences in word recognition skill for multiple document comprehen- strategies in challenging learning contexts.
sion. As has been shown in prior research focusing on the comprehension Thus, research suggests that adaptive strategy use in challenging
of single texts (Andreassen & Brten, 2010; Cunningham, Stanovich, & task contexts something that incremental theorists do more often
Wilson, 1990; Samuelstuen & Brten, 2005; Stanovich, Cunningham, & supports learning and achievement. Moreover, withdrawing cognitive
Feeman, 1984), word recognition was a unique predictor of multiple doc- effort or perseverating on futile strategies in response to difculties
ument comprehension among lower-secondary school students. Finally, something that is more common for entity theorists proves detrimen-
Banas and Sanchez (2012) demonstrated that individual differences in tal for learning and achievement. Comprehending multiple documents
working memory capacity uniquely inuenced learning of implicit rela- in the current information age can pose quite a challenge for high school
tionships underlying textual materials distributed across multiple web and college students for several reasons (Britt & Aglinskas, 2002;
pages. Goldman et al., 2012; Wiley et al., 2009). First, complex conceptual rela-
Obviously, not all individual reader characteristics of importance are tionships within single texts are often left implicit, requiring that
cognitive in nature. Other research has demonstrated that psychosocial readers strategically elaborate information to establish coherence (Chi,
factors, such as personality traits or dispositions, can strongly inuence 2000; McNamara, Kintsch, Songer, & Kintsch, 1996). Second, documents
individuals' propensities to expend efforts towards achievement and are seldom written for the student's exact inquiry purpose. That is,
performance in a number of contexts and domains. In the current readers must select and re-purpose concepts from single documents
work, we used the cognitive individual difference factors mentioned and infer connections across multiple documents that are otherwise un-
above as control variables to rule out plausible alternative explanations certain. Finally, document authors' motives are not always transparent,
that they produced the effects, as opposed to the personality predictor with documents often originating from unknown or questionable
of focal interest in this article: readers' implicit theories of intelligence. sources that may vary in expertise on the topic, biases, and so forth.
J.L.G. Braasch et al. / Learning and Individual Differences 31 (2014) 1120 13

Taken together, a multiple document context presupposes that readers homogeneous (i.e., middle class) with respect to socioeconomic status.
move beyond simply grasping the content information in each text. Most (77%) were native-born Norwegians who learned Norwegian as
They must also strategically construct links among the key concepts a rst language. The remaining students were raised in Norway in bilin-
presented by different documents, and between each key concept and gual households.
the information source(s) conveying them (Britt & Rouet, 2012).
Given prior research on implicit theories of intelligence (see above), it 2.2. Materials
is reasonable to assume that this individual difference factor may also
play a role in the challenging reading-task context of multiple document 2.2.1. Individual differences
comprehension by promoting or constraining strategic processing of
multiple, variably-reliable documents. Thus, such a reading-task con- Word recognition. Strms, Hagtvet, Lyster, and Rygvold's (1997)
text might be a welcome challenge for incremental theorists leading assessment of word recognition skill presented 360 words written in 30
to their exible use of reading strategies (e.g., evaluating source infor- rows. In every row, four word clusters of three high-frequency words
mation, making inferences by combining information across docu- were printed without spaces (e.g., stopsleepyellow). Students indicated
ments). At the same time, a multiple document comprehension as many words as possible for 3 min by drawing vertical lines between
context might tap into entity theorists' maladaptive strategies (e.g., the words, with the number of correct word clusters used as an index of
perseverating on content at the expense of source evaluation, locally word recognition skill. Maximum score was 120. In the present sample,
processing each text without concern for inferred relationships across the reliability estimate (Cronbach's ) for scores on this measure was
texts). To explore this in the current work, we related secondary school .93.
students' scores on measures of implicit theories of intelligence to their Please note that reading researchers who have studied individual
evaluation and comprehension performance after reading complex, differences in word recognition skills in relation to comprehension of
variably-reliable documents on a scientic issue. text (e.g., Bell & Perfetti, 1994; Cunningham et al., 1990) have demon-
strated that word recognition may be related to text comprehension
1.3. The present study even in adults and, moreover, that it may be an independent predictor
(Stanovich, 2000). For example, Cunningham et al. showed that word
The current work extends investigations of the general detriments of recognition skills accounted for signicant additional variance in the
holding entity theories of intelligence and the general benets of hold- text comprehension of adult college readers even after measures of gen-
ing incremental theories of intelligence. Whereas previous work has fo- eral intelligence, listening comprehension, sentence memory, and vo-
cused on longer-term, macro-level learning and course achievement, cabulary were entered into a regression equation.
we investigate whether these theories are functional in more micro-
level multiple document comprehension contexts. Do beliefs that intel- Prior knowledge. Ten open-ended questions assessed pre-
ligence is xed confer costs for multiple document comprehension, in reading knowledge of the central concepts underlying weather sys-
terms of reducing readers' use of evaluation and integration strategies tems discussed in the documents (e.g., How are rain clouds formed?).
when reading multiple documents? Do beliefs that intelligence is mal- A preliminary version of the measure was constructed in consultation
leable, something that can be modied by using a host of appropriate with an earth scientist with expertise on the topic. Participants re-
strategies, benet multiple document evaluation and integration? ceived 2 points if a response reected complete, accurate understand-
These central research questions were addressed in the current work. ing, 1 point if it was accurate yet incomplete, and 0 for an incorrect
We investigated costs and benets using four measures frequently response or blank. Thus, prior knowledge scores reected 20 possible
viewed as indications of successful multiple document evaluation and points. A Cohen's Kappa interrater reliability score of .82 for 25% of
comprehension. These included the number of concepts from higher- the larger sample of Braasch, Brten, Strms, Anmarkrud, and
quality documents that were retained and included in student- Ferguson (2013) was obtained. Disagreements were resolved in discus-
generated essays, propensities to differentiate between more and less sion; one rater scored the remaining participants. Cronbach's for the
useful documents, critical considerations of trustworthiness in doing sample was .76.
so, and generation of cross-document inferences as a measure of con-
ceptual integration. On the basis of theoretical assumptions and previ- Working memory. Working memory capacity was assessed
ous ndings concerning the inuences of implicit theories of using an adaptation of Swanson and Trahan's (1992) Working Mem-
intelligence on performance outlined above, we expected that entity ory Span Task, which was derived from Daneman and Carpenter's
theories of intelligence would negatively predict multiple document (1980) original Reading Span Test. Twelve sets of unrelated
evaluation and comprehension, while incremental theories would dis- sentences were read aloud with a 2-s interval between each sen-
play positive prediction. tence. Sets gradually increased from two to ve sentences. For each
set, participants were tasked to simultaneously a) comprehend the
sentences well enough to answer a comprehension question about
2. Method
an unknown sentence after reading was completed, and b) remem-
ber the nal words from each of the sentences. After writing a re-
2.1. Participants
sponse to the comprehension question, participants recalled the
sentences' nal words. For each set, participants received one point
Participants were 59 students1 (M age = 17.96, SD = 0.35; 48% fe-
for correct comprehension response and one additional point for
male) completing college preparatory courses at a public upper-
each of the nal words recalled. Because working memory refers to
secondary school in southeast Norway. The sample was relatively
a cognitive system involved in simultaneous processing and storing
of information (Baddeley, 1990), no points were received for a set
if the comprehension response was incorrect. In this way, we en-
sured that participants had comprehended (i.e., processed) the
Seventy-four students from three classrooms took part in a control condition within a sentences and not only treated the task as one of verbal memory
larger intervention study described in Braasch et al. (2013). Of those students, 59 were for the nal words. Maximum score was 54. Cronbach's was .81.
present in two sessions and completed all individual difference, evaluation, and compre-
hension measures included in the current study. There is no reason to believe that they dif-
fered in any particular way from the 15 students that were not included in this study Implicit theories of intelligence. A Norwegian version of Dweck's
because they did not produce a complete data set due to absences or partial participation. (1999) Theories of Intelligence Scale was used (Brten & Strms,
14 J.L.G. Braasch et al. / Learning and Individual Differences 31 (2014) 1120

2004). Four items assessed the degree to which students identied with 2.2.3. Post-reading materials and scoring
an entity theory of intelligence reecting a belief that intelligence is a
xed trait, a personal attribute or quantity that cannot be changed Essay task. Explain the causes of the typical weather patterns in
(e.g., You have a certain amount of intelligence, and you can't really the Pacic Ocean and the processes that make El Nio change these
do much to change it). Four additional items assessed the degree to weather patterns was printed at the top of a sheet of paper. Two inde-
which students identied with an incremental theory of intelligence pendent judges used the 30-concept scoring rubric in Appendix A for
reecting a belief that intelligence is malleable, that is, that individuals 25% of the larger study's essay responses. Interrater agreement (Cohen's
can become more intelligent through their efforts (e.g., You can always Kappa) was .80, and all disagreements were resolved in discussion. One
substantially change how intelligent you are). Entity and incremental judge scored the remaining essays.
items were presented in a mixed order; participants rated agreement
with each statement on a 6-point scale (1 = strongly disagree, 6 = Rank-order and justication tasks. You are now going to rank-
strongly agree). Cronbach's reliabilities were .84 for entity items and order the texts from most (1) to least useful (6) for understanding
.89 for incremental items. the El Nio weather phenomenon was printed at the top of another
sheet of paper. The documents' titles and source features were then
2.2.2. Documents presented with blank spaces to write the ranks. Below you are
The topic was the typical weather patterns across the Pacic Ocean going to explain why you assigned the texts the ranks that you
and El Nio's disruption of them. The rst author identied 30 core con- assigned to them. Provide as much detail as you can in your explana-
cepts underlying typical and El Nio-related weather patterns in con- tion was written on a second page. The remainder of the worksheet
sultation with an earth scientist with expertise on the El Nio weather presented headings (Rank 1 reason, Rank 2 reason, and so forth)
phenomenon (Braasch & Goldman, 2010). The set of concepts, shown with blank lines. Participants may consider a document's usefulness
in Appendix A, guided the development of three more useful docu- using a variety of different criteria based on their interpretation. Like
ments that were consistent with current scientic thinking and evi- Barzilai and Zohar (2012), we used a general usefulness prompt to in-
dence. Appendix B provides a brief description of the source features vestigate to which extent students raised trustworthiness issues in
and content information in each of the more useful documents. their ranking justications spontaneously, that is, without a direct
To ensure that our document set closely mirrors the kinds of diverse prompt to do so. Evaluating usefulness in light of trustworthiness is
information sources that students might encounter when reading mul- typically not done with less successful students, but when it occurs
tiple documents, three less useful documents were also designed and spontaneously it facilitates multiple document comprehension as is
presented. All were based on authentic yet unsubstantiated, less reliable more typically the case for good students and experts (Goldman
accounts found on the Internet (see brief descriptions of each et al., 2012; Wineburg, 1991).
document's source features and content information in Appendix B). To represent students' abilities to discriminate among diverse docu-
Across the three less useful documents, a total of eight erroneous con- ments, several multiple document researchers have relied upon ranking
cepts were reported. We note that, with authentic documents, useful- discrimination scores (Braasch et al., 2009; Wiley et al., 2009). In align-
ness, reliability, and accuracy are all inextricably correlated. ment with this work, we computed a difference score for each student
Documents containing accurate information on a topic conveyed by re- by subtracting their average rankings for the three more useful from
liable sources will necessarily provide the most useful information for their average rankings for the three less useful documents. As such,
answering an inquiry question. The converse is also true: Inaccurate in- higher positive ranking discrimination scores reected better differenti-
formation conveyed by questionable or unreliable sources will neces- ation; ranking discrimination scores closer to 1 signied a lack of
sarily be the least useful in an inquiry context. discrimination.
To conrm a priori distinctions of the usefulness of the documents, Rank-order justications were coded for the presence of several
we tasked six multiple document reading experts to independently types of content-based justications that fall outside the scope of the
read then rank-order all six documents using the same prompt that current work (see Braasch et al., 2013, for the complete set of catego-
was provided for the participating students (see below). Average expert ries). For example, a statement like I ranked this document third be-
ranks for the three more useful documents, that is, for the textbook ex- cause the title interested me received a coding of interest; a sentence
cerpt (M = 2.17, SD = 0.98), the Illustrated Science article (M = 1.83, like I ranked this last because it was complicated received a code for
SD = 0.75), and the popular science article (M = 2.00, comprehensibility. We additionally coded for mentions of the informa-
SD = 0.89), were all approximately 2 (with 1 being the most useful tion sources (author, publication, venue, type, or date). For example, a
designation and 6 being the least). Average expert ranks for the three statement such as I ranked this fourth because it was published in
less useful documents, that is for the environmental blog entry 2002 received a code of publication date. Of interest for the current
(M = 4.67, SD = 0.51), the political commentary article (M = 4.33, work, we coded each justication for an indication of a more critical re-
SD = 0.52), and the astrological newsletter (M = 6.00, SD = 0.00), liability assessment. These took many forms. Positive trustworthiness
were considerably higher. Thus, the six documents represented a assessments included mentions of trustworthiness or reliability, or
range of usefulness for answering the focal inquiry question, with ex- whether the information had been quality-checked or reviewed by
perts corroborating a priori distinctions. experts prior to publication, to name a few. Negative assessments men-
For all documents, a bolded title (containing the key words typical tioned a lack of trustworthiness or reliability, biases to persuade or pro-
weather patterns or El Nio) was followed by the source feature informa- mote the author's case, or mentions of subjectivity or one-sidedness.
tion (author, type, venue, publication date), which was in turn followed Each justication entry was coded for assessment of the trustworthiness
by the text's content information. As an indication of text difculty, we of the document for that particular rank-order decision. Thus, trustwor-
computed readability scores for each of the six documents based on the thiness mention scores were out of six. Two independent judges used
formula proposed by Bjrnsson (1968), which is based on word length the various coding categories described in Braasch et al. (2013) to
and sentence length. When using this formula, readability scores nor- score a random selection of 25% of the justication protocols. The
mally range from 20 (very easy text) to 60 (very difcult text). The Cohen's Kappa agreement for all categories was .75. All disagreements
mean readability score of the three more useful documents was 50.0 were resolved in discussion between the two raters. One judge scored
and the mean readability score of the three less useful documents was the remaining participants.
44.3. According to readability standards presented by Bjrnsson
(1983), all documents could be described as average to difcult texts Intertextual inference verication task (IIVT). Previous research
typical of factual prose. has demonstrated that intertextual verication tasks provide a fruitful
J.L.G. Braasch et al. / Learning and Individual Differences 31 (2014) 1120 15

way to measure the inferred connections that readers make among con- 3. Results
cepts distributed across multiple documents (Brten et al., 2009;
Strms et al., 2008). In alignment with the previous research, we creat- Descriptive statistics as well as zero-order correlations for all mea-
ed a verication task to measure the degree to which students could sured variables are reported in Table 1. To answer our research ques-
make inferences by combining information from the more useful docu- tions regarding the unique predictability of entity and incremental
ments that we provided. Please note, however, that the IIVT was created theories of intelligence for aspects of multiple document comprehen-
for this particular research and has not been used in any previous stud- sion, we computed four regression equations. Outcome measures
ies. Twenty sentences (7 valid, 13 invalid inferences) were constructed were students' inclusion of scientic concepts in their essays (i.e., num-
such that none could be answered correctly by judging the validity of ber of scientic concepts), their abilities to discriminate amongst the
only one piece of information at a time, but rather involved cross- more and less useful documents (i.e., ranking discriminations), their
textual inferences about information from at least 2 of the 3 more useful mentioning of trustworthiness issues in their ranking justications
documents. Thus, readers had to consider the whole, integrated mean- (i.e., trustworthiness assessments), and their abilities to draw infer-
ing of each sentence to decide whether it was a valid or an invalid infer- ences across documents (i.e., IIVT scores). In addition to using the two
ence. Appendix C provides illustrative IIVT examples documenting the theories of intelligence measures as predictors for each of these out-
concepts that were covered for each item, the texts from which the con- comes, we included word recognition, prior knowledge, and working
cepts could be gleaned, and explanations concerning the connections memory to control for variance in the outcome measures due to these
among the concepts that one would have to make to respond correctly. individual difference measures.
Performance scores reected the number of correct inference distinc- Table 2 shows the results of the regression analyses for all four out-
tions out of 20 items. Cronbach's was .64. come variables. The ve predictors together explained a statistically sig-
While the reliability estimate of the IIVT was lower than desirable, it nicant amount of variance in all four outcome measures, with F(5,
was comparable to those obtained in other studies using similar tasks 52) = 6.46, p = .000 (R2 = .38) for the number of scientic concepts,
(Brten et al., 2009; Brten & Strms, 2010; Salmern, Gil Brten & F(5, 52) = 4.17, p = .003 (R2 = .29) for the ranking discriminations,
Strms, 2010). Presumably, the somewhat low reliability of the IIVT F(5, 52) = 3.18, p = .014 (R2 = .23) for the trustworthiness assess-
was related to the relatively short length of the scale. Thus, Royer, ments, and F(5, 53) = 2.63, p = .034 (R2 = .20) for the IIVT. Moreover,
Carlo, Dufresne, and Mestre (1996) suggested that to obtain a really for the number of scientic concepts ( = .41, p = .012), for the trust-
high reliability estimate, a verication task might have to include almost worthiness assessments ( = .35, p = .037), and for the IIVT ( = .43,
100 items, which was not feasible given the time available for data col- p = .017), students' endorsement of an incremental theory of intelli-
lection in this study. gence was a unique positive predictor, while it for the ranking dis-
criminations did not quite reach a conventional level of statistical
2.3. Procedure signicance ( = .31, p = .055). In contrast, no statistically signicant
results were obtained for students' endorsement of an entity theory of
2.3.1. Day 1: Assessing reader characteristics intelligence. Among the control variables, prior knowledge uniquely
Participants were rst group-administered the word recognition predicted students' inclusion of scientic concepts in their essays
(5 min) and working memory measures (20 min). Then, for 25 min ( = .57, p = .000) and trustworthiness assessments ( = .44,
they independently completed the prior knowledge and implicit theo- p = .003), while working memory uniquely predicted students' abili-
ries of intelligence assessments. ties to discriminate between more and less useful documents
( = .41, p = .003) as well as their abilities to draw inferences across
2.3.2. Day 2: Multiple document comprehension assessments documents ( = .35, p = .015). Finally, word recognition was not a
Eleven days later, participants were provided with a folder con- unique predictor of any of the outcome measures at this educational
taining the six documents, the essay response sheet, the rank-order level.
and justication tasks, and the IIVT. Students were told that the doc-
uments resulted from a Google search using the key words El Nio 4. Discussion
changes typical weather in the Pacic Ocean. They were allotted
30 min of reading time to be able to explain the causes of the typical This study uniquely contributes to research on implicit theories of in-
weather patterns in the Pacic Ocean and the processes that make El telligence as well as on multiple document reading. To the best of our
Nio change these weather patterns. Although the documents were knowledge, it is the rst study that examined the role of this individual
presented in counterbalanced orders, students could read and reread difference variable in the context of learning from multiple information
any information that they felt would help them to answer the inquiry sources. Moreover, by including both entity and incremental views on
question in any order. intelligence in the same regression equations and, at the same time, con-
After the reading time expired, students replaced the documents in trolling for several other reading-related variables, we were able to
the folder. They then wrote their essays for up to 20 min without having probe the relative as well as the unique contribution of the two implicit
access to the documents. Students then replaced the essay response theories to multiple document evaluation and integration.
sheets and took out the rank-order and justication evaluation tasks. As expected, beliefs in intelligence as a malleable quality uniquely
For 15 min, students rank-ordered the documents from most (1) to and positively predicted performance for all outcome measures. Specif-
least useful (6) for understanding the El Nio weather phenomenon ically, the multiple regression analyses showed that students holding
and provided justications for their rank-order assignments. After plac- such beliefs were more likely to include concepts from higher quality
ing the rank-order and justication tasks back in their folders, they re- documents in their essays, to discriminate between more and less use-
moved the IIVT task. Instructions were to mark each test sentence ful documents given the task, to consider the trustworthiness of the
with a yes if the statement could be inferred by combining information documents as a basis for discrimination, and to construct deeper, inter-
from one or more of the documents that were just read and no if it could textual understandings of the documents by drawing bridging infer-
not. Twenty minutes were allotted for the IIVT. ences between them. Presumably, in accordance with other work on
It should be noted that the sequence of tasks was used to minimize the role of students' implicit theories of intelligence in learning and per-
reactivity. That is, we intended that participants should write essays formance (e.g., Blackwell et al., 2007; Hong et al., 1999; Robins & Pals,
based on the mental representations that they constructed during read- 2002), beliefs in intelligence as a malleable quality facilitated students'
ing without cues from source information (rank-order and justication employment of strategies that help them meet the challenges of
evaluation tasks) or concepts presented in the IIVT task. comprehending multiple documents. Specically, incremental theorists
16 J.L.G. Braasch et al. / Learning and Individual Differences 31 (2014) 1120

Table 1
Descriptive statistics and zero-order correlations for all measured variables.

Variable 1 2 3 4 5 6 7 8 9

1. Word recognition
2. Prior knowledge .37
3. Working memory .25 .38
4. Entity theory .06 .13 .27
5. Incremental theory .23 .19 .21 .73
6. Number of science concepts .16 .55 .35 .09 .03
7. Ranking discriminations .07 .11 .25 .33 .34 .19
8. Trustworthiness assessments .24 .41 .16 .10 .19 .45 .36
9. Intertextual inferences .22 .08 .29 .02 .19 .30 .15 .23
M 55.62 6.18 20.87 4.61 2.68 3.80 1.22 1.19 10.23
SD 11.98 3.71 10.33 .94 1.02 4.34 1.85 1.53 3.45
Skewness .04 .29 .12 .52 .20 1.32 .87 1.61 .22
p b .05.
p b .01.
p b .001.

may have sustained their cognitive engagement in the face of uncertain them, posed enough of a risk of revealing that they were not endowed
outcomes, if merely to take advantage of the opportunity to gain new with sufcient intelligence. This would reduce the saliency and function
understandings (Dweck, 1999; Hong et al., 1999; Robins & Pals, 2002). of entity theories. However, the same task may have been enough of a
An incremental theory of intelligence may also involve greater cognitive welcome challenge for students endorsing incremental theories. The op-
exibility (Dweck, 1999), which, according to cognitive exibility theo- portunity to learn may have oriented them towards strategic processing
ry (Spiro, Coulson, Feltovich, & Anderson, 1994) may be needed to gain and possibly sustained engagement. Because the current information
rich and exible understandings of complex knowledge distributed age guarantees that readers will need to incorporate several kinds of
across multiple sources. reading strategies when dealing with complex and variably-reliable doc-
On the other hand, beliefs in intelligence as a xed, unchangeable uments, students' propensities to rise to such challenges might create
quantity did not emerge as a unique negative predictor, contrary to ex- gaps in knowledge acquisition and integration between those who en-
pectations. Thus, although prior research on implicit theories of intelli- dorse incremental and entity theories. Intriguing future research could
gence has indicated that entity theorists are prone towards disengaging investigate longer-term benets of incremental theories of intelligence
from tasks posing challenges and perseverating on inappropriate strate- when secondary school students must accrue knowledge from multiple
gies (Blackwell et al., 2007; Rhodewalt, 1994; Robins & Pals, 2002), com- information sources over prolonged periods of time (e.g., over the course
prehension costs were not apparent in this study. In the complex reading of a semester).
task context that we created, the benets of endorsing an incremental Thus, while entity-oriented learners that are anxiously trying to
theory therefore seemed more pronounced than the costs of endorsing prove themselves (Molden & Dweck, 2006) may be strategically
an entity theory of intelligence. One possible reason for this might be handicapped when they perceive the stakes to be high for example
that an incremental theory and an entity theory are not located at the op- in a competitive setting where their performance relative to others
posite ends of a single construct. This possibility is unlikely, however, also becomes public the maladaptive strategic pattern found to char-
given that about 85% of students (later grade school through college) acterize entitists (Dweck & Leggett, 1988) may not bear out in lower
have been shown to agree consistently with either an incremental theory stakes testing environments like the one used in this study. On the
or with an entity theory and thus hold a clear theory about intelligence other hand, incremental-oriented learners that are excitingly exploring
(evenly split among the two theories) (Dweck, 2002; Dweck & Molden, new realms of knowledge (Molden & Dweck, 2006) may be concerned
2005). Please also note that a strong negative correlation between the in- with strategically mastering the materials whatever the stakes and
cremental and entity theory scores in this and other research (e.g., Brten thus perceive the multiple document task context that we created as
& Strms, 2006) shows that there is a clear distinction between the two an opportunity to gain new knowledge and experience. This implies
implicit theories, in the sense that students who are high in incremental that our ndings may contribute to an understanding of each implicit
beliefs are low in entity beliefs and vice versa. We would therefore sug- intelligence theory's functionality in lower stake testing environments,
gest that another, more likely, possibility is that students endorsing entity with entity theories only detrimental in higher stake situations and in-
theories did not perceive that the task of reading to learn from multiple cremental theories benecial regardless of the stakes involved. Future
documents in this setting, without any real, public consequences for research in this area might experimentally manipulate the stakes

Table 2
Results of multiple regression analyses for variables predicting number of scientic concepts, ranking discriminations, trustworthiness assessments, and intertextual inferences.

Predictor Scientic concepts Ranking discriminations Trustworthiness assessments Intertextual inferences


Word recognition .07 .05 .18 .03 .02 .17 .00 .02 .03 .01 .04 .02
Prior knowledge .67 .15 .57 .03 .07 .06 .18 .06 .44 .05 .14 .06
Working memory .08 .05 .19 .07 .02 .41 .01 .02 .06 .12 .05 .35
Entity theory 1.17 .77 .25 .47 .36 .23 .15 .30 .09 .90 .68 .25
Incremental theory 1.73 .73 .41 .55 .34 .31 .53 .29 .35 1.44 .66 .43
p b .05.
p b .01.
p b .001.
J.L.G. Braasch et al. / Learning and Individual Differences 31 (2014) 1120 17

involved for example by putting more emphasis on competition and interaction effects between implicit theories of intelligence and the
publicity to investigate whether implicit theories of intelligence pro- other reading-related individual difference variables on multiple
vide different contributions to multiple document comprehension de- document evaluation and integration. Finally, because the correla-
pendent on the perceived stakes. On a related note, our ndings may tional data that we analyzed do not warrant conclusions regarding
also suggest that entity theories of intelligence are more culturally sen- causality, further experimental work is needed to more rmly estab-
sitive than incremental theories in the sense that the former operate lish causal relations between students' implicit theories of intelli-
less similarly and strongly across cultures than the latter. More speci- gence and aspects of their multiple document comprehension.
cally, in the context of the egalitarian Norwegian educational system, Despite such limitations, our ndings may suggest some important
with relatively little emphasis on competition and individual distinction implications not only for theory but also for educational practice. While
(Undheim, Nordvik, Gustafsson, & Undheim, 1995), the task that we cognitive variables have thus far taken the front seat in work on multiple
created may have been perceived as less challenging or threatening document comprehension, this study uniquely contributes to the litera-
than it would have been in a less egalitarian, more competitive system, ture by indicating that not only the cognitive but also the personality sys-
with the cost of endorsing an entity theory of intelligence therefore not tem of readers is involved in evaluating and learning from multiple
apparent on such a task in this particular context. More research is cer- information sources. Moreover, it indicates that personality in terms of
tainly warranted that investigates whether students perceive the stakes self-theories concerning readers' beliefs about their intelligence (Dweck,
involved in similar tasks differently across cultures, with a possible con- 2008; Dweck & Leggett, 1988; Molden & Dweck, 2006) plays a role in
sequence being that entity theories of intelligence contribute differently such complex reading-task contexts independent of cognitive variables.
to performance on similar tasks in different cultures. However, in addition to contributing to eshing out a more comprehen-
Of note is that the unique predictability of incremental beliefs about sive theory of individual differences that underlie performance on essen-
intelligence was observed when several well-established predictors of tial 21st century literacy tasks, our study may inform educators about
reading competence were simultaneously controlled for. With respect reader characteristics that may facilitate adaptive strategic processing of
to the control variables, prior knowledge and working memory were multiple documents. Importantly, these reader characteristics should
found to uniquely predict different important aspects of multiple docu- not be regarded as static (Dweck & Molden, 2005). To the contrary,
ment comprehension. It was not surprising that prior topic knowledge prior intervention work (Aronson et al., 2002; Blackwell et al., 2007;
predicted students' inclusion of scientic concepts in their essays. Howev- Good et al., 2003) has shown that students can be taught an incremental
er, prior knowledge was also the better predictor of students' trustworthi- theory of intelligence, with incremental messages conveyed during in-
ness assessments, suggesting that they used their knowledge of struction, in turn, improving student learning and performance. Future
conceptual content, rather than source information, when judging the work should therefore examine the extent to which teaching an incre-
trustworthiness of the documents (cf., Brten et al., 2011; Rouet et al., mental theory of intelligence and how to apply it to multiple document
1996). In contrast, working memory was the better predictor of students' reading may promote students' learning from heterogeneous textual
ability to discriminate between more and less useful documents as well materials.
their performance on the IIVT. The IIVT may have required working mem-
ory resources because successful performance involves not only compre- Appendix A. Core concepts in the domain and their distribution
hension of and memory for individual documents, but also constructions across the three more useful documents
of inferential links among them (Wiley, Grifn, & Thiede, 2005). An inter-
pretation of the unique predictability of working memory for ranking dis-
criminations is more speculative at present. It is important to revisit that
Textbook Magazine Research
the students did not have access to the full texts when ranking the useful-
excerpt article website
ness of the documents. Thus, when using document content to help de-
Typical weather pattern concepts
cide upon an order of usefulness, they would have to retrieve content
High air pressure in east
information from two or more documents from long-term memory to Low air pressure in west
compare and rank order them. Current characterizations of working Trade winds east to west
memory highlight its role in accessing and utilizing information from Water is dragged west
long-term memory during complex, attention-demanding tasks Sun heats water
Warm water bulge in western Pacic
(Baddeley, 2000), which may have supported rank-order performance Warm air temperatures
for these participants. Finally, although students' word recognition skill High levels of evaporation
has been shown to uniquely predict their learning from multiple docu- less dense, air rises
ments among younger students (Brten et al., 2013), one likely reason Cooling reduces moisture
Cloud formation
for the lack of contribution of word recognition in this study may be
Heavy rainfall in west
that most of the participants had reached a level so high that further im- Cool water upwelling in eastern Pacic
provement did not matter much in terms of performance (cf., Strms Cooler air temperatures
et al., 2008). Future work in this area could therefore supplement or re- Low levels of evaporation
place measures of word recognition skills with measures of reading com- Denser air does not rise
Less cloud formation
prehension when trying to isolate variance due to implicit theories of Lack of rainfall in east
intelligence. Walker circulation perpetuates patterns
Of course, our ndings are contextualized not only by the Norwe-
El Nio-related weather concepts
gian sample of secondary school students that participated but also
Pressure difference weakens
by the particular text materials and tasks that we presented to Trade winds relax
them. Future work examining the role of implicit theories of intelli- Warm water bulge is in east
gence in multiple document evaluation and integration should High evaporation
Warm moist air
therefore probe the generalizability of our ndings. For example, re-
Less dense, rises
search could include other student populations working with multi- Cloud formation
ple documents on other issues and demonstrating their competence Heavy rainfall in east
on other outcome measures. As indicated above, cross-cultural re- Cool water upwelling in west
search in this area is also warranted. Moreover, future work should Low evaporation
Lack of rainfall in west
use larger sample sizes, also allowing for the testing of possible
18 J.L.G. Braasch et al. / Learning and Individual Differences 31 (2014) 1120

Appendix B. A description of more and less useful documents' source features and content information

More useful documents Description of source features Description of content information

Typical weather patterns in the Pacic An excerpt from an upper secondary school geography textbook This document explains the relationships between atmospheric
Ocean: When the air meets the water 444 written by a science teacher published by a well-known processes and differences in oceanic temperatures, and that these
words publishing house in 2007 processes result in predictable weather patterns in the western and
eastern Pacic Ocean.
What causes typical weather patterns in the Popular science article for Illustrated Science magazine written by This document explains the reasons that the different ocean
Pacic Ocean and how do scientists predict a researcher (oceanographer) at an ocean research institute temperatures across the Pacic Ocean result in the typical weather
anomalies like El Nio? 565 words published in 2002 patterns in the western and eastern Pacic Ocean by informing on
chemical processes associated with the water cycle.
A world of change the El Nio weather A popular science article from an online research magazine The document explains that El Nio events are triggered when air
phenomenon 520 words published by a group of research and educational institutions pressure differentials decrease across the Pacic Ocean, causing
( written by a senior researcher at a anomalous weather patterns.
meteorological institute published in 2011

Less useful documents Description of source features Description of content information

El Nios on the rise We only have An environmental blog entry written by a freelance writer inThis document explains that El Nio events are becoming stronger
ourselves to blame!!! 566 words 2008 and more frequent over the past several decades because of
changes in the direction of global warming. It claims that increases
in global warming due to manmade causes have weakened the
typical relationships between air pressure and air movement,
thereby increasing the frequency of El Nio events. The document
makes inappropriate causal interpretations of computer simulation
data and uses anecdotal evidence.
Is global warming increasing the occurrences A commentary article from a Norwegian liberal daily newspaper This document also claims that El Nio events are becoming stronger
of El Nios? What we do and do not know written by a political editor in 2004 and more frequent over the past several decades, related to global
547 words warming. A computer simulation nding is used as support. Although
this article goes on to present some researchers' cautions against
conclusions based solely on computer simulation data (e.g., inaccu-
rate prediction), the author ultimately rejects the cautions outright.
Instead, she makes a call to arms for more serious environmental
regulations on the use of fossil fuels. Thus, there are implications that
her political agenda motivates what was presented.
El Nio comes from space, not humanity 521 A newsletter excerpt written by a global astrologer for his This document explains that weather anomalies such as El Nio
words personal website in 2011 events result from cosmic forces (e.g., the alignment of planetary
bodies). As such, this document is less useful due to an
inappropriate reasoning based on circumstantial, unscientic

Appendix C. Example items from the intertextual inference verication task

Example IIVT items Concepts involved Texts conveying Explanation

these concepts

Droughts tend to occur in areas Typically, the eastern Pacic experiences low levels More useful text 2 This is a valid inference because, during typical and El
experiencing lower evaporation. of evaporation. Nio-related weather patterns, areas with lower
Typically, there is a lack of rainfall in the eastern More useful text 2 levels of evaporation experience a less rainfall and
Pacic. droughts (relative to regions of higher evaporation).
Response YES During El Nio, the western Pacic experiences More useful text 3
low levels of evaporation.
During El Nio, there is a lack of rainfall in the More useful text 3
western Pacic.
During an El Nio event, the relaxation During El Nio, the trade winds relax. More useful text 3 This is a valid inference because a relaxation of the
of the trade winds results in a rainy Typically, there is a lack of rainfall in east. More useful text 1 and 2 trade winds would increase the amount of warm
season in the eastern Pacic Ocean. water, evaporation, condensation, and ultimately the
Response YES During El Nio, there is heavy rainfall in east. More useful text 3 amount of rainfall in the eastern Pacic relative to
typically dry conditions.
Within the Pacic Ocean, upwelling Typically, the eastern Pacic has higher air More useful text 1 This is an invalid inference because, during both
occurs in regions with relatively lower pressure. typical and El Nio-related weather patterns, areas
surface air pressure. Typically, upwelling occurs in the eastern More useful text 1 with higher air pressure experience upwelling.
Response NO During El Nio, air pressure increases in More useful text 2
western Pacic.
During El Nio, upwelling increases in western Pacic. More useful text 2
J.L.G. Braasch et al. / Learning and Individual Differences 31 (2014) 1120 19

References Daneman, M., & Carpenter, P. A. (1980). Individual differences in working memory and
reading. Journal of Verbal Learning and Verbal Behavior, 19, 450466.
Aferbach, P., & Cho, B. -Y. (2009). Identifying and describing constructively responsive Dupeyrat, C., & Marin, C. (2005). Implicit theories of intelligence, goal orientation, cogni-
comprehension strategies in new and traditional forms of reading. In S. E. Israel, & tive engagement, and achievement: A test of Dweck's model with returning to school
G. G. Duffy (Eds.), Handbook of research on reading comprehension (pp. 6990). New adults. Contemporary Educational Psychology, 30, 4359.
York: Routledge. Dweck, C. S. (1999). Self-theories: Their role in motivation, personality, and development.
Alexander, P. A. (2012). Reading into the future: Competence for the 21st century. Philadelphia, PA: Psychology Press.
Educational Psychologist, 47, 259280. Dweck, C. S. (2002). The development of ability conceptions. In A. Wigeld, & J. S. Eccles
Andreassen, R., & Brten, I. (2010). Examining the prediction of reading comprehension (Eds.), Development of achievement motivation (pp. 5788). San Diego, CA: Academic
on different multiple-choice texts. Journal of Research in Reading, 33, 263283. Press.
Anmarkrud, ., Brten, I., & Strms, H. I. (2014). Multiple-documents literacy: Strategic Dweck, C. E. (2008). Can personality be changed? The role of beliefs in personality and
processing, source awareness, and argumentation when reading multiple conicting change. Current Directions in Psychological Science, 17, 391394.
documents. Learning and Individual Differences (in press). Dweck, C. S., & Leggett, E. L. (1988). A social-cognitive approach to motivation and per-
Aronson, J., Fried, C. B., & Good, C. (2002). Reducing the effects of stereotype threat on sonality. Psychological Review, 95, 256273.
African-American college students by shaping their theories of intelligence. Journal Dweck, C. S., & Master, A. (2008). Self-theories motivate self-regulated learning. In D. H.
of Experimental Social Psychology, 38, 113125. Schunk, & B. J. Zimmerman (Eds.), Motivation and self-regulated learning: Theory, re-
Baddeley, A. (1990). Human memory: Theory and practice. Hove and London: Erlbaum. search, and applications (pp. 3151). New York: Erlbaum.
Baddeley, A. (2000). The episodic buffer: A new component of working memory? Trends Dweck, C. S., & Molden, D. C. (2005). Self-theories: Their impact on competence motiva-
in Cognitive Sciences, 4, 417423. tion and acquisition. In A. J. Elliot, & C. S. Dweck (Eds.), Handbook of competence and
Banas, S., & Sanchez, C. A. (2012). Working memory and learning underlying conceptual motivation (pp. 122140). New York: Guilford.
relationships across multiple documents. Applied Cognitive Psychology, 26, 594600. Faria, L., & Fontaine, A. -M. (1997). Adolescents' personal conceptions of intelligence: The
Barzilai, S., & Zohar, A. (2012). Epistemic thinking in action: Evaluating and integrating development of a new scale and some exploratory evidence. European Journal of Psy-
online sources. Cognition and Instruction, 30, 3985. chology of Education, 12, 5162.
Bell, L. C., & Perfetti, C. A. (1994). Reading skill: Some adult comparisons. Journal of Goldman, S. R. (2004). Cognitive aspects of constructing meaning through and across
Educational Psychology, 86, 244255. multiple texts. In N. Shuart-Faris, & D. Bloome (Eds.), Uses of intertextuality in class-
Bjrnsson, C. H. (1968). Lsbarhet [Readability]. Stockholm: Liber. room and educational research (pp. 317351). Greenwich, CT: Information Age.
Bjrnsson, C. H. (1983). Readability of newspapers in 11 languages. Reading Research Goldman, S. R., Braasch, J. L. G., Wiley, J., Graesser, A.C., & Brodowinska, K. (2012).
Quarterly, 18, 480497. Comprehending and learning from Internet sources: Processing patterns of better
Blackwell, L. S., Trzesniewski, K. H., & Dweck, C. S. (2007). Implicit theories of intelligence and poorer learners. Reading Research Quarterly, 47, 356381.
predict achievement across an adolescent transition: A longitudinal study and an in- Good, C., Aronson, J., & Inzlicht, M. (2003). Improving adolescents' standardized test per-
tervention. Child Development, 78, 246263. formance: An intervention to reduce the effects of stereotype threat. Journal of
Braasch, J. L. G., Brten, I., Strms, H. I., Anmarkrud, ., & Ferguson, L. E. (2013). Promot- Applied Developmental Psychology, 24, 645662.
ing secondary school students' evaluation of source features of multiple documents. Graesser, A.C., Wiley, J., Goldman, S. R., O'Reilly, T., Jeon, M., & McDaniel, B. (2007). SEEK
Contemporary Educational Psychology, 47, 447479. Web tutor: Fostering a critical stance while exploring the causes of volcanic eruption.
Braasch, J. L. G., & Goldman, S. R. (2010). The role of prior knowledge in learning from Metacognition and Learning, 2, 89105.
analogies in scientic texts. Discourse Processes, 47, 447479. Greene, J. A., Costa, L. -J., Robertson, J., Pan, Y., & Deekens, V. M. (2010). Exploring relations
Braasch, J. L. G., Lawless, K. A., Goldman, S. R., Manning, F. H., Gomez, K. W., & MacLeod, S. among college students' prior knowledge, implicit theories of intelligence, and
M. (2009). Evaluating search results: An empirical analysis of middle school students' self-regulated learning in a hypermedia environment. Computers & Education, 55,
use of source attributes to select useful sources. Journal of Educational Computing 10271043.
Research, 41, 6382. Henderson, V. L., & Dweck, C. S. (1990). Achievement and motivation in adolescence: A
Brand-Gruwel, S., & Stadtler, M. (2011). Solving information-based problems: Evaluating new model and data. In S. Feldman, & G. Elliot (Eds.), At the threshold: The developing
sources of information. Learning and Instruction, 21, 175179. adolescent. Cambridge, MA: Harvard University Press.
Brten, I., Anmarkrud, ., Brandmo, C., & Strms, H. I. (2014). Developing and testing a Hong, Y. Y., Chiu, C. Y., Dweck, C. S., Lin, D., & Wan, W. (1999). Implicit theories, attribu-
model of direct and indirect relationships between individual differences, processing, tions, and coping: A meaning system approach. Journal of Personality and Social
and multiple-text comprehension. Learning and Instruction, 30, 924. Psychology, 77, 588599.
Brten, I., Ferguson, L. E., Anmarkrud, ., & Strms, H. I. (2013). Prediction of learning Leu, D. J., Kinzer, C. K., Coiro, J., Castek, J., & Henry, L. A. (2013). New literacies: A
and comprehension when adolescents read multiple texts: The roles of word-level dual-level theory of the changing nature of literacy, instruction, and assessment.
processing, strategic approach, and reading motivation. Reading and Writing: An In D. E. Alvermann, N. J. Unrau, & R. B. Ruddell (Eds.), Theoretical models and pro-
Interdisciplinary Journal, 26, 321348. cesses of reading (pp. 11501181) (6th ed.). Newark, DE: International Reading
Brten, I., & Strms, H. I. (2004). Epistemological beliefs and implicit theories of intelligence Association.
as predictors of achievement goals. Contemporary Educational Psychology, 29, 371388. McNamara, D. S., Kintsch, E., Songer, N.B., & Kintsch, W. (1996). Are good texts always
Brten, I., & Strms, H. I. (2006). Predicting achievement goals in two different academic better? Interactions of text coherence, background knowledge, and levels of under-
contexts: A longitudinal study. Scandinavian Journal of Educational Research, 50, 127148. standing in learning from texts. Cognition and Instruction, 14, 143.
Brten, I., & Strms, H. I. (2010). When law students read multiple documents about Molden, D. C., & Dweck, C. S. (2006). Finding "meaning" in psychology: A lay theories ap-
global warming: Examining the role of topic-specic beliefs about the nature of proach to self-regulation, social perception, and social development. American
knowledge and knowing. Instructional Science, 38, 635657. Psychologist, 61, 192203.
Brten, I., & Strms, H. I. (2011). Measuring strategic processing when students read Rhodewalt, F. (1994). Conceptions of ability, achievement goals, and individual differ-
multiple texts. Metacognition and Learning, 6, 111130. ences in self-handicapping behavior: On the application of implicit theories. Journal
Brten, I., Strms, H. I., & Britt, M.A. (2009). Trust matters: Examining the role of source of Personality, 62, 6785.
evaluation in students' construction of meaning within and across multiple texts. Robins, R. W., & Pals, J. (2002). Implicit self-theories in the academic domain: Implications
Reading Research Quarterly, 44, 628. for goal orientation, attributions, affect, and self-esteem change. Self and Identity, 1,
Brten, I., Strms, H. I., & Salmern, L. (2011). Trust and mistrust when students read 313336.
multiple information sources about climate change. Learning and Instruction, 21, Rouet, J. -F., Britt, M.A., Mason, R. A., & Perfetti, C. A. (1996). Using multiple sources of ev-
180192. idence to reason about history. Journal of Educational Psychology, 88, 478493.
Britt, M.A., & Aglinskas, C. (2002). Improving students' ability to identify and use source Royer, J. M., Carlo, M. S., Dufresne, R., & Mestre, J. (1996). The assessment of levels of do-
information. Cognition and Instruction, 20, 485522. main expertise while reading. Cognition and Instruction, 14, 373408.
Britt, M.A., & Gabrys, G. (2002). Implications of document-level literacy skills for web-site Salmern, L., Gil, L., Brten, I., & Strms, H. I. (2010). Comprehension effects of signalling
design. Behavior Research Methods, Instruments, and Computers, 34, 170176. relationships between documents in search engines. Computers in Human Behavior,
Britt, M.A., Perfetti, C. A., Sandak, R., & Rouet, J. -F. (1999). Content integration and source 26, 419426.
separation in learning from multiple texts. In S. R. Goldman, A.C. Graesser, & P. van Samuelstuen, M. S., & Brten, I. (2005). Decoding, knowledge, and strategies in compre-
den Broek (Eds.), Narrative, comprehension, causality, and coherence: Essays in honor hension of expository text. Scandinavian Journal of Psychology, 46, 107117.
of Tom Trabasso (pp. 209233). Mahwah, NJ: Erlbaum. Spiro, R. J., Coulson, R. L., Feltovich, P. J., & Anderson, D. K. (1994). Cognitive exibility the-
Britt, M.A., & Rouet, J. F. (2012). Learning with multiple documents: Component skills and ory: Advanced knowledge acquisition in ill-structured domains. In R. B. Ruddell, M. R.
their acquisition. In J. R. Kirby, & M. J. Lawson (Eds.), Enhancing the quality of learning: Ruddell, & H. Singer (Eds.), Theoretical models and processes of reading (pp. 602615)
Dispositions, instruction, and learning processes (pp. 276314). New York: Cambridge (4th ed.). Newark, DE: International Reading Association.
University Press. Stadtler, M., & Bromme, R. (2008). Effects of the metacognitive computer-tool met.a.ware
Britt, M.A., Rouet, J. -F., & Braasch, J. L. G. (2013). Documents experienced as entities: Extend- on the web search of laypersons. Computers in Human Behavior, 24, 716737.
ing the situation model theory of comprehension. In M.A. Britt, S. R. Goldman, & J. -F. Stanovich, K. E. (2000). Progress in understanding reading: Scientic foundations and new
Rouet (Eds.), Reading from words to multiple texts (pp. 160179). New York: Routledge. frontiers. New York: Guilford.
Chi, M. T. H. (2000). Self-explaining expository texts: The dual processes of generating in- Stanovich, K. E., Cunningham, A. E., & Feeman, D. J. (1984). Intelligence, cognitive skills,
ferences and repairing mental models. In R. Glaser (Ed.), Advances in instructional psy- and early reading progress. Reading Research Quarterly, 19, 278303.
chology (pp. 161238). Mahwah, NJ: Erlbaum. Stipek, D., & Gralinski, J. H. (1996). Children's beliefs about intelligence and school perfor-
Cunningham, A. E., Stanovich, K. E., & Wilson, M. R. (1990). Cognitive variation in adult mance. Journal of Educational Psychology, 88, 397407.
college students differing in reading ability. In T. H. Carr, & B.A. Levy (Eds.), Reading Strms, H. I., Brten, I., & Samuelstuen, M. S. (2008). Dimensions of topic-specic episte-
and its development: Component skills approaches (pp. 129159). San Diego, CA: Aca- mological beliefs as predictors of multiple text understanding. Learning and
demic Press. Instruction, 18, 513527.
20 J.L.G. Braasch et al. / Learning and Individual Differences 31 (2014) 1120

Strms, H. I., Hagtvet, B. E., Lyster, S. A. H., & Rygvold, A. L. (1997). Lese- og Wiley, J., Goldman, S. R., Graesser, A.C., Sanchez, C. A., Ash, I. K., & Hemmerich, J. A. (2009).
skriveprver for studenter p hyskole-og universitetsniv. [Reading and spelling Source evaluation, comprehension, and learning in Internet science inquiry tasks.
tests for students in higher education.]. University of Oslo: Department of Special American Educational Research Journal, 46, 10601106.
Needs Education. Wiley, J., Grifn, T. D., & Thiede, K. W. (2005). Putting the comprehension in
Swanson, H. L., & Trahan, M. F. (1992). Learning disabled readers' comprehension of com- metacomprehension. The Journal of General Psychology, 132, 408428.
puter mediated text: The inuence of working memory, metacognition, and attribu- Wineburg, S. S. (1991). Historical problem solving: A study of the cognitive processes
tion. Learning Disabilities Research & Practice, 7, 7486. used in the evaluation of documentary and pictorial evidence. Journal of Educational
Undheim, J. O., Nordvik, H., Gustafsson, K., & Undheim, A.M. (1995). Academic achieve- Psychology, 83, 7387.
ments of high-ability students in egalitarian education: A study of able 16-year-old Wolfe, M. B. W., & Goldman, S. R. (2005). Relations between adolescents' text processing
students in Norway. Scandinavian Journal of Educational Research, 39, 157167. and reasoning. Cognition and Instruction, 23, 467502.