Professional Documents
Culture Documents
New Rule Use Drives the Relation Between Working Memory Capacity
and Raven’s Advanced Progressive Matrices
The correlation between individual differences in working memory capacity and performance on the
Raven’s Advanced Progressive Matrices (RAPM) is well documented yet poorly understood. The present
work proposes a new explanation: that the need to use a new combination of rules on RAPM problems
drives the relation between performance and working memory capacity scores. Evidence for this account
This article is intended solely for the personal use of the individual user and is not to be disseminated broadly.
is supported by an item-based analysis of performance during standard administration of the RAPM and
This document is copyrighted by the American Psychological Association or one of its allied publishers.
an experiment that manipulates the need to use new rule combinations across 2 subsets of RAPM items.
Keywords: problem solving, working memory capacity, executive function, attentional control
The need to hold multiple rules, subgoals, or the results of One task frequently investigated in conjunction with individual
previous solution attempts in memory is thought to be a major differences in WMC is Raven’s Advanced Progressive Matrices
determinant of problem-solving difficulty (Baddeley & Hitch, (RAPM; Raven, Raven, & Court, 1998). This visuospatial reason-
1974; Cary & Carlson, 2001; Kotovsky, Hayes, & Simon, 1985; ing and problem-solving task, developed as a psychometric assess-
Sweller, 1988). As working memory capacity (WMC) is thought to ment of intellectual ability (especially Gf), requires participants to
relate to the number of distinct representations that can be main- derive a set of rules or relations between stimuli in order to
tained, manipulated, or integrated in active memory, models of complete a visual pattern. Performance on this test has been
problem solving usually include a parameter that represents a suggested to be dependent on executive control processes that
limited capacity buffer in order to accurately fit human perfor- allow a solver to analyze complex problems, assemble solution
mance (Anderson & Lebiere, 1998; Carpenter, Just, & Shell, 1990; strategies, monitor performance, and adapt as performance pro-
Newell & Simon, 1972). However, surprisingly few studies have ceeds (Marshalek, Lohman, & Snow, 1983).
used individual differences approaches to explore how variations Thus, performance on both complex span tasks and RAPM has
in WMC may actually affect the problem-solving process (Ash & been argued to rely on executive function or attentional control,
Wiley, 2006; Hambrick & Engle, 2003). and, across a number of studies in college populations, simple
The working memory system was originally conceptualized by correlations between scores on OSpan and RAPM assessments are
Baddeley and Hitch (1974) as a temporary storage and processing typically found to be around .30 (r ⫽ .20, Conway, Cowan,
space for information that contains phonological and visuospatial Bunting, Therriault, & Minkoff, 2002; r ⫽ .32, Kane et al., 2004;
storage buffers, as well as a more domain-general central executive r ⫽ .34, Unsworth & Engle, 2005). What drives the relationship
charged with regulating the flow of information in and out of those between these tasks is not yet clear. As such, our goal in the
buffers. In order to involve both the buffers and the central current paper was to obtain a finer grained understanding of the
executive, assessments of individual differences in WMC involve association between problem-solving ability on the RAPM and
a storage component as well as a concurrent processing compo- individual differences in WMC.
nent. For example, in the operation span (OSpan) task (Conway et In an earlier analysis of the RAPM problem set, Carpenter et al.
al., 2005), participants need to remember a list of words while they (1990) suggested the relation with WMC might exist because
evaluate the correctness of simple math equations. Performance on RAPM problems vary in the number of rules and instances of those
complex span tasks, such as OSpan, is thought to be driven by the rules (tokens) that must be considered to solve the problems. More
ability to maintain information and control attention in the face of difficult problems were seen to require more rules or rule tokens to
distraction (Conway et al., 2005). be solved. Consistent with this explanation, they generated two
computational models of performance, one with a limited WMC
and one with a larger WMC. Carpenter et al. demonstrated that a
Jennifer Wiley, Andrew F. Jarosz, Patrick J. Cushen, and Gregory J. H. more limited model provided a good fit for average performance
Colflesh, Department of Psychology, University of Illinois at Chicago. but a model with a larger WMC provided a better fit for superior
This research was supported by National Science Foundation CAREER performance.
Grant REC 0347887 to Jennifer Wiley. We are indebted to Andy Conway,
Logically, this rule/capacity account suggests that individuals
Thomas Griffin, Melinda Jensen, Tim Miura, Travis Ricks, and Chris
Sanchez for their assistance in collecting the data and discussions about
with low WMC should do more poorly on RAPM problems as the
this line of research. number of rules that are required to solve a problem increases.
Correspondence concerning this article should be addressed to Jennifer However, subsequent studies have failed to support this prediction.
Wiley, Department of Psychology, University of Illinois at Chicago, 1007 Salthouse (1993) examined how individual differences in WMC
West Harrison Street (M/C 285), Chicago, IL 60607. E-mail: jwiley@uic.edu related to solution success on early versus later items. The Raven
256
RESEARCH REPORTS 257
problem sets are traditionally arranged in order of increasing identify new rule combinations as one works through the problem
normative difficulty. Salthouse reasoned that if WMC helped set could potentially underlie the relation between RAPM and
solvers to keep multiple rules, subgoals, or intermediate products WMC. This view makes the opposite prediction from the learning
in memory, WMC should have a larger effect on later problems in efficiency account. In particular, it suggests that the relation be-
the set. However, he found the relation between WMC and solu- tween WMC and RAPM performance should be highest on prob-
tion success was constant across problem order. lems that require the first use of a rule combination.
Unsworth and Engle (2005) also found that the relation between
RAPM and WMC did not increase as the normative difficulty of
problems increased. In addition, they tested the rule/capacity ac- Study 1
count more directly. Using the Carpenter et al. (1990) rule tokens,
In the first study, we explored these alternative hypotheses by
Unsworth and Engle tested the prediction that WMC should be
creating a new coding scheme for the standard RAPM Set II. In
most strongly related to performance on RAPM problems requir-
this analysis, as opposed to Carpenter et al. (1990), which omitted
ing the most rule tokens. This prediction was not supported. The
nine of 36 items in its coding, all items were coded for the rules
relation between WMC and RAPM performance was similar across
This article is intended solely for the personal use of the individual user and is not to be disseminated broadly.
Table 2
Correlations of RAPM Subsets With OSpan (N ⫽ 255) in
1 2 3 4 Study 1
Variable 0 1 2 3 4
ⴱⴱ ⴱⴱ ⴱⴱ
Quartiles .27 .27 .24 .21ⴱⴱ
All rule tokens
(with constant) .25ⴱⴱ .24ⴱⴱ .33ⴱⴱ .21ⴱⴱ
5 6 7 8 Rule tokens (no
constant) .14ⴱⴱ .28ⴱⴱ .29ⴱⴱ .20ⴱⴱ
Rule combination
repetitions .39ⴱⴱ .21ⴱⴱ .16ⴱⴱ .18ⴱⴱ .20ⴱⴱ
Table 3 Table 4
Simultaneous Regressions Predicting OSpan for Study 1 Correlations Among WMC–RAPM Point-Biserial Correlations
(Correlation) and Predictor Variables (Solution Rate, Problem
Model and predictor  t sr2 R2 F Order, Rule Tokens Without Constant Rules, Switch, First Use)
Model 1 for Each Problem (N ⫽ 36) in Study 1
First use .42 3.60ⴱⴱ 0.22
Overall RAPM ⫺.04 ⫺0.34 ⫺0.02 .15 22.2ⴱⴱ Variable 1 2 3 4 5
Model 2
Repeated rule use ⫺.73 ⫺3.56ⴱⴱ ⫺0.22 1. Solve rate —
Overall RAPM 1.03 5.00ⴱⴱ 0.30 .15 22.1ⴱⴱ 2. Order ⫺.96ⴱⴱ —
Model 3 3. Rule tokens ⫺.65ⴱⴱ .66ⴱⴱ —
First use .46 3.56ⴱⴱ 0.22 4. Switch .15 ⫺.09 ⫺.15 —
Switch ⫺.08 ⫺0.61 ⫺0.04 .15 22.4ⴱⴱ 5. First use ⫺.13 .15 .11 .36ⴱ —
Correlation .23 ⫺.26 ⫺.19 .16 .60ⴱⴱ
Note. OSpan ⫽ operating span; RAPM ⫽ Raven’s Advanced Progressive
Note. WMC ⫽ working memory capacity; RAPM ⫽ Raven’s Advanced
This article is intended solely for the personal use of the individual user and is not to be disseminated broadly.
Matrices.
This document is copyrighted by the American Psychological Association or one of its allied publishers.
ⴱⴱ
p ⬍ .01. Progressive Matrices.
ⴱ
p ⬍ .05. ⴱⴱ p ⬍ .01.
Figure 2. Point-biserial correlations with working memory capacity for performance on individual Raven’s
Advanced Progressive Matrices problems in Study 1. (Values for first-use problems are circled.)
260 RESEARCH REPORTS
Table 5 Method
Simultaneous Regression Predicting WMC–RAPM Point-Biserial
Correlations for Study 1 An independent sample of 50 undergraduates at the same uni-
versity participated as part of the introductory psychology subject
Variable  t sr2 R2 F pool. To ensure reliable measurement of WMC, we administered
multiple span tasks following Conway et al. (2005). Participants
Order ⫺.34 ⫺1.97† ⫺.33
completed both OSpan and Reading Span (RSpan), and we created
Rule tokens .04 0.25 .04
Switch .06 0.42 .08 a composite measure by averaging across the tasks. The scores on
First use .64 4.61ⴱⴱ .64 .50 7.59ⴱⴱ the two tasks were correlated at .69, which is a high level of
agreement (correlations among span tasks typically range from .4
Note. WMC ⫽ working memory capacity; RAPM ⫽ Raven’s Advanced to .6; Conway et al., 2005). For the RAPM, all participants
Progressive Matrices.
†
p ⬍ .10. ⴱⴱ p ⬍ .01. completed two practice items (Items 1 and 2 from RAPM Set I)
with no time limit and then 16 items from Set II with a 20-min
limit. In the repeated rules condition, items were presented in
This article is intended solely for the personal use of the individual user and is not to be disseminated broadly.
blocks based on rule, with all items of the same rule combination
This document is copyrighted by the American Psychological Association or one of its allied publishers.
only the first-use variable significantly predicted WMC,  ⫽ .38, being presented in series from the easiest to the most difficult.
t(26) ⫽ 2.05, p ⬍ .05.Thus, the results of all analyses are consis- Following the final (most difficult) item of that rule combination,
tent with the interference/distraction account, as the relation be- the easiest item from the next rule combination was presented.
tween WMC and RAPM is being driven by performance on Items in the novel rules condition were presented in an order that
problems that require the first use of a rule combination. matched the difficulty of items in the repeated rules condition on
a problem-by-problem basis. The items used for the novel rules
Study 2 condition, in order, were 4, 35, 7, 17, 12, 13, 10, 20, 25, 18, 14, 21,
30, 33, 29, and 32. The items used for the repeated rules condition,
Based on the observation that problems with novel rule combi- in order, were 5, 24, 7, 9, 11, 19, 16, 22, 23, 33, 8, 26, 27, 28, 29,
nations are critical for the WMC and RAPM correlation, the goal and 34. Odd– even reliability estimates computed using Cron-
in our second study was to experimentally manipulate the presence bach’s alpha were similar for the two subsets (novel ⫽ .67,
of novel rule combinations and examine the resultant effect on the repeated ⫽ .74).
WMC–RAPM relationship. To do this, we selected one subset of
16 RAPM items such that each problem required a unique rule Results
combination (referred to as the novel rules condition). Then,
we selected a second set of 16 RAPM items that required only five Descriptive statistics for RAPM and WMC performance in both
different rule combinations (the repeated rules condition). The conditions are shown in Table 1. No differences were found
normative difficulty of these sets was matched. If the explanation between conditions in either RAPM performance or WMC (ts ⬍
supported by our item analyses in Study 1 is correct, performance 1). However, as shown in Figure 3, the relation between RAPM
in the novel rules condition should be more related than perfor- and WMC differed as a function of condition. A regression pre-
mance in the repeated rules condition to WMC. dicting RAPM performance using WMC and condition in the first
Figure 3. Relation of working memory capacity to Raven’s Advanced Progressive Matrices (RAPM) performance
in novel rule and repeated rule conditions in Study 2.
RESEARCH REPORTS 261
step demonstrated a significant improvement in model fit with the problems. Thus, differences in the quality of executive function,
addition of the dummy-coded interaction in the second step, R ⫽ and not capacity per se, may be responsible for the relationship
.45, R2 change ⫽ .08, F(1, 46) ⫽ 4.71, MSE ⫽ .01, p ⬍ .04. The between WMC and RAPM.
interaction term significantly predicted RAPM performance,  ⫽ We note that there is not yet direct evidence for the interference/
.873, t(46) ⫽ 2.17, p ⬍ .04, sr2 ⫽ .31, indicating that performance distraction account and suggest that the examination of process mea-
in the novel rules condition was more strongly related to WMC sures collected during performance of both tasks is an important next
(r ⫽ .62, p ⬍ .001) than performance in the repeated rules step in order to further test this explanation. Although there is much
condition (r ⫽ .02, ns). left to explore, the present results do support the conclusion that
individual differences in WMC are related to an ability to identify and
General Discussion apply new relations when engaged in problem solving and provide a
much-needed explanation for the association between performance on
The results of both studies offer a new insight into the relation WMC measures and performance on the RAPM.
between individual differences in WMC and problem solving,
particularly on the RAPM. In the first study, it was performance on References
This article is intended solely for the personal use of the individual user and is not to be disseminated broadly.
This document is copyrighted by the American Psychological Association or one of its allied publishers.
problems in which a combination of rules was used for the first Anderson, J. R., & Lebiere, C. (1998). Atomic components of thought.
time that was critical for the relationship between WMC and Mahwah, NJ: Erlbaum.
RAPM. Confidence in the generalizability of this result is garnered Ash, I. K., & Wiley, J. (2006). The nature of restructuring in insight: An
by its replication in the data from Unsworth and Engle (2005). The individual differences approach. Psychonomic Bulletin & Review, 13,
second study further supports this finding by manipulating the 66 –73.
novelty of rule combinations needed to solve problems and dem- Baddeley, A. D., & Hitch, G. (1974). Working memory. Psychology of
onstrating that WMC was critical to performance when solving Learning and Motivation, 8, 47– 89. doi:10.1016/S0079-7421(08)60452-1
required the frequent use of new rules. Bunting, M. (2006). Proactive interference and item similarity in working
Our results converge with previous findings in terms of the memory. Journal of Experimental Psychology: Learning, Memory, and
Cognition, 32, 183–196. doi:10.1037/0278-7393.32.2.183
correlations found between WMC and RAPM. They are consistent
Carpenter, P. A., Just, M., & Shell, P. (1990). What one intelligence test
with previous findings in that neither the normative difficulty of measures: A theoretical account of the processing in the Raven Progres-
RAPM items nor the number of rule tokens required for solution sive Matrices Test. Psychological Review, 97, 404 – 431. doi:10.1037/
showed the positive relation with WMC that would be predicted by 0033-295X.97.3.404
a rule/capacity explanation. Additionally, the relationship between Cary, M., & Carlson, R. A. (2001). Distributing working memory resources
WMC and RAPM performance did not increase as a function of in problem solving. Journal of Experimental Psychology: Learning,
repeated rule use, arguing against the learning efficiency account. Memory, and Cognition, 27, 836 – 848. doi:10.1037/0278-7393.27.3.836
Note that these results should not be taken to suggest that the Colflesh, G. J. H., & Conway, A. R. A. (2007). Individual differences in
number of rules required to solve problems, or other aspects of working memory capacity and divided attention in dichotic listening.
item complexity, will not affect problem-solving performance. Psychonomic Bulletin & Review, 14, 699 –703.
Conway, A. R. A., Cowan, N., Bunting, M. F., Therriault, D. J., & Minkoff,
These factors clearly can relate to problem difficulty, and our data
S. R. B. (2002). A latent variable analysis of working memory capacity,
do, indeed, demonstrate a significant relationship between the short-term memory capacity, processing speed, and general fluid intel-
number of rule tokens required to solve a problem and perfor- ligence. Intelligence, 30, 163–184. doi:10.1016/S0160-2896(01)00096-4
mance on the RAPM items. However, these factors do not seem to Conway, A. R. A., Kane, M. J., Bunting, M. F., Hambrick, D. Z., Wilhelm, O.,
be what drives the relation with WMC. & Engle, R. W. (2005). Working memory span tasks: A methodological
The breakthrough finding here, that performance on first-use review and user’s guide. Psychonomic Bulletin & Review, 12, 769 –786.
problems underlies the WMC–RAPM relation, can be seen as Darowski, E. S., Helder, E., Zacks, R. T., Hasher, L., & Hambrick, D. Z.
consistent with recent conceptions of individual differences in (2008). Age-related differences in cognition: The role of distraction
WMC that emphasize the role of executive functioning, including control. Neuropsychology, 22, 638 – 644. doi:10.1037/0894-
the ability to flexibly control one’s focus of attention (Colflesh & 4105.22.5.638
Engle, R. W. (2002). Working memory capacity as executive attention.
Conway, 2007; Engle, 2002; Kane, Bleckley, Conway, & Engle,
Current Directions in Psychological Science, 11, 19 –23. doi:10.1111/
2001) and susceptibility to distraction or interference during cog- 1467-8721.00160
nitive tasks (Bunting, 2006; Hasher & Zacks, 1988; Unsworth & Guthke, J. J., & Stein, H. (1996). Are learning tests the better version of
Engle, 2007). The present results may be explained by the notion intelligence tests? European Journal of Psychological Assessment, 12,
that previously learned and used rules may interfere with perfor- 1–13. doi:10.1027/1015-5759.12.1.1
mance on problems when a new rule combination is needed. High Hambrick, D. Z., & Engle, R. W. (2003). The role of working memory in
WMC participants may do better on the RAPM because they are problem solving. In J. E. Davidson & R. J. Sternberg (Eds.), The
better able to ignore or reject known rules when they do not lead psychology of problem solving (pp. 176 –206). Cambridge, England:
to a solution on new-rule-combination problems. On the other Cambridge University Press. doi:10.1017/CBO9780511615771.007
hand, low WMC participants may be less successful at removing Hasher, L., & Zacks, R. T. (1988). Working memory, comprehension, and
aging: A review and a new view. In G. H. Bower (Ed.), The psychology
their attention from previously learned rules, resulting in deficien-
of learning and motivation: Advances in research and theory (Vol. 22,
cies in finding new rules, or perseverating in attempts to retrieve pp. 193–225). San Diego, CA: Academic Press.
and apply ineffective rules. A similar finding was obtained by Ash Kane, M. J., Bleckley, M. K., Conway, A. R., & Engle, R. W. (2001). A
and Wiley (2006), who found that low WMC participants were less controlled-attention view of working-memory capacity. Journal of Experi-
likely than high WMC participants to recognize that their first mental Psychology: General, 130, 169 –183. doi:10.1037/0096-
solution attempts were inappropriate when solving a set of insight 3445.130.2.169
262 RESEARCH REPORTS
Kane, M. J., Hambrick, D. Z., Tuholski, S. W., Wilhelm, O., Payne, T. W., Progressive Matrices and Vocabulary Scales: Section 4. The Advanced
& Engle, R. W. (2004). The generality of working memory capacity: A Progressive Matrices. San Antonio, TX: Harcourt Assessment.
latent-variable approach to verbal and visuospatial memory span and Salthouse, T. A. (1993). Influence of working memory on adult age
reasoning. Journal of Experimental Psychology: General, 133, 189 – differences in matrix reasoning. British Journal of Psychology, 84,
217. doi:10.1037/0096-3445.133.2.189 171–199.
Kotovsky, K., Hayes, J. R., & Simon, H. A. (1985). Why are some Sweller, J. (1988). Cognitive load during problem solving: Effects
problems hard? Evidence from Tower of Hanoi. Cognitive Psychology, on learning. Cognitive Science, 12, 257–285. doi:10.1207/
17, 248 –294. doi:10.1016/0010-0285(85)90009-X s15516709cog1202_4
Marshalek, B., Lohman, D. F., & Snow, R. E. (1983). The complexity Unsworth, N., & Engle, R. W. (2005). Working memory capacity and fluid
continuum in the radex and hierarchical models of intelligence. Intelli- abilities: Examining the correlation between Operation Span and Raven.
gence, 7, 107–127. doi:10.1016/0160-2896(83)90023-5 Intelligence, 33, 67– 81. doi:10.1016/j.intell.2004.08.003
Miyake, A., Friedman, N. P., Emerson, M. J., Witzki, A. H., Howerter, A., & Unsworth, N., & Engle, R. W. (2007). The nature of individual differences
Wager, T. (2000). The unity and diversity of executive functions and their in working memory capacity: Active maintenance in primary memory
contributions to complex “frontal lobe” tasks: A latent variable analysis. and controlled search from secondary memory. Psychological Review,
Cognitive Psychology, 41, 49 –100. doi:10.1006/cogp.1999.0734 114, 104 –132. doi:10.1037/0033-295X.114.1.104
This article is intended solely for the personal use of the individual user and is not to be disseminated broadly.
Newell, A., & Simon, H. A. (1972). Human problem solving. Englewood Verguts, T., & De Boeck, P. (2002). The induction of solution rules in
This document is copyrighted by the American Psychological Association or one of its allied publishers.
Cliffs, NJ: Prentice Hall. Raven’s Progressive Matrices Test. European Journal of Cognitive
Raven, J., Raven, J. C., & Court, J. H. (1998). Manual for Raven’s Psychology, 14, 521–547. doi:10.1080/09541440143000230
Appendix
Rule Coding Details Distribution of 3. Three discrete values appear in each row or
column such as the shapes in example below. Code three orienta-
Rules Identified With Examples tions of objects as Dist 3.
(Appendix continues)
RESEARCH REPORTS 263
Rule combinations
RAPM II All tokens Rule count (ⴱ First-Use) (I - Set I)