Professional Documents
Culture Documents
Please address correspondence to: Alex Kozulin, The International Center for
the Enhancement of Learning Potential, 47 Narkis Street, PO Box 7755,
Jerusalem 91077, Israel. Email: AKozulin@Compuserve.com
112
113
School Psychology International (2002), Vol. 23(1)
first fully operationalized versions of dynamic cognitive assessment
were developed by Budoff and Friedman (1964) and Feuerstein and
Shalom (1968). The dynamic assessment tasks themselves were similar
to those used in standard psychometric tests but the assessment proce-
dure was radically changed to include a learning phase. The whole
philosophy of assessment has also been changed. For example, the
Feuerstein et al. (1979) version of dynamic assessment was based on the
following theoretical assumptions:
(1) The students' cognitive processes are highly modifiable. The task of
assessment is to ascertain the degree of modifiability rather than
the manifest level of functioning;
(2) The reduced modifiability is the result of insufficient type or
amount of mediated learning experience (MLE) received by a
student;
(3) Dynamic assessment, which includes a mediated learning phase,
provides better insight into students' learning capacity than
unaided performance;
(4) The evaluator plays an active role by mediating cognitive strategies
during the learning phase;
(5) The goal of dynamic assessment is to reveal the students' learning
potential and to formulate optimal educational intervention (see
Kozulin and Falik, 1995).
Two aspects of Feuerstein's dynamic assessment are particularly
important for the present discussion. His selection of assessment tasks
was based on the assumption that while students' performance on some
tasks depends on more conservative functions such as prior knowledge,
memory or highly automatized skills, performance on other tasks is
more flexible and modifiable, and depends more on students' cognitive
strategies. For the purpose of dynamic assessment one should therefore
select only the latter group of tasks, i.e. only those problems whose
solution depends on the use of cognitive strategies.
The role of the evaluator, according to Feuerstein, is to identify the
students' problems during the pre-test and to provide the necessary
mediation during the learning phase. The concept ofMLE (Feuerstein,
1990) specifically addresses such issues as the evaluator's sensitivity to
students' questions and responses, the transfer of principles beyond a
here-and-now given task and mediation of the meaning of the assess-
:.I>""~"-l:i!s,ituation.
Only if all these criteria are met may the situation
quality as an MLE-based dynamic assessment.
s well as MLE-based assessments, a whole group of dynamic cogni-
tive assessment techniques is currently available. What unites all those
approaches is their reliance on the test-teach-test paradigm (Campione
1996; Haywood and Tzuriel, 1992). At the same time the goals of
114
;~.
115
School Psychology International (2002), Vol. 23(1)
For example, new approaches in science teaching present the learning
process as leading to a conceptual change (Driver et al., 1985). Students'
intuitive concepts of natural phenomena undergo modification and
change under the influence of their learning experience. Thus content
learning is conceived as a process of change in the students' thinking
about nature. To produce such a change, it is not enough to simply
present scientific concepts to the students. Students should be taught
how to think scientifically. At the same time, some authors (Adey, 1999)
claim that if students are given proper 'thinking science' activities, the
outcomes will be positive not only in science knowledge but also in more
general thinking abilities.
A somewhat similar process can be observed in mathematical instruc-
tion. The process of maths teaching is no longer seen as a mere
transmission of operations, rules and standard algorithms for solving
standard problems, but rather as a constructive, interactive and reflec-
tive practice of developing and applying maths concepts (Nelissen,
1999).
In reading instruction, researchers have been coming to similar
conclusions. That reading is not a passive, but rather an active, and in
fact an interactive process, has been recognized for some time (Goodman,
1967; Rumelhart, 1977; Smith, 1971). Current theories regard reading
as an active process in which the reader constructs meaning from text
cues, calling upon knowledge of language, text structure, conventions,
content concepts, etc. This knowledge is organized in the reader's mind
in schemas, which are activated when a good reader approaches a text
(Carrel, 1984).
Activating or constructing a schema includes arriving at a coherent
explanation of the relations between elements in a discourse, and thus
knowledge of text structure is a key feature. The distinction between a
text and unrelated sentences is one of cohesion, (Halliday and Hasan,
1976) including elements such as reference, substitution, ellipsis, con-
junction and lexical cohesion. Other textural features which influence
comprehension include awareness of the arrangement of ideas in texts,
i.e. the rhetorical and logical organization, vocabulary, syntax, clarity of
author's intentions and readers' familiarity with the content. Salient
findings from the research indicate that readers can optimize their
comprehension by becoming aware of the text structures and the
resultant effect they have on learning. By detecting the organizational
patterns of texts, students can observe how authors arrange ideas and
which kind of structures are used to interrelate ideas.
A significant implication for curriculum development that the re-
search suggests is that knowledge precedes control, and thatmetacognition
ofthe above factors can therefore optimize their influence (Armbruster et
al., 1983). Thus reading involves both knowledge - of strategies for
116
-------------- ~.----.
-~--~
(a) Teachers must devote time to the processes involved in reading and
learning. Brown (1981) is only one of many analysts to have noted
that product rather than process is stressed in most traditional
classrooms. In order to stress processes, the covert cognitive and
metacognitive processes must first be rendered into overt form, i.e.
suitable materials must be devised;
(b) Teachers must do task analyses of strategies to be taught. Garner
suggests that strategic problem-solving activities be examined and
strategies broken down into global steps;
(c) Teachers must demonstrate a variety of situations in which learn-
ers might profitably use the strategies taught and transfer to these
situations must be explicitly taught.
117
School Psychology International (2002), Vol. 23(1)
training outperformed other groups when asked to read selections on
their own, without the teachers' support, pointing to the transfer value
of teaching students to become independent learners. This study, like
earlier ones, (Palinscar and Brown, 1984; Wong, 1985) shows that lower
achievers benefit particularly from learning specific strategies. In addi-
tion, the research indicated that strategies could compensate for lack of
background knowledge. Although the experimental group was not given
advance background knowledge provided to other groups, this did not
place them at a serious disadvantage.
"
118
Kozulin & Garb: Dynamic Assessment of EFL Text Comprehension
therefore assess the student's ability to learn, activate and use effective
strategies for text comprehension.
119
School Psychology International (2002), Vol. 23(J)
Stage 3. Re-test. A second test was designed, where each item
matched that of the pre-test with regard to information, strategies,
length and level of difficulty. Both tests were piloted without mediation
at one of the pre-academic centres. Student scores on all items were
almost identical, indicating that the second test was equivalent to the
first.
Procedure
The entire pre-test was given to the students. The maximum time
allowed was 90 minutes. Dictionaries were not allowed, so that students
would need to rely for word meaning on strategies such as prediction
and hypothesis. Mediation and re-testing were done in two stages, since
the pilot experiment indicated that this population of students was
unable to absorb mediation ofthe entire test at one sitting. Part 1 was
mediated and re-tested a few days later. Part 2 was then mediated, and
re-tested a few days later. (Appendix A will illustrate the procedure by
giving one example of the pre-test and the mediation instructions to
teachers).
Sample
We report here results obtained in one ofthe pre-academic centres in the
rural area ofthe country. Assessment was conducted with a group of 23
students, 4 males and 19 females, aged 20-23 years. Three of the
students were relatively new immigrants for whom English was their
third language. All students were considered the weaker group of
academically at-risk students, having failed to obtain a matriculation
certificate. In addition to studying EFL, these students were concomi-
tantly taking up to a total of seven other subjects.
Results
The comparison of pre- and post-test performance of the students can be
analysed in a number of ways. First of all one can see that on average
students' scores improved by more than one standard deviation (see
Table 1). The effect size is 1.2 (Effect size = Gain score/SD ofthe post-
test). This result indicates that many of the students indeed benefited
from mediation and were able to apply the acquired strategies to the new
text. Secondly, it became clear that while the pre- and post-rest scores
are highly correlated (r = 0.8), the gain scores are negatively correlated
with the pre-test scores. This means that the pre-test scores reflect the
students' actual performance level but not their learning potential.
120
Kozulin & Garb: Dynamic Assessment of EFL Text Comprehension
Table 1 Mean scores and standard deviations in the pre- and post-
=
test of EFL text comprehension. Max score 34; N 23 =
Mean SD
where 8 pre and S post are pre- and post-test scores, and Max 8 is a
maximum obtainable score. The above formula provides a theoretical
basis for distinguishing between high learning potential and low learn-
ing potential students. For example, a student who at a pre-test had a
low score = 50, but made a significant progress and reached the maxi-
mum score of 100 at the post-test would have a very high LPS = 1.5;
whereas a student who at a pre-test had a low score = 50 and made no
=
progress receiving at the post-test the same score 50, would have a very
low LP8 = 0.5. All other cases characterized by different combinations of
gain scores and absolute post- test score can be plotted against these
extremes.
The LP8 of the students from our sample ranged from 0.4 7 to 1.21. One
can distinguish three sub-groups: a high learning potential sub-group
with LPS>1.0, a low learning potential sub-group with LPS<O.71 and a
large sub-group with LPS ranging from 0.79 to 0.88 (see Figure 1). It is
important to emphasize that all three groups include some students who
performed at the pre-test at the same level, which means that the
121
School Psychology International (2002), Vol. 23(1)
Learning Potential Score
1.4
1.2
1
(f)
~ 0.8
o
eno 0.6
0.4
0.2
o
Students
Figure 1 Distribution of learning potential scores in a group of
students.
Discussion
The goal of this article was to explore the feasibility ofthe development
and implementation of the dynamic EFL assessment procedure in the
pre-academic classroom. The results indicate that the procedure is both
feasible and effective in obtaining information on students' learning
potential. It was confirmed that students with similar performance
levels demonstrated different, and in some cases dramatically different
abilities to learn and use new text comprehension strategies. Because of
this we can affirm that the paradigm of dynamic assessment is useful not
only in the field of cognitive performance but also in such curricular
domains as EFL learning.
At the same time one should be aware of those characteristic features
ofthe dynamic assessment procedure that impose certain limitations on
the generalizability ofthe results. Any dynamic assessment that includes
an element of intervention depends on the quality of mediation provided
by the assessor. In this respect dynamic assessment is closer to a
situation of instruction rather than examination. Results obtained in the
present study thus reflect not only the students' learning potential but
also the quality of mediation provided during the assessment. One may
always suspect that another assessor with a different mediational style
might reveal a somewhat different pattern of learning abilities in the
same group of students. Thus one of the possible directions for the future
122
Kozulin & Garb: Dynamic Assessment of EFL Text Comprehension
research is a study of reliability of learning potential scores obtained
during assessment sessions conducted by different mediators.
The second limitation inherent in any curriculum-based assessment
is its dependence on students' content knowledge. There is no such thing
as a content-free language task which means that students with better
vocabulary and better knowledge of grammatical rules would always
have a certain advantage. Though our selection of assessment materials
was guided by the ideal of strategies-based text comprehension, it was
impossible to eliminate the element of content knowledge as such. In
other words, students who have a very poor knowledge base cannot
expect to reach a high achievement level even if their use of strategies is
quite good.
The instructional value of the dynamic EFL assessment lies in the fact
that its results can be used for the development of individual learning
plans for students with different learning needs. For example, work with
students who demonstrated an average pre-test performance but insuf-
ficient learning potential should focus on providing them with learning
and information-processing strategies, i.e. teaching them 'how to learn'.
Students with an average pre-test performance and high learning
potential should be given more challenging material and more opportun-
ity for independent study. Students with low pre-test performance and
low learning potential need an intensive investment into their general
learning and problem-solving skills that should be based on very simple
EFL material. Only after these students acquire the basic learning skills
should they be challenged by the standard EFL tasks.
References
Adey, P. (1999) 'Thinking Science: Science as a Gateaway to General Thinking
Ability', in J. Hamers, J. Van Luit and B. Csapo (eds) Teaching and Learning
Thinking Skills, pp. 63-80. Lisse: Swets & Zeitlinger.
Armbruster, B.B. et al. (1983) The Role of Meta cognition in Reading to Learn: A
Developmental Perspective. Urbana, IL: Center for the Study of Reading.
Baker, L.B. and Brown, A.L. (1984a) 'Cognitive Monitoring in Reading', in
J. Flood (ed.) Understanding Reading Comprehension: Cognition, Language,
and the Structure of Prose, pp. 21-44. Newark, DE: International Reading
Association.
Baker, L.B. and Brown, A.L. (1984b) 'Metacognitive Skills and Reading', in P.D.
Pearson (ed.) Handbook of Reading Research (Vol. 353-394). New York:
Longman.
Brown, A.L. (1981) 'Metacognition: The Development of Selective Attention
Strategies for Learning from Texts', paper presented at the National Reading
Conference, Washington, DC.
Brown, A.L., Armbruster and Baker, Linda (1986) 'The Role of Meta cognition in
Reading and Studying', in J. Orasanu (ed.) Reading Comprehension: From
Research to Practice, pp. 49-76. Hillsdale, NJ and London: Lawrence Erlbaum.
Brown, A.L., Bransford, J.D., Ferrara, R.A. and Campione, J. C. (1983) 'Learning,
123
School Psychology International (2002), Vol. 23(1)
Remembering and Understanding', in W. Kessen (ed.) Handbook of Child
Psychology: Cognitive Development 3, pp. 77-166. New York: Wiley Collins.
Brown, A.L., Campione, J.C. and Day, J.D. (1981) 'Learning to Learn: On
Training Students to Learn from Text', Educational Researcher 10: 14-21.
Budoff, M. and Friedman, M. (1964) '"Learning Potential" as an Assessment
Approach to the Adolescent Mentally Retarded', Journal of Consulting
Psychology 28: 34-439.
Burden, R. and Williams, M. (eds) (1998) Thinking Through the Curriculum.
London: Routledge.
Campione, J. (1996) 'Assisted Assessment: A Taxonomy of Approaches and an
Outline of Strengths and Weaknesses', in H. Daniels (ed.) An Introduction to
Vygotsky, pp. 219-50. London: Routledge.
Carrell, P.L. (1984) 'Schema Theory and ESL Reading: Classroom Implications
and Applications', The Modern Language Journal 68(iv): 332-43.
Carrell, P.L. (1988) 'Interactive Text Processing: Implications for ESUSecond
Language Reading Classrooms', in L. Carrell, J. Devine and D.E. Eskey (eds)
Interactive Approaches to Second Language Reading, pp. 239-59. Cambridge:
Cambridge University Press.
Carrell, P.L. (1989) 'Metacognitive Awareness and Second Language Reading',
The Modern Language Journal 73(ii): 121-31.
Cioffi, G. and Carney, J. (1983) 'Dynamic Assessment of Reading Disabilities,
The Reading Teacher 36: 764-68.
Clay, M. (1993) An Observation Survey of Early Literacy Achievement. Port-
smouth, NH: Heinemann.
Clarke, M. and Silberstein, S. (1977) 'Toward a Realization of Pycholinguistic
Principles for the ESL Reading Class', Language Learning 2: 135-54.
Coady, J. (1979) 'A Psycholinguistic Model of the ESL Reader', in R. Mackay, B.
Barkman and RR Jordan (eds) Reading in a Second Language, pp. 5-12.
Rowley, MA: Newbury House.
Dole, J., Brown, K. and Trathen, W. (1996) 'The Effects of Strategy Instruction
on the Comprehension Performance of At-Risk Students', Reading Research
Quarterly, 31(1): 62-88.
Driver, R., Guesne, E. and Tieberghien, A. (1985) Children's Ideas in Science.
Milton Keynes: Open University Press.
Feuerstein, R. (1990) 'The Theory of Structural Cognitive Modifiability', in B.
Presseisen (ed.)Learning and Thinking Styles: Classroom Interaction. Wash-
ington, DC: National Education Association.
Feuerstein, R. and Shalom, H. (1968) 'The Learning Potential Assessment
Device', in B.W. Richards (ed.) Proceedings of the First Congress of the
InternationalAssociation for the Scientific Study of Mental Deficiency. Reigate:
Michael Jackson.
Feuerstein, R., Rand, Y. and Hoffman, M. (1979) Dynamic Assessment of
Retarded Performer. Baltimore, MD: University Park Press.
Flavell, J.H. and Wellman, H.M. (1977) 'Metamemory', in J. Kail and J. W. Hagen
(eds) Perspectives on the Development of Memory and Cognition. Hillsdale,
NJ: Erlbaum.
Garb, E. and Kozulin, A. (1998) I Think . .. Therefore I Read: A Cognitive
Approach to English Teaching. Academon: Jerusalem.
Garb, E. and KozuIin, A. (1999) 'Dynamic Assessment of EFL at the Pre-
Academic Centres'. Unpublished report.
Garner, R (1987) Metacognition and Reading Comprehension. Norwood, NJ:
Ablex Publishing Corporation.
124
·.
125
..
•
Pre-test
Text:
'David and I were tourists in Europe last summer. We ate cheese in
Denmark and olives in Italy. The cheese and olives were very good. We
drank wine in France and tea in England. The tea from England and the
wine from France were excellent.'
Questions:
(1) The people in the story
(a) live in Europe;
(b) came to Europe on a visit;
(c) have never been to Europe;
(d) visit Europe every summer.
(2) How many places in Europe did they visit? What were they?
(3) They the English tea.
(a) didn't drink;
(b) ate;
(c) didn't like;
(d) liked.
Question 1.
Consider all alternatives one by one. First eliminate improbable [a) and
[c), then compare (hI and [d]. What is the difference between them? One
visit vs. visit every summer. Check the when - 'last summer'. There is no
information about visits every summer, therefore (h] is a correct answer.
126
!
127