SREE Fall 2011

The Effects of Feedback

During Exploratory Math
Practice
Emily R. Fyfe & Bethany Rittle-Johnson
Vanderbilt University
Marci S. DeCaro
University of Louisville
1

How do children learn best?

Two schools of thought have emerged
Advocates of Direct Instruction
(Kirschner, Sweller, & Clark, 2006)

Advocates of Discovery Learning

(Bruner, 1961; Kuhn, 1989; Piaget, 1973)

An Integrated Perspective
No need for this strict dichotomy
Proponents of
constructivism and direct
A mixture of
instructioneach have
guidance and
something to learn from
exploration is
the other (Rieber, 1992)
needed (Mayer,
Theres a place for
2004)
Characterizing
discovery
both direct
and direct instruction as
instruction and
diametrically opposedhas student-directed
done a disservice to both
inquiry (Kuhn,
approaches (Wilson,
2007)
et al, 2010)

Combining Instruction and

Discovery
Exploration prior to instruction facilitates learning
(DeCaro & Rittle-Johnson, 2011; Schwartz & Bransford, 1998)

Hints or coaching during problem solving is better

than pure problem solving alone
(Mayer, 2004)

Recent meta-analysis indicates that guided

discovery is better than unassisted discovery or
direct instruction
(Alfieri et al, 2010)
4

Questions remain
Which elements of instruction are most
suitable to incorporate within
exploration?
For whom is guided exploration most
advantageous and why?

Feedback During Exploration

Feedback represents one element of
instruction that may be particularly
effective in combination with exploration
What is Feedback?
Any information that the learner can use
to confirm, reject, or modify prior
knowledge
Accuracy information, hints, etc.
6

Types of Feedback
Outcome Feedback
Provides information
about learners answer

Strategy Feedback
Provides information
about how answer was
obtained

Used extensively

Only examined in a few

previous studies
Related to strong,
positive effects in past
work compared to no
feedback

Better than outcome

feedback in terms of
strategy selection

(Ahmad, 1988; Kluger & DeNisi, 1996; Luwel et al., 2011)

7

Why feedback?
May reduce disadvantages of discovery by
guiding the learners search for information
Helps identify errors and encourages search
for plausible alternatives (e.g., new
strategies)
(Hattie & Timperley, 2007; Mory, 2004)

But
Past research indicates variable efficacy
(Kluger & DeNisi, 1996)

May only benefit a subset of learners

8

What about prior knowledge?

Expertise reversal effect
Instructional technique is effective for novices,
but loses its benefits for high-knowledge
learners
(Kalyuga, Ayres, Chandler, & Sweller, 2003)

Example: worked examples vs. problem solving

Low knowledge learners benefit from more

external guidance; high knowledge learners
benefit from less
Perhaps feedback during exploration only helps
children who have low prior knowledge
9

Why the reversal?

Due to existing cognitive resources
(Paas, Renkl, & Sweller, 2003)

Novices lack schemas; need external

guidance to reduce cognitive load
High-knowledge learners have schemas;
additional guidance creates more
cognitive load
10

Goals of this study

Examine the effects of feedback during
exploratory math practice for children
with varying levels of prior knowledge
Specifically
Compare feedback vs. no feedback
Compare outcome vs. strategy feedback
Look at effects of prior knowledge
11

Hypotheses
Ho
1:

Feedback > No Feedback

Ho
2:

Strategy Feedback > Outcome Feedback

Ho
3:

Feedback better for children with low prior

knowledge
12

Domain: Mathematical
Equivalence
Concept that two sides of an equation
represent the same amount and are
interchangeable
Commonly represented by equal sign
(=)

3+7+8=
3+_

6+4=_+
8
13

Why Study Math Equivalence?

Fundamental concept in arithmetic and algebra
Very difficult for children in U.S.
Interpret equal sign as an operator symbol that
means the total as opposed to relational
symbol
(McNeil, 2008; Rittle-Johnson & Alibali, 1999)

In one study, only 24% of U.S. children in 3rd

and 4th grade solved math equivalence
problems correctly
(McNeil & Alibali, 2000)
14

Participants
Worked with 91 children (2nd & 3rd grade)
-

M age = 8 yrs, 7 mo
53 females, 38 males
45% white, 40% black, 15% other
47% receive free or reduced lunch

15

Design and Procedure

Session 1: Pretest (~25 minutes)
Excluded if score >80% on pretest
measures

Session 2: Intervention & Posttest (~50

minutes)
Session 3: Two-week Retention Test (~25
minutes)
16

Tutoring Intervention
Exploratory Practice
Attempt to solve 12 math equivalence
problems
Randomly assigned to 1 of 3 conditions
No Feedback (n = 31)
Outcome Feedback (n = 32)
Strategy Feedback (n = 28)

Midtest
Brief conceptual instruction
(DeCaro & Rittle-Johnson, 2011)
17

Exploratory Practice
Find the number that goes in the blank.
3+4+8=3+
How did you solve that problem?

No Feedback: OK, lets move on to the next problem.

Outcome Feedback: Good try, but thats not the correct
answer. The correct answer is 12.
Strategy Feedback: Good try, but thats not a correct
way to solve that problem.
18

Assessment of Math
Equivalence
Procedural Knowledge
Use correct strategy to solve
7+6+4=7+_

Used at
Pretest,
Midtest,
Posttest, &
problems
Retention Test

6-4+3=_+3

Conceptual Knowledge
Understand concept of equivalence
What does the
equal sign mean?

4+8=8+4
True or False?

(Rittle-Johnson, Matthews, Taylor, & McEldoon, 2011)

19

Analysis & Results

Contrast-based ANCOVA

(West, Aiken, & Krull, 1996)

Two contrast-coded condition variables

-

Feedback

(no feedback vs. two feedback conditions

combined)

Feedback Type

(outcome feedback vs. strategy feedback)

Two condition x prior knowledge

interactions
Three covariates

Interaction follow-up
Categorize as low vs. high knowledge

(median

split)

Simple main effects of condition

20

Procedural Knowledge
Repeated Measures ANCOVA: Midtest, Posttest, and
Retention Test.

21

Procedural Knowledge
Overall feedback x prior knowledge interaction, F(1, 83) = 7.05, p = .01
Low Knowledge: Feedback vs. No Feedback, F(1, 83) = 3.84, p = .05
High Knowledge: Feedback vs. No Feedback, F(1, 83) = 4.61, p = .04

22

Procedural Knowledge Results

Expertise reversal effect
Feedback during exploratory math practice
is more beneficial than no feedback, but
only for children with low knowledge
For children with high prior knowledge, the
reverse is true; they benefit more from
no feedback
23

Intervention Activities
Subjective Cognitive Load

(NASA-TLX, Hart & Staveland,

1988)

I had to work hard to solve those problems.

I was stressed and irritated when I had to
solve those problems.
Mean rating on agreement scale from 1 to 5.

Problem-Solving Strategy Use

Variability of correct & incorrect strategies

24

Cognitive Load

Low Knowledge: Feedback vs. No Feedback, F(1, 83) = 1.15, p = .29

High Knowledge: Feedback vs. No Feedback, F(1, 83) = 6.05, p = .02

25

Cognitive Load
Supports expertise reversal effect
explanation
Feedback may have hurt high-knowledge
learners performance because of
increased cognitive load
An effect not found for low-knowledge
learners
26

Strategy Coding Scheme

Strategy

Sample explanation (4 + 5 + 8 = __ + 8)

Correct Strategies
Equalize

I added 4, 5, and 8 and got 17. And 9 plus 8 is

17.

Add-Subtract

I added 4, 5, and 8 and got 17. And 17 minus

8 is 9.

Grouping

I took out the 8s and I added 4 plus 5.

Incorrect Strategies
Add-All

I added the 4, 5, 8 and 8.

Add-to-Equal

I just added the first three, the 4, 5, and 8.

Carry

I saw a 4 here, so I wrote a 4 in the blank.

Ambiguous

I used 8 plus 8, and then 5.

27

Perseveration
Perseveration = Using the same incorrect strategy on all
the problems.

* P = .01
28

Incorrect Strategy Variability

*
*

Ranged from 0 to 5.
Low Knowledge: Feedback vs. No Feedback, F(1, 83) = 5.66, p = .02
High Knowledge: Feedback vs. No Feedback, F(1, 83) = 4.56, p = .04

29

Correct Strategy Variability

Ranged from 0 to 3.
Low Knowledge: Feedback vs. No Feedback, F(1, 83) = 4.76, p = .03
High Knowledge: Feedback vs. No Feedback, F(1, 83) = 0.16, p = .90

30

Strategy Variability
For children with low prior knowledge,
feedback prevented perseveration and led
to the generation of diverse strategies
-

May explain why these children learned more

when they received feedback than when they
did not

For children with higher prior knowledge,

feedback led to the generation of more
incorrect, but not correct strategies
-

May help explain why feedback had

negative impact
31

Summary
Feedback led to higher procedural
knowledge of math equivalence than no
feedback, but only for children with low
prior knowledge
For children with high prior knowledge, no
feedback was better
No differences between outcome feedback
and strategy feedback
32

Implications
Theoretical
Extends expertise reversal effect to
feedback
Clarifies research on discovery learning

Practical
Pay more attention to when you give
feedback during tutoring and teaching
33

Thank You
Childrens
Learning Lab
Bethany RittleJohnson
Marci DeCaro
Laura McLean
Maryphyllis Crean
Lucy Rice

Funding Sources
ExpERT Training
Grant, through IES
to Fyfe
NSF CAREER grant to
Rittle-Johnson

34

Instruct vs. Discover

Direct Instruction
Told or shown how to
solve the problems
Provides structure and
reduces task ambiguity
But
Limits self-discovery
Might limit learner
engagement

Discovery Learning
Explore the problems on your
own with no guidance
Find target information and
new strategies independently
But
Overwhelms working
memory
Might never locate target
info or invent correct
strategy

(Kirschner, Sweller, & Clark, 2006; Mayer, 2004; Sweller, 1988)

35

Exploratory Practice
Childrens performance on 12 problems
during intervention (Percent Correct)

36

Instruction
Lets take a look at this problem.
3+4=3+4
There are two sides to this problem, one
on the left side of the equal sign and
one on the right side of the equal sign
The equal sign means that the left side of
the equal side is the SAME AMOUNT AS
the right side of the equal sign. That is,
things on both sides of the equal sign
are equal or the same.

37

Conceptual Knowledge

Feedback type x prior knowledge interaction, F(1, 83) = 4.82, p

= .03.
Low knowledge: No effect of feedback type, F(1, 83) = 0.51, p = .
48

38

Intervention Strategy Use

Incorrect

Correct

*
*

Note. Differences are between no feedback condition and two feedback

conditions combined: * p < .05

39

Strategy Variability

Note. Difference is between no feedback condition and two feedback

conditions combined: * p < .001

40