CHAPTER 4 Tutorials 113

Although judgment of drawing is difficult for computers, it is easier for people. A

learner may be instructed to draw. then be shown a correct drawing overlaid for com.
parison, and asked to indicate whether his or her response is sirnilar to the overlay. With
two leamers working together, one may draw and the other may evaluate the drawing.

General Judging Considerations

Tie following judging considerations are relevant for several or all of the response types
discussed above.

Length In judging text situadons a ümit should be set on the permissible length of a leamer
response. For a multiple-choice question this is a single character. For a sentence completion
question expecting a single word, the limit might be ten or fifteen letters. A short-answer question
might allow thirty or more letters. In all cases, if the learner exceeds the permissible length, the
program should not judge the response incorrect, but should inform the learner of a format
problem and await alteration of the response.

Time Limits We discourage a time limit on answering questions in tutorials. If a long response
latency indicates a problem, such as the leamet not knowing what to do, the program
should ask whether help is needed. rather than determining that the lack of response is
equivalent to an incorrect response.
Help and Escape Options Even when the learner is engaged in questions, global options
such as asking for help or leaving the program should still be provided. Consequendy, the
judging machinery must recognize certain mouse positions, keyptesses. or words as
special requests rather than as responses. It is common for programs that otherwise have
good global. controls to disable them during questions simply because they are harder to
program within question sequences. Rollovers, if generally available, should still operate
correctly during questions.

Conclusion
Given the many. considerations involved in judging, it is not surprising that most
multimedia programs do it poorly. Two main recommendations address judging. First,
strive for a program that judge.s responses the same way a human would. Second, design
questions that foster response economy, thus making judgment easier.

Feedback about Responses

Feedback is the reaction of a program to the learner's response and may take many forms,
including text messages and graphic illustrations. The research literature conceming the
role of feedback in learning is extensive and ongoing. Several good summaries can be
found in Dempsey and Sales (1993). The most common function of feedback is to inform
the learner about the correctness of a response. Providing reinforcement for the learner
should follow correct responses. Providing correction, with the

114 PART Il Methodologies

 purpose of improving future performance, should follow incorrect responses. In tutorial programs, especially,
feedback should encourage the learner to improve thinking and comprehension (Schirnmel, 1988).
As discussed in the previous section, several judgments are possible for a response. Let us consider the
feedback appropriate for each.

Feedback Following Format Errors A format error is an error of fom rather than content, such as using letters instead
of numbers. Feedback should prompt the leamer to correct the format and to try again. For example, feedback
should say, "Please use numerals only. Press <ENTER> to try again," rather than saying, "Your answer is wrong, try
again." Figure 4.24 shows feedback for a format error.

Feedback Following Correct Responses When a response is correct, a short amrmation is made,
usually with a single word such as good or correct. Many programs randomly select different "correct"
words for the sake of variety. Figure 4.25 demonstrates feedback after a correct response. Programs
for children frequently engage in procedures to reinforce correct responses as well. This may be done
with encouraging words such as "You're doing a great job!" or with an interesting picture or animation.
Such reinforcers should have variety and should not be too time consuming, especially if they occur
frequently.

Feedback Following Neutral Responses A response may be neither right nor wrong, as when you are
asked for your name. Simple confirmation feedback such as •Thank you. Press <ENTER> to continue"
is appropriate in this case.

Feedback Following Content Errors A response may be incorrect or only partially correct. Feedback
following errors has a great effect on the sudess ofsinsüuction, and the remainder of this section deals
with the nature of feedback in this situation.

How many planets are there in the solar

system?

answerb nine

Please use numerals. Press and try again.

4
CHAPTER Tutorials
115

Positive and Corrective Feedback

Timing of Feedback
116 PA RT Il Methodologies

delayed feedback shows an advantage. But studies in which the nature of leaming is more like a
typical multimedia program show an advantage for immediate feedback. They recommend that
immediate feedback always be used in multimedia programs.
Additionally, research showing an advantage for delayed feedback has generally been
demonstrated for propositional knowledge (verbal information, knowledge, principres). In contrast,
immediate feedback is more likely to enhance leaming procedural knowledge (Anderson, 1982).
Irrespective of its effects on learning, immediate feedback is easier to administer in tutorials than is
delayed feedback. It may also be the case. as Gaynor (1981) found, that learners believe the
computer is not working properly when feedback is delayed. In general, immediate feedback works
best in tutorials.

Types of Feedback
Text Feedback The most common type of feedback is to give the cortect answer in text form, below the
leamer's incorrect response. For completion questions, this is usually the word or phrase most
preferred in the blank. For altemate-response questions, it is some indication of the correct
alternative, such as the correct letter for a multiplechoice question. For completion questions, feedback
in the form of the correct answer may be inserted directly into the blank (or blanks) in the original
question. Although aesthetically pleasing, feedback of this type might go unnoticed by the learner
because the eyes would be looking for a response lower on the screen. A learner is more likely to
miss someth.ing that is added to the screen above the last item read than below it. Some technique for
highlighting text is probably advisable to make feedback obvious.
Text feedback need not supply the correct answer. It may supply a hint so the leamer can try
again. Common ways of providing a hint are

Rewording the question or problem; highlighting key words%r parts

Showing the solution for a similar problem
Giving the learner part of the correct answer
Giving the learner information that helps generate the corfect answer
Graphic Feedback Effective feedback can be graphic. Extended verbal explanations may sometimes be
eliminated in favor of a well-placed arrow or picture. Figure 4.26 shows feedback following erors for the
question that was first presented in Figure 4.19. The graphic feedback is the rulers prompting the learner to
deter-nine whether the lenses are thicker in their centers or on their edges. This feedback also encourages
further analysis and thinking, rather than giving the correct answer to the leamer (Schimmel, 1988).

Audio and Video Feedback I..alley (1998) demonstrated the potential benefit of video for feedback,
attributing it to Pavio's (1986) Dual Coding Theory. Audio feedback may possess similar advantages
and be easier to implement. Both audio and video are effective at attracting attention, increasing the
 effectiveness of feedback. Video may be especially beneficial when a visual' sequence or process is
being taught. Audio may be most useful as feedback related to visual but nonverbal information (such
as a picture or map) because it allows inspection of the visual information while listening to verbal
feedback
4

Use the rulers to Identify your mistakes.

Drag your errors to the correct places.

Drag concave lenses here. Drag convex lenses

here.
mm.m.mm.mmmmm

One incorrect. One incorrect.

F I G U R E 4.2 6
Graphic Feedback (the
Rulers) Following an Error

Markup Another form of graphic feedback is answer markup, which is used when a response is partially correct. Special
symbols indicate errors and missing information. figure 4.27 demonstrates markup in the foreign language program Dasher
(Pusack. 1990). Correct letters in the response remain on the display and incorrect or n•issing letters are replaced with
dashes. The typing prompt is positioned at the first dash. The learner corrects the response by typing in ute missing or
incorrect letters, rather than retyping the entire word. This technique is simple for learners to understand and use. It also
fosters response economy, especially recommended for multiple-string response judging.

Edit Belp
FI GU RE :4.2
-

7
talking about plum for smmer vacation. Feedback in Dasher The
qtven to utelr sUuments In ete p cesent learner enters only the
tense Ling or -stay". missing or incorrect part.
Sue: My and t/ to Bt•ussets
Counesy of CONDUrr.

boyfriend and I go— to Brucs•lc.

Hy Soyfrl•ad •ad I go to [l I
that's not quite right.
 CHAPTER Tutorials
117

118PA RT Il Methodologies

Error-Contingent Feedback
Feedback tailored to the nature of the learner's error is called error-
contingentfeedback. Response markup is an example, because
different symbols indicate specific errors in the response. Response
markup is usually employed when the response is partially
correct. However, even when a response is totally wrong, feedback
specific to the error is useful. Consider the examples in Figures 4.28
and 4.29. The feedback in Figure 4.29 is better because it brings
attention to the fact that the leamer is probably adding instead of
multiplying. The responses and feedback in Figures 4.30 through 4.33
also illustrate crror-contingent feedback.

How much is 4 times 5 ?

answerb 9

Incorrect. Press to try again.

How much is 4 times 5 ?

answerb 9
You are adding. Times means
multiply.

Press to try again.

 R4

Subsequent
Attempts

learner to request the answer. 120 PART Il Methodologiß

 Who invented the electric light
bulb?
CHAPTER Tutorials
answer> Thomas Edison 119

Almost perfect. Try again, using

his middle name as well.
FIGURE4.32
Press Error-Contingent Feedback for a Partially Correct Response

Who invented the electric light

bulb?

answer> Thomas Alva Edison

Very good. Click to continue.

F I G U R E 4 . 33
Féédback When the
Response Is Finally
Correct

Remediation
Whereas feedback is generally concise information about a response, remediation refers to the more
extensive presentation of information for the learner who is consistently not learning the material.
Tutorials vary from giving no remediation to giving remediation following each learner response error.
The types of remediation used in programs are those used by most teachers in classroom instruction.
The most common remediation procedure is to repeat information already seen. Al though this is
not elegant, it is often effective, for example, for learners who are not reading carefully. A similar technique
is to provide restatements of information with new and simpler wording. Providing new information or
repeating old information with more detail also aids the leamer who is paying attention, but who did not
understand the initial
 CHAPTER a Tutorials

presentation. ms may take the form of more examples, pictures. sample problems, or practice with
simpler parts of the material.
A computer program is not always effective for all learners. A remediation technique that
should not be overlooked is having the learner use other media (textbooks, workbooks, films) or work
with a live instructor. The use of remediation techniques is one of the most undeveloped areas of
multimedia instruction. Little research guides either when to initiate remedial instruction or what method
of remediation to use. Programs should keep track of learner responses and, when errors are being made
repeatedly, should provide remediation appropriate to the content and to the leamers.

Organization and Sequence

Program Segments
Types of Information Organization
The sequence of program segments depends in part on the nature of the information taught. Four types of
infonnation are common:

• Verbal information
Concepts
• Rules and principles
• Skills
Verbal Information Verbal infomation may present many kinds of relationships: temporal (do A after- you
do B), causative (A causes B), categorical (A is a member of B), exemplary (A is an example of B),
characteristic (A is a characteristic or property of B), or comparative (A and B are compared for their
similarities and differences). Verbal leaming is facilitated when individual elements are presented. in a
logical fashion and their relationships are examined. Organizational summaries are useful for making
these relationships to students. Such summaries may be textual, in the fonn of lists of points or
outlines (Figure 4.34), or pictorial, with textual infonnation arranged spatially and connected with arrows
or other symbols to show relationships (Figure 4.35).

Conceptual Information Conceptual information includes concrete concepts such as cat or circle, and
defined concepts, such as the chemical concept oxidation or the social concept city. Some concepts are
difficult to define (for leamers and instructors alike), such as the concepts love, sentence, or theme of a
short story. A concept is a class of things with common characteristics that are distinguished from other
things not sharing those characteristics. Much of what we learn consists of concepts. Programs teaching
concepts typically use the following organization.
First the characteristics that define the concept are taught. These are called the relevantfeatures or the
essential features. Teaching the characteristics of a concept is frequently done by stating a definition in
terms of the relevant features. Next, simple instances of the concept are given, such as a dog being an
instance of the concept mammal. Simple instances are those that contain all or many of the cetevant
features, and few
 121

122 PART Il Methodologies

Summary of lesson COMMAS

Commas are used
• to separate Items In a series,
• following introductory phrases,
• preceding coordinating conjunctions
that link main clauses, and
• to separate parenthetical elements in
a sentence.
 or no irrelevant or incidental features. Irrelevant features are neither necessary nor commonly
possessed by instances of a concept. Incidental features are those commonly possesscd but not
necessary to a concept. Simple noninstances of the concept are given for contrast. Noninstances are
those things not of the class. Simple noninstances contain few or no relevant or incidental features and
many irrelevant featurés.
After the learner has learned the concept at the level of simple instances and noninstances, difficult instances
and noninstances are introduced. Difficult instances are those with few relevant features and many irrelevant
and incidental features. Difficult noninstances have many relevant and incidental features and few irrelevant
features. For ample, to teach the concept mammal, a program would begin with cats and dogs (simple

Organization and Sequence

Program Segments
Types of Information Organization
The sequence of program segments depends in part on the nature of the information taught. Four types of
infonnation are common:

• Verbal information
Concepts
• Rules and principles
• Skills
Verbal Information Verbal infomation may present many kinds of relationships: temporal (do A after- you
do B), causative (A causes B), categorical (A is a member of B), exemplary (A is an example of B),
characteristic (A is a characteristic or property of B), or comparative (A and B are compared for their
similarities and differences). Verbal leaming is facilitated when individual elements are presented. in a
logical fashion and their relationships are examined. Organizational summaries are useful for making
these relationships to students. Such summaries may be textual, in the fonn of lists of points or outlines
(Figure 4.34), or pictorial, with textual infonnation arranged spatially and connected with arrows or other
symbols to show relationships (Figure 4.35).

Conceptual Information Conceptual information includes concrete concepts such as cat or circle, and
defined concepts, such as the chemical concept oxidation or the social concept city. Some concepts are
difficult to define (for leamers and instructors alike), such as the concepts love, sentence, or theme of a
short story. A concept is a class of things with common characteristics that are distinguished from other
things not sharing those characteristics. Much of what we learn consists of concepts. Programs teaching
concepts typically use the following organization.
First the characteristics that define the concept are taught. These are called the relevantfeatures or the
essential features. Teaching the characteristics of a concept is frequently done by stating a definition in
terms of the relevant features. Next, simple instances of the concept are given, such as a dog being an
instance of the concept mammal. Simple instances are those that contain all or many of the cetevant
features, and few