(To be optimal in the college setting, it seems that students need access to all TELL components in

their rooms at any time of the day.) What is the value of collaborative work
among students using TELL outside the class- room? How could TELL be
adapted to enhance learning by targeting the various multiple intelligences?
How could TELL components be individu- alized for remediation of students
with particularly weak language skills in language learning? And finally, can
TELL be developed to track effec- tively the linguistic development and
cultural understanding of students? Clearly, the examination of the integration
of technology into second lan- guage teaching and learning is fertile ground for
future research.

Volume 17 Number 2 295

Evaluating the Integration of Technology

APPENDIX
A

Lab Assignment Sheet

le français 102A/Youngs/Printemps 97 Exercice de labo # 1

L'étudiant(e) Date:

Fill out this checklist for each assignment you do using multimedia or lan-
guage software.
Task Where? Berapa lama? Assessment Scale: Check tools used
1 (poor) to 5 (great)

Reader Fre Glossary (elec.) Chapt. 6 1 - 2 - 3 - 4 - 5 Paper Dictionary

(server) Netscape
Dasher ModLang
Server LLRC
computers
Spellchecker
Grammar reference
Allons-y textbook
Verb book Other?
(please list)

Additional information (you may be brief):

1. Please comment on why you chose the assessment (1-5) above.

2. How did you feel during this assignment? (upset, nervous, enthusias-
tic, interested, bored...)

3. Les Devoirs (essayez d'utiliser le glossaire français):

296 CALICO Journal

Adair-Hauck, Willingham-McLain, and Youngs

a. A l'écrit: Qu'est-ce que vous faites pour vous préparer pour la rentrée?
 b. A l'oral: Préparez un dialogue entre vous et le vendeur ou la vendeuse chez
la librairie. Achetez tout ce qu'il vous faut pour la rentrée.

APPENDIX B

Francophone Cultural Knowledge Quiz

Answer the questions in English, or give the French expression where asked.
1. Define “Francophone.” 2. Name five Francophone countries (not France). 3.
Name five cities in France. 4. Name denominations of French and Canadian
currency. 5. Name as many forms of public transportation in Paris as you can.
6. What can you use, besides coins, to make a phone call in France, and
where would you go to get one? 7. a.
Name a French political figure.

b. Name a French film star.

c. Name a French scientist.

Volume 17 Number 2 297

Evaluating the Integration of Technology

8. a. What do you say if someone says “merci” to you?

 b. How do you say you're feeling fine?

9. What are some things that French people typically have for breakfast? 10.
What is a croque-monsieur?
Sebuah. a drink b. an opera c. a sandwich d. a type of building 11. Label the
order in which the following items are typically served dur-
ing a French dinner:
salad cheese coffee main dish dessert 12. What would you
purchase at a “charcuterie?” 13. Which best describes a
traditional French university.
Sebuah. a complex of classroom buildings and sports facilities b. a
sprawling suburban campus c. an urban campus, no green areas
14. Which field of study does “lettres” refer to?
Sebuah. education b. fine arts c. humanities d. sciences 15. Where do most
French university students live?
Sebuah. in rented rooms in town b. in university dorms c.
with their families 16. How are floors of a building
numbered in France? 17. What's a “deux-pièces?”
Sebuah. a coin b. an apartment c. a theater d. a vehicle 18. What does a green
cross refer to?
Sebuah. a church b. a pharmacy c. a political party 19. What are the following (city,
region, country) and where are they lo-
cated? What do you associate with each one?
Apa? Dimana? Association? Sebuah. Guadeloupe b. Morocco c.
Louisiana d. Normandy e. Bordeaux 20. How would you describe French
people to someone from your coun-
try? What stereotypes do you have of French people? Mengapa?

298 CALICO Journal

Adair-Hauck, Willingham-McLain, and Youngs

APPENDIX C

Oral Test Grading Grid

Picture description/comparison Vocabulary/content 5 4 3 2 1 0 Structures

5 4 3 2 1 0 Accomplishes the task 5 4 3 2 1 0 /15
 Role play Vocabulary/content 5 4 3 2 1 0 Structures 5 4 3 2 1 0
Level of formality 3 2 1 0
(polite) Accomplishes the task 5 4 3 2 1 0 /18

Telling a story in the past Vocabulary/content 5 4 3 2 1 0 Structures 5 4 3 2

1 0 Accomplishes the task 5 4 3 2 1 0 /15

Overall Communication strategies 3 2 1 0

(asking for clarity, getting point across) Pronunciation/fluency 3 2 1 0 /6
(generally accurate pronunciation,
not groping for words)

TOTAL /54

Volume 17 Number 2 299

Evaluating the Integration of Technology

APPENDIX D

Final Questionnaire (for treatment group)

1. What did you like most about the multimedia activities (Dasher, reader,
video)? 2. What did you like least about the multimedia activities (Dasher, reader,
video)? 3. Were they effective in helping you develop the following skills in
French? Yes or no? Please explain your answers. Reading Writing Listening
Speaking 4. Were they effective in helping you to increase your knowledge of
francophone culture? Mengapa atau mengapa tidak? 5. At times were you frustrated with
the multimedia activities? If so,
please tell which aspects frustrated you. 6. Did the benefits of the multimedia activities
 outweigh your frustra-
tion? Please explain. 7. Did the multimedia activities help your self-confidence to speak,
read,
write and listen to French? Please explain. 8. Did you ever do parts of the lab in
collaboration with another stu- dent? Yes or no (circle one)? If yes, do you feel
that working together facilitated your learning of French? 9. Were the speaking
and writing follow-up activities done on/for Wednes- day effective ways to
synthesize and apply what you had learned in the lab? Please comment:
Speaking activities: Written work: 10. Rank these language learning resources
from 1 (most helpful) to 7
(least helpful). Feel free to comment.
Allons-y textbook Allons-y
video Dasher exercises
French glossary (electronic)
paper dictionary Reader
(electronic) Spellchecker

300 CALICO Journal

Adair-Hauck, Willingham-McLain, and Youngs

NOTES

1
Nieves (1994) did a pilot study exploring the use of a multimedia program in Spanish
entitled Exito. The project included the development of a first-semester Spanish course
using multimedia, classroom-based activities based on the multi- media materials and
assessment of the students' language development. The re- search project highlighted
that students could develop a beginning level of profi- ciency in Spanish by interacting
with the multimedia program and by meeting with a professor in small groups one

period per week. 2 We used the Dasher authoring system to create vocabulary and

grammar exer- cises (Pusack & Otto, 1992). 3 It is not our intent to promote any

particular software, video, or textbook mate- rials. Since we were working at Carnegie
Mellon University, we chose materials already adopted and tested by the Department of

Modern Languages. 4 Following Chamot, (personal communication, November 1995),

we also col- lected information about students' affect at the beginning and end of the

semes- ter. It is beyond the scope of this article to report those findings. 5 We would
 have preferred to administer externally rated pre- and posttests on all skills but this
would have taken too much class time and would thus have been unfair to the students.

Therefore, we chose to use data from classroom tests. 6 The writing was analyzed using

an adapted form of the ESL Composition Profile (Glisan, 1981). 7 Given the wide range

of time students reported, we wonder if some students may have mistakenly included
the time they spent preparing the follow-up writing and speaking assignments in the

early weeks. 8 We chose to use the Reader because the courseware had been used

successfully and repeatedly in fourth semester French classes at Carnegie Mellon

University. However, neither the courseware nor the readings themselves had been

piloted in second semester classes. 9 Appendix D contains the final open-ended

questionnaire administered to the treatment group. Control group students filled out an

adapted questionnaire. 10 At Carnegie Mellon University, the Department of Modern

Languages offers instructors the option of capping classes at 18 students. 11 In

1998-1999, CMU does not use the Reader in the first year French curricu- lum but
instead requires students to work with documents on the web. These documents afford
even greater authenticity and provide completely up-to-date information (eg, weather
reports, news).

Volume 17 Number 2 301

Evaluating the Integration of Technology

REFERENCE
S

Alderson, JC, & Scott, M. (1992). Insiders, outsiders and participatory evalua- tion. In
JC Alderson (Ed.), Evaluating second language education (pp. 25-58). Cambridge,
Inggris: Cambridge University Press. Armstrong, KM, & Yetter-Vassot, C. (1994).
Transforming teaching through tech-
nology. Foreign Language Annals, 27, 475-486. Barnett, MA (1989). Writing as a process.
French Review, 63 (1), 34-44. Beauvois, MH (1992). Computer-assisted classroom
instruction in the foreign language classroom: Conversation in slow motion. Foreign
Language Annals, 25, 455-463. Beretta, A. (1986). A case for field-experimentation in
 program evaluation. Lan-
guage Learning, 36, 295-309. Bernhardt, SA, Edwards, P., & Wojahn, P. (1989). Teaching
college composition
with computers. Written Communication, 6 (1), 108-133. Braegger, JD, & Rice, DB
(1996). Allons-y: Le Français par étapes (4th ed.).
Boston, MA: Heinle & Heinle. Brandl, K. (1995). Strong and weak student preferences for
error feedback options
and responses. The Modern Language Journal, 79, 194-211. Brown, JD (1989).
Language program evaluation: A synthesis of existing possi- bilities. In RK Johnson
(Ed.), The second language curriculum (pp. 222-241). Cambridge, Inggris: Cambridge
University Press. Brown, JD (1995). Language program evaluation: Decision, problems
and solu-
tion. Annual Review of Applied Linguistics, 15, 227-248. Bump, J. (1990). Radical
changes in class discussion using networked computers.
Computers and the Humanities, 24, 49-65. Chun, DM (1994). Using computer
networking to facilitate the acquisition of
interactive competence. System, 22, 17-31. Cononelos, T., & Olivia, M. (1993). Using
computer networks to enhance foreign
language/culture education. Foreign Language Annals, 26, 527-534. Conrad, BK (1996).
CALL: non-English L2 instruction. Annual Review of Ap-
plied Linguistics, 16, 158-181. Costanzo, W. (1994). Reading, writing and thinking in an age
of electronic literacy. In CL Selfe & H. Susan (Eds.), Literacy and computers (pp.
54-73). New York: The Modern Language Association of America. Cummins, J.
(1991). Introductory remarks. In R. DeVillar & C. Faltis (Eds.), Com- puters and
cultural diversity: Restructuring for school success (pp. vii- ix). Albany, NY:
Universitas Negeri New York Press. Daiute, C. (1984). Can the computer stimulate
writer's inner dialogues? In W. Wrench (Ed.), The computer in composition instruction.
Urbana, IL: NTCE. DeVillar, R., & Faltis, C. (1991). Computers and cultural diversity:
Restructuring
for school success. Albany, NY: Universitas Negeri New York Press.

302 CALICO Journal

Adair-Hauck, Willingham-McLain, and Youngs

DiMatteo, A. (1990). Under erasure: A theory of interactive writing in real life.

Computer and Composition, 7, 71-89. Donato, R., Antonek, J., & Tucker, GR (1996). Monitoring and
assessing a Japa- nese FLES program: Ambiance and achievement. Language
Learning, 46 (3), 497-528. Garrett, N. (1991). Technology in the service of language
learning: Trends and
issues. The Modern Language Journal, 75 (1), 74-101. Glisan, E. (1981). ESL
Composition profile. In HL Jacobs, S. Zingraf, D. Wormuth, V. Hartfield, & J. Hughey
(Eds.), Testing ESL composition: A practical approach (pp. 323-324). Rowley, MA:
Newbury House. Greenia, GD (1992). Computers and teaching composition in a
foreign language.
Foreign Language Annals, 25, 33-47. Hass, C. (1989). How the writing medium shapes the writing
 process: Effects of word processing on planning. Research in the Teaching of English,
23, 181-207. Hawisher, GE (1989). Research and recommendations for computers and
com- position. In GE Hawser, & CL Selfe (Eds.), Critical perspectives on computers
and composition instruction (pp. 44-69). New York: Teach- ers College Press. Herron,
C., & Moos, M. (1993). Electronics media in the foreign language and literature
classroom: A fusion between science and the humanities. For- eign Language Annals,
26 (4), 478-490. Horwitz, EK, & Young, DJ (1991). Language anxiety: From theory
and re-
search to classroom implications. Englewood Cliffs, NJ: Prentice Hall. Johnson, DM (1985). Using
computers to promote the development of English as a second language: A report to the
Carnegie Corporation. Tuscon: Uni- versity of Arizona. (EDRS:ED 278 211). Johnson,
DM (1991). Second language and content learning with computers: Research in the role
of social factors. In P. Dunkel (Ed.), Computer-as- sisted language learning and
testing: Research issues and practice (pp. 61-83). New York, NY: Newbury House.
Johnson, DM (1992). Approaches in second language learning. White Plains,
NY: Longman Publishing Company. Kelm, OR (1992). The use of synchronous computer networks in
second lan- guage instruction: A preliminary report. Foreign Language Annals, 25 (5),
441—454. Lafford, PA, & Lafford, BA (1997). Learning language and culture with
internet technologies. In MD Bush, & RM Terry (Eds.), Technology-enhanced language
learning (pp. 215-262). Illinois: National Textbook Company. Liu, M., & Reed, WM
(1994). The relationship between learning strategies and learning styles in a
hypermedia environment. Computers in Human Be- havior, 10 (4), 419-434. Long, MH
(1984). Process and product in ESL program evaluation. TESOL
Quarterly, 18, 409-425.

Volume 17 Number 2 303

Evaluating the Integration of Technology

Lynch, B. (1990). A context-adaptive model of program evaluation. TESOL Quar-

terly, 24, 23-42. Meunier, LE (1994). Computer-assisted language instruction and cooperative
learning. Applied Language Learning, 5 (2), 31-56. Musumeci, D. (1998). Mallard, a
web-based application in use at the University of Illinois (technology demonstration).
Pittsburgh, PA: University of Pitts- burgh. Mydlarski, D. (1987). Cooperative
computer-assisted language learning: Is it living up to its promise? Journal of
Educational Techniques and Technologies, 20 (1), 26-29. Nagata, N. (1993). Intelligent
computer feedback for second language instruction.
The Modern Language Journal, 77, 330-339. Nagata, N., & Swisher, VM (1995). A study of
consciousness-raising by com- puter: The effect of metalinguistic feedback on second
language learn- ing. Foreign Language Annals, 28, 337-347. Neu, J., & Scarcella, R.
(1991). Word processing in the ESL writing classroom. In P. Dunkel (Ed.),
Computer-assisted language learning and testing: Re- search issues and practices (pp.
169-187) New York: Newbury House. Nieves, KA (1994). The development of a
technology-based class in beginning level Spanish: Experiences with Exito. (Doctoral
dissertation, George Mason University). Phinney, M. (1988). Computers, composition
and second language teaching. In MC Pennington (Ed.), Teaching languages with
computers: The state of the art. La Jolla, CA: Althesan. Phinney, M. (1991).
 Computer-assisted writing and writing apprehension in ESL students. In P. Dunkel
(Ed.), Computer-assisted language learning and testing: Research issues and practices
(pp. 189-204). New York: Newbury House. Piper, A. (1986). Conversation and the
computer: A study of the conversational spinoff generated among learners of English as
a foreign language work- ing in groups. System, 14, 187-198. Pitiyanuwat, S. (1986).
On-the-job behavior of trainees as a function of their reac- tion to the training program
and learning achievement. In A. Wangsotorn, A. Maurice, K. Prapphal, & B. Kenny
(Eds.), Trends in language pro- gram evaluation (pp. 142-154). Bangkok:
Chulalongkorn University. Pusack, JP, & Otto, S. (1997). Taking control of multimedia.
In MD Bush & RM Terry (Eds.), Technology-enhanced language learning (pp.1-46).
Lincolnwood, IL: National Textbook Company. Raschio, R. (1990). The role of
cognitive style in improving computer-assisted lan-
guage learning. Hispania, 73 (2), 535-541.

304 CALICO Journal

Adair-Hauck, Willingham-McLain, and Youngs

Reid, JM (1987). The learning style preferences of ESL students. TESOL Quar-
terly, 21, 87-111. Robinson, GL (1989). The CLCCS CALL study: Methods, error feedback, atti-
tudes, and achievement. In WF Smith (Ed.), Modern technology in foreign language
education: Application and projects (pp. 119-134). Lincolnwood, IL: National
Textbook Company. Robinson, GL (1991). Effective feedback strategies in CALL:
Learning theory and empirical research. In P. Dunkel (Ed.), Computer-assisted
language learning and testing: Research issues and practice (pp. 155-167). New York:
Newbury House. Roblyer, M., Castine, W., & King, F. (1988). Assessing the impact of
computer-
based instruction. New York: Haworth Press. Rodrigues, D. (1985). Computers and
basic writers. College Composition and Com-
munication, 36 (3), 336-339. Ross, S. (1992). Program-defining evaluation in a decade of eclecticism.
In JC Alderson & A. Beretta (Eds.), Evaluation in second language education (pp.
167-195). Cambridge, Inggris: Cambridge University Press. Snow, MA, & Brinton,
DM (1988). Content-based instruction: Investigating the effectiveness of the adjunct
model. TESOL Quarterly, 22, 553—574. Stenson, N., Downing, B., Smith, J., & Smith,
K. (1992). The effectiveness of com-
puter-assisted pronunciation training. CALICO Journal, 9 (4), 5-18. Svenconis, DJ, & Kerst, S.
(1995). Investigating the teaching of second language vocabulary through semantic
mapping in a hypertext environment. CALICO Journal, 12 (2/3), 33-57. Tharp, R., &
Gallimore, R. (1988). Rousing minds to life: Teaching, learning, and schooling in
social context. Cambridge, Inggris: Cambridge University Press. Thiesmeyer, J. (1989).
Should we do what we can? In GE Hawser, & CL Selfe (Eds.), Critical perspectives on
computers and composition instruction. New York: Teachers College Press. Tremblay,
PF, & Gardner, RC (1995). Expanding the motivation construct in
 language learning. The Modern Language Journal, 79 (6), 505-520. Youngs, BE, &
Jones, CM (1996). CMU French Reader [Computer software].
Pittsburgh, PA: Carnegie Mellon University. Vygotsky, LS (1978). Mind in society: The
development of higher psychological
processes. Cambridge, MA: Harvard University Press. Wertsch, J. (1979). From social interaction to
higher psychological processes: A clarification and application of Vygotsky's theory.
Human Development, 22, 3-22. Wiggins, G. (1997). Educative assessment: Designing
assessments to inform and improve student performance. San Francisco, CA:
Jossey-Bass.

Volume 17 Number 2 305

Evaluating the Integration of Technology

ACKNOWLEDGMENT
S

We are grateful to G. Richard Tucker, Barbara Freed, Christopher M. Jones,

Keiko Koda and Brian MacWhinney for their feedback on the pilot study and
their encouragement throughout this project. We are indebted to Elaine
Rubinstein in the Office of Measurement and Evaluation of Teaching at the
University of Pittsburgh for conducting the statistical analysis, to Michael
West for his help in drafting the culture quiz, and to Mark Sanford for rating
the writing samples. We also want to acknowledge the work of our two
undergraduate research assistants during the spring of 1997: Ena Khan and
Emily Spencer.

AUTHORS'
ADDRESSES

Bonnie Adair-Hauck University of

Pittsburgh Department of Instruction and
Learning 4M20 Forbes Quad Pittsburgh, Pa
15260 Phone: 724/935-8275 Fax:
724/934-8832 E-mail:
adairhauck@mindspring.com

Laurel Willingham-McLain, Associate Director

Center for Teaching Excellence 312
Administration Bldg Duquesne University
Pittsburgh, PA 15282 Phone: 412/396-1760 Fax:
412/396-6577 E-mail: willingham@duq.edu

Bonnie Earnest Youngs Department of Modern Languages

Carnegie Mellon University Baker Hall 160 Forbes
Avenue Pittsburgh, PA 15213-3890 Phone: 412/268-8050
(office); 412/621-2517 (home) Fax: 412/268-1328 E-mail:
byoungs@andrew.cmu.edu

306 CALICO Journal