You are on page 1of 21

Project Report: TOEFL iBT Speaking Blog

Problem or Need
The overall goal of this project is to improve scores of the Speaking section
for Korean students taking the iBT TOEFL test. Korean students are among
the lowest ranked speakers taking the iBT TOEFL test.
http://www.ets.org/Media/Research/pdf/TOEFL-SUM-0506-iBT.pdf (see page
9/Korea). If this problem is not addressed, students are less likely to get into
the high schools and universities of their choice. While a high score on the
TOEFL Speaking section of the test is not the only important factor into
getting into the school of one’s choice in Korea, it does carry significant
importance. Other factors to consider include the other three sections of the
TOEFL test (Reading, Listening, and Writing), the Su-Neung (Korean college
entrance exam), and participation in a variety of social activities and or
clubs.

Goals and Objectives
After instruction students will be able to answer the six types of speaking
questions found on the TOEFL iBT exam. It is a lofty goal but, after this
course, I hope that students will be able to reach a score of at least 23/30 on
their TOEFL Speaking Test.

I have set up 12 learning objectives as seen below that are based on the
need aforementioned.

1.After reading a sample TOEFL iBT question #1 and/or #2, students will
write 15 words or less in 15 seconds to prepare a spoken response.

2.Using 15 words or less, students will create 45 seconds of speech.

3. As they are reading a 150 word TOEFL iBT sample question #3 passage,
students will take notes on the main idea.

4. As they are listening to a 45 - 60 second long TOEFL iBT sample #3
conversation, students will take notes on the opinions of the people talking in
the conversation.

5. As they are reading a 150-200 word TOEFL iBT sample question #4
academic passage, students will take notes on the main ideas of the
passage.
6. As they are listening to a 60 second long TOEFL iBT sample question #4
lecture, students will take notes on the main ideas the professor talks about.

7. Using the information they gathered in their reading notes and from their
listening notes, students will make a spoken statement that discusses the
relationship between the reading passage and the lecture excerpt.

8. In 30 seconds or less, students will combine notes they took from the
reading to notes they took from the listening in question #4 in order to
prepare a 60 second spoken response.

9. As they are listening to a sample iBT question #5 sample question,
students will take notes on the problem and two solutions that are presented
in the conversation using 15 words or less.

10. In 30 seconds or less, students will state their opinion and support for
their opinion in 20 words or less about which solution they prefer from
sample question #5.

11. As they are listening to a sample iBT question #6 academic lecture
excerpt, students will take notes on the main ideas of the lecture, writing 20
words or less, using the format of; Introduction, Body, Body.

12. In 30 seconds or less, students will prepare a 60 second response to
sample question #6.

* Additionally, a possible ad-on activity mentioned below involves students
doing an exploratory type activity where they listen to the responses of
others and find the student who has the best pronunciation, intonation,
pacing, grammar, word choice, development and so on.

Because of the nature of the test being timed, each learning objective is
connected to the measurable results listed above since each learning
objective is timed.

All of these learning objectives are connected to the indicators listed above
because each question students will be addressing in the learning objective
is a sample question similar to that of what they will see on the sample iBT
TOEFL Speaking pre and post tests.

Because everything is timed, learning objectives 1-12 can be considered met
if they have been done in the appropriate amount of time allotted for. Each
can be measured through use of a stopwatch. There are also timed sample
questions available on CD-rom that can be used.
Analysis of your learners, content, and setting
The learners are middle school Korean students (at least for now) who have
tested into at least an intermediate-level class. These students have the
ability to speak without being prompted for one minute on a given topic.

Because my target population is Korean middle school students, who have
undoubtedly spent countless hours on the computer online, I feel that they
have been exposed to some glimpses of an eLearning environment. While
most of what they have likely spent time with is video games, I know that
many also make use of such eLearning tools as chat rooms and blogs. I will
develop a poll or survey to get a better sense of how much background
students have in Learning. Most of my students can type in English and even
if they can't type well, they will still be able to perform all of the actions
necessary to be successful in my course. As far as self-directed learning
goes, I feel that students have a lot of interest in this type of learning, yet
they have not been given many opportunities to explore this style of learning
in their traditional classrooms. From my experience here so far, I know that
the popular computer game, World of Warcraft is a favorite among students.
This game opens students up to a world they are able to explore on their
own, where the student directs his or her own learning of this new world. For
this reason I feel that Second Life would be a great way to approach my
students (but that is a different course all together). As far as isolation of
eLearning goes, Korean students tend to be taught mostly through the use of
listening to lectures and reading. This does not lend itself well to the
isolation of eLearning since most of my students' experience has come from
a very social atmosphere: that of which is in the classroom. Ultimately, yes, I
do feel that the isolation of eLearning will be a challenge for my students and
it is for this reason that I feel blended learning is the best approach for this
course.

Because students are so young and because students are often enrolled into
a TOEFL Speaking course by their parents and not on their own free will,
motivation is often lacking. Most middle school students have little to no
interest in studying for the TOEFL test. Their only real motivation, if there is
any, is to get into the high school of their choice. That said, I have found
that students who have already taken the actual iBT TOEFL test before
starting my course are much more motivated to learn more from me. This
most certainly pertains to relevance as a learning value. This makes me want
to require students to take the actual test before registering for my course;
yet, this would be an expense and a fear that would make most students
look to a program with less stringent requirements. As stated earlier, I feel
that a pre-test that I score, will reflect an accurate score on which students
can gauge their progress. And, as such, it will serve to help motivate them as
well. The idea of taking a test is certainly one that brings on a great sense of
anxiety for students. Some students simply shut down with the thought of
how important this test may be to them. In particular, they get overwhelmed
by the self-perceived importance of this test. This can often impede
progress as students that shut down tend to be a lot harder to reach. I
strongly feel that by requiring the students to develop and maintain a
portfolio of their speaking responses, students will take on more
responsibility for their own success and in doing so, will be more motivated
to improve.

The physiological characteristics of my students are that of normal, average
(not sure how I'm measuring this) Korean, middle school students. In
reference to cognitive characteristics, students come in all varieties of
ability. More specifically, students’ ability to read, listen and speak in English
is largely attributed to their cognitive ability. My students tend to have a
very solid foundation in both reading and listening. Speaking is where they
need the most work. This lends itself well to the need for this course. Since
four of the six questions on the TOEFL speaking test integrate reading,
listening and speaking, students will be using their strengths while also
working to improve their weaknesses.

Korean students face constant competition and seem to thrive in such an
environment. Their whole life they are pushed to outperform their peers. For
this reason, a huge incentive is to constantly outperform each other. One
activity that could serve as incentive to work hard and deliver a good
response could be to divide students into teams and have them compete for
the best total scores. Student responses will be evaluated during class time,
for all students to hear. The embarrassment of sounding terrible in English
in front of their classmates is often enough incentive to work hard on each
response. However, some students shut down in this environment so it’s
worth considering each student. As a disincentive, students are required to
do a walk of shame to the Korean manager if they do not do their homework
assignments. Typically, students will end up enduring some form of crude
physical punishment. Sometimes it’s holding their hands above their head
for five to ten minutes at a time. Other times, it’s doing squat thrusts, and
sometimes it’s just taking slaps to the palms of their hands. As a last resort,
students’ parents are called if their bad behavior persists. It seems to be a
strong disincentive for students who do not do their work to my
expectations.

Korea is one of the most, if not the most, connected country in the world in
regards to Internet access. Because much of what my class will require
includes accessing the internet, this serves my course well. I think
something like 85% of all Koreans have high speed internet. And even if
some don't have it, there are PC rooms on every corner that provide
broadband internet access at affordable prices. As far as time goes, I think
that meeting once or twice a week for two or one hours respectively will
suffice. If we work in the computer lab in our school, there is a computer lab
attendant available during class time to help with any technical problems.
However, my plan to be proficient in the technologies I plan on using, well
before this class rolls out.

This course will take place in the computer lab, classroom, and at home.
This blended form of delivery is essential to my students because they are
studying English as a Foreign Language. Students will need to use classroom
time to see their native speaking teacher, and in particular to evaluate the
subtleties of expression used by a native speaker speaking live. Additionally,
students will create a blog portfolio of the work they have done to track their
progress and consolidate internet sources that have helped them improve
their speaking ability. Student’s blogs will be added as a link list on the
course shell home page, which will allow them to see each other’s blogs and
also allow me easy access to student blogs.

After taking this course students will increase their TOEFL sample test scores
by 20%

Use of practice tests and the subsequent scoring of each question will allow
students to demonstrate their proficiency in speaking. I am having some
trouble here trying to figure out what type of measurable I want to achieve. I
would love to say that students will increase their actual scores on their
TOEFL speaking section as a result of my course. Yet, this claim is a very
difficult one to achieve. I don’t want to make any false claims. I especially
don’t want to make claims involving getting students into the school they
want since, as stated above, other factors contribute to that as well.
Experience in teaching TOEFL has taught me that even students, who, excel
in my class, who are good speakers of English, and who have previously
taken the test before the class, don’t always improve their speaking scores.
One factor that contributes to this is that many students come to my class
having just returned from the U.S. or Canada where they had lived for an
extended period of time. As soon as they come back, they speak English well
but gradually, they lose much of their English skill through lack of use. While
this is not surprising in any way, since they spend their waking hours
speaking and listening in Korean, it is something that I need to consider
when making goals and claims about students making improvements on
their TOEFL test.

Students will take a TOEFL iBT speaking pre-test before any lessons are
taught. Scores will be given by the instructor based on the ETS-made TOEFL
speaking scoring rubric. I feel comfortable doing this since I was once
certified by ETS to score this test. Scores are then converted using the
scoring conversion chart, again provided to the public by ETS. After students
have finished the 6 week course, they will once again test, using a different
sample iBT TOEFL speaking test. Scores will again be given by the instructor
based on the scoring rubric and conversion chart available on www.ets.org/.
Again, I would love to say the desired outcome of my course is that students
will receive a minimum score on their TOEFL speaking section of 23/30. Yet,
as stated above, I’m not sure if this kind of claim would benefit me since
students test results are scored by someone other than me and student test
anxiety and other factors such as a gradual decrease in speaking ability are
unpredictable and very difficult to measure. I like the idea of a 10-20%
increase in practice test scores because I can measure it.

Instructional Plan
The information provided below is an outline of what will be taught in the
course.

I. Introduction

A. What is the TOEFL Test?

1. How many sections on the test?

B. What is the TOEFL Speaking Section?

1. 6 Questions/20 minutes

II. Tools Needed

A. Book

1. Syllabus for book use

B. Creating a TOEFL Blog

1. Blogger.com

C. Audacity/Gom Recorder

1. Recording your responses and creating an html media player

through www.snapdrive.net.

a. linking the the media player to your blog

D. www.ets.org
1. Using resources

2. registration for test

III. Question One

A. Personal Preference (Description)

1. Thinking Fast

2. Taking notes

a. 15 words/15 seconds

3. 45sec. Response

a. using your notes

(1) expanding your thoughts

(2) use of transition words

(3) organization

IV. Question Two

A. Choice Question (Description)

1. Thinking Fast

2. Taking notes

a. 15 words/15 seconds

3. 45sec. Response

a. using your notes

(1) expanding your thoughts

(2) use of transition words

(3) organization
V. Question Three (Integrated: Reading, Listening, Speaking)

A. Reading: Campus Announcement

B. Listening: Students talking

1. Taking notes on student opinions

C. Combining input (reading/listening) to answer question

VI. Question Four (Integrated: Reading, Listening, Speaking)

A. Reading: Academic Passage

B. Listening: Lecture Excerpt

C. Relationship question

VII. Question Five (Integrated: Listening, Speaking)

A. Listening (Campus Situation)

1. Problem and Two Solutions

a. taking notes

2. Forming your opinion

a. using your notes to create your opinion

VIII. Question Six (Integrated: Listening, Speaking)

A. Academic Lecture

B. Organizing the listening as you hear it

C. Summarizing the lecture

Students will be given homework assignments twice a week. These
assignments will include but are not limited to: answering sample questions
via mp3 voice recording, uploading the mp3s to the web, and posting them
to the student blogs. Additionally, there will be exploratory tutorials available
for students to examine websites that pertain to speaking. Such activities will
require students to find good quality speakers on You Tube or TED, evaluate
the speaker using the TOEFL rubric, and then post the video as a link, along
with their evaluation on their blog.

In my course the “Learner Centered” Instructional Value will be
accomplished in a few ways. First of all, students are required to record their
voices on a regular basis. In doing so, students will be taking ownership for
their responses. Students will be made aware of the Speaking rubric
available from ETS, and will also have access to a number of benchmark and
sample responses available for them to listen to and base their responses
and evaluations on. Allowing the students access to listen to benchmarks,
this serves as a presentation type of absorb activity, which Horton discussed
in chapter two. Of course, students will be listening to my presentations and
lectures on the subject matter as well. Additionally, students will be divided
into teams in which they will be evaluating the responses of their
classmates. Obviously this serves the student who is evaluating more than
the one who is being evaluated since it allows for a worry free chance to
learn more about what types of things scoring is based on. Part of this
evaluation process will involve students evaluating other students based on
the scoring rubric. As an extra activity, especially in regards to question two,
students could be divided into groups and made to debate the sample topic.
Finally, the use of podcasts, made up primarily of benchmark responses
available from ETS could be made available for students to listen to and
download from the instructional website I create. I hope to make this
available from a blog. It would also be a good idea to make video podcasts of
my lectures available for students to view and download.

As stated above, students will be assigned into groups throughout the course
in order to evaluate each other’s responses. Additionally, students can be
given a list of commonly used vocabulary used when scoring/evaluating a
response. A drill –and- practice activity could be made from this that includes
the use of flash cards with definitions and most likely in combination with
either a simple matching type internet game or even something in the
classroom, like vocabulary bingo. One website that could possibly used to
make a vocabulary matching game for free is
https://egames.carsonmedia.com/ but I am also hoping to make some games
with Flash as well. Another activity that may be useful here comes from
Horton’s chapter four, “connect-type” activity, guided research. After
students have posted some sample responses to their blogs, they could be
led to their classmates’ blogs and asked to research the responses of other
students based on the criteria of: Pronunciation, intonation, pacing,
grammar, word choice and vocabulary, use of transition words and overall
topic development. Students could be assigned to find the response that
they feel best represents the highest level of pronunciation in the class. Then
they could cite examples of words that the chosen student is able to
pronounce that others may have difficulty with.

Another activity that would serve the students well socially is one that has
them competing in the classroom or an online chat session. This activity is
specifically related to my first learning objective. Since students are required
to think quickly and get their ideas on paper this activity can help them do
this in a simple and fun way. Students will be asked to come to the white
board at the front of the room. Or, if in a chat room, students get ready to
type. This game is an association game. I will say one word, for example,
“chocolate.” Then the two students have fifteen seconds to write as many
words that they can associate with chocolate (ie. Sundae, ice-cream, dark,
milk, hersheys, etc). Some word associations will be obvious while others will
be more of a stretch. Yet, the most beneficial part of this activity, and for
that matter the most fun part, will be having the students try to explain their
associations. The student with the most word associations in fifteen seconds
is the winner. By using key words from sample test questions, this activity
can directly help students become more confident with the types of
information they will need to mentally access quickly in order to answer
these questions.

One type of contextual activity that would serve as a good “Do activity”
would be a discovery activity that led learners to the ETS website,
www.ets.org/ and have them find and print certain types of information,
such as statistics about past Korean performance, registration information
and/or a variety of other types of information available there. Then students
could post an aha-type-learning to a group discussion. I consider this type of
activity as contextual because students who are taking the TOEFL test, really
have a need to visit the ETS website if they really want to understand what is
expected of them. Additionally, this type of activity could serve as a
research activity as well, where students are sent out to “discover and use
their own sources of information” (p. 168 Horton Chapter 4).The ETS website
may also serve as a good place to set up a scavenger hunt activity as well. In
the case of a scavenger hunt, students could be required to answer specific
questions about test content, how questions are created, fees involved,
registration, test locations, test preparation, getting scores, FAQs and more.

Having the students do sample iBT TOEFL question responses is one of the
most important types of activities that addresses the active learning value,
especially, if I can develop a way to mimic the timing based principles on my
website. Additionally, by having students develop their own educational
blog in which they will store all of their practice responses on, students will
be keeping a portfolio of their work. This again reinforces the Learner-
centered value because of the ownership that students will take in building
this blog but also serves the “Active” instructional value since students will
be creating their site and will be involved in the decision making involved in
producing the site.

Games will play a role in integrating the supportive value for sure. I need to
develop more games. This is easier to do with vocabulary than it is with
speaking. I look forward to developing a pronunciation game for students to
use. For now, one thing I do feel confident I can do is give students constant
audio feedback. The way I am doing this is my essentially recording mp3s of
myself making comments and/or corrections to students’ responses,
uploading those mp3s onto www.snapdrive.net, and then requiring students’
to upload the feedback to their blog. Perhaps the most time consuming but
supportive technique I have found success with has been having students
dictate what they said in their response, and then I correct their grammar
and word choice, post my corrections/comments to their blog comment
section and then record a revised edition of their response. Then, students
are required to listen to my recording, and sentence-by-sentence, record
themselves repeating my response word-for-word. By recording themselves
this way and then playing back the recording, students start to hear both, a
native speaker’s pronunciation, intonation, overall delivery. Furthermore,
students can then hear themselves using some of the same pronunciation,
intonation and delivery patterns as they playback these updated recordings.
I have found students to respond well to this since they can go back and
listen to their old speaking habits and compare the difference with the
revised versions. Lastly, I will have an English/Korean translator available on
each page of the blog tutorial so that students can quickly translate any
vocabulary that is new to them.

Design Rationale
I’m going to base the color scheme of my website on blues with touches of
brown. The main reason for using blue is because it is a calming color
and because the type of activities students will be doing while studying
for their TOEFL test tend to be high stress. Because students will be
recording themselves and posting it online, I feel a calming color will
help to keep student anxiety at a low level. Additionally, I feel that
brown will be a good color because is it somewhat of a warm color yet
not over-stimulating, and one that represents stability and encourages
trust. Brown is one that should help keep students grounded and help
them stay focused on improving their skills for the speaking test. Brown
is a practical color and I as such, an important color for students who
will be using the TOEFL test to get ahead in their lives. Lastly, I would
like to add touches of orange, even though I haven’t actually found this
color yet. The reason I think orange would be good for my site is for
contrast, in particular if/when I put games on the site. I would like to
develop games where students are forced to think fast, games that are
timed, possibly a game that will involve matching the pronunciation of
sounds with the corresponding spelling of those sounds. I feel that
orange will give them that touch of urgency that is needed to move at a
fast pace, when practicing for the TOEFL iBT test. The colors I have
chosen came from the Kuler website, which has the colors I mentioned
above.
Navigation Scheme: I have chosen to use a rich navigation scheme. One
rationale for this is because my students will be young, web-saavy, and
impatient. Additionally, my students will unfortunately have a varied
degree of previous TOEFL iBT knowledge. Because of this I plan to have
menus and indexes on each page making it possible for the more
advanced users to skim through information they already know and
delve into information of interest to them.

I plan to chunk information on my site by using blogger blog posts for each
of my six task tutorials and secondary menu links on the left column of
the site to display links for other resources. Both of these links will be on
each page of the site as constantly displayed menus, allowing users to
quickly and easily navigate throughout the site. This type of navigation
is somewhat limited by the constraints of blogger.com but once the site
is up, it should be relatively easy to transfer any and all of the
information to a new host. Mechanisms for navigation will include
scrolling through the tutorial posts and through links on the side of the
page. I suppose, mostly because it’s recommended by Horton, that I
should use paging as another mechanism for navigation. This may
actually be useful for novices who would like to simply move from
question one to question six of the TOEFL iBT test. There is no real
sequence of difficulty on the test which I will point out from the
beginning but it would be practical for students to move sequentially
from the first to the last question. I could set this up by having a small
bar at the bottom that reads, “back/next.”

The pictures most often used on the TOEFL test and in TOEFL preparation
books include those of professors lecturing, students talking, or buildings on
campus. These pictures can assist in educating students on which type of
question they are answering and it could help them remember the type of
information that they should include in their response. For example, in a
question 3 type of question, a picture of two students seemingly discussing
something they are reading in a newspaper article could serve as powerful
reminder that question three always involves two students discussing some
sort of university announcement and then requires the test taker to listen to
one of the student’s opinion’s about the article, and explain what the
student’s opinion is. I plan to use these types of pictures, those that
correspond to the question in both my tutorial and my sample test sections
of my edWeb. Likewise, dual coding will be an important part of this site as
much of the site will be narrated by me throughout, in particular for the note
taking sections in the tutorials. These will serve as representative visuals.
Additionally, free pdf files, available from the ETS website, will be available,
some of which have charts and graphs of regional student performance.
These files will be available in my resources link and serve as organizational
visuals. Additionally, I would like to add a picture/graphic of a map of some
sort simply as a decorative visual, to serve as a motivator to get moving.
Contrast: As far as contrast goes on my prototype, I have some variation in
font size that should help certain things stand out over others. This is
particularly true in the Flash tutorials created from Voicethread.com. The
contrast is not bold, and in fact may be somewhat wimpy for what Williams
suggests, but I feel that going from size 18, bold font to a regular 14 font is
sufficient when going from a heading to my text. I feel like the title of my site
stands out, as it sits a 26 size font, yet it’s not overwhelming. And especially
because of the docile brownish color, it grabs attention but grounds the
viewer, and pushes them downward to read on. The contrast in color from
the dark colored menus on the top and side to the lighter blue colored text
box in the middle of the page allows the content of each individual page to
stand out while the other elements take on more of a background role when
not being used.

Alignment: I feel that this design has a strong alignment element. All edges
are aligned with each other. My main (top) bar is aligned with my side bar,
which encapsulates the text in the middle of the page. While there are no
breaks in alignment at the moment, if and when I make one, I think it will
look intentional and appropriate.

Repetition: I have consistent repetition throughout my protocol. When a user
clicks on a particular lesson, bullets pop up under the lesson title on the side
bar, while the main page loads up in the middle of the page with all subtitles
listed accordingly. This consistency brings cohesiveness to the site and will
make the viewer/participant comfortable as they will know what to expect as
they are navigating through the tutorial. Also, the use of one single font,
Garamond and Garamond bold is repeated throughout, as to not distract
from the material being presented. Again, one of the goals of my design is to
help keep my participants calm yet focused on the work. Having the site be
repetitive adds to this goal.

Proximity: I feel like the content on my page has been condensed
appropriately so as not to confuse the viewer. The alignment of the menus
frame the content in the middle of the page helps define the relationship of
the content.

Formative Evaluation Plan
Questions to be answered:

Will learners like the color scheme I’ve selected?

Are my Flash LOs effectively addressing the learning objective(s) they
aim to?
Will my learners be able to successfully navigate my EdWeb Web site?

Participants:

The first two participants (listed below) involved in my formative
evaluation are students of mine who have studied for the TOEFL Speaking
test with me using methods more traditional than those presented in the
formative evaluation. The third participant in my evaluation was a colleague
of mine who also teaches the TOEFL Speaking test and one that showed
interest in using technology in the classroom.

Oh, Se-Young: A middle school student who has studied for the TOEFL
Speaking test for nearly one year with myself and with other native-English-
speaking teachers. He is a self-motivated 13 year old that doesn’t require a
lot of discipline during class.

Park, Jin-Young: Also a middle school student who has studied for the
TOEFL test for nearly a year. She is a 13 year old who has a good control
over the English language and is a good speaker; however, she is easily
distracted during class and is much different than Se-Young in her approach
to learning.

Lee, Heidi: Heidi has been teaching the TOEFL Speaking test for nearly
one full year now and has incorporated the use of blogs and recording
software in her classes. She is a 30-year-old and dedicated to the constant
improvement of her students.

Process:

I actually started by having Heidi evaluate my work first. Before even
showing her anything, I explained to her what I am planning. The main
reason I started by having her evaluate my edweb first was because, of the
people I planned on having evaluate my work, she was the most accessible.
However, another reason I started with Heidi was because she is interested
in learning more and teaching about the TOEFL speaking test. Additionally, I
trust her to give me feedback that is as objective as possible because she is
a professional. After gathering data from Heidi, I took my laptop computer
and went into the classroom with it for my first run through of data
collection. I asked Jin-Young and Se-Young to take a look at my first Flash LO
storyboard and provide me with feedback. They obliged me and gave me
their thoughts on what was presented to them. I played them the audio
scripts while showing them both my storyboard for my Flash LO and also the
revised Flash Quiz (at least the part that worked). I took notes on what they
said while gathering as much information as I could from them. Originally, I
had planned to record my students’ feedback as to use it to replay later.
However, instead I just ended up taking notes on what they said. I was
hoping to use a screen capture program called Jing, to record the students
interacting with what I had on the screen. But, I was unable to provide them
with much to interact with, specifically with the Flash LO that I was much
hoping to have done by the time I got to the formative evaluation. Still, as I
said, I was able to gather some good feedback from them and get my basic
questions answered enough to make my formative evaluation worthwhile.

Unexpected Problems:

One issue was obviously in not having a lot of material for my evaluators to
evaluate. I was able to get valuable feedback on what was presented but
having not put as much together as I would have liked, I think brought some
concern as to how effective my formative evaluation is and will be in this
case. I am hoping that by the next time I get around to creating an
educational website, I’ll be able to put together more information for my
evaluators. One issue I had to work around was finding times that
coordinated between my student’s schedule and my own. I thought that it
would be enough to offer the students some snacks and refreshments in
order to get them motivated to meet but other priorities made getting them
when I was free a little difficult. Still, we were able to find two times for
about 45 minutes where we were able to meet. One issue that came up that
wasn’t so surprising, after the fact, was having the students be somewhat
reluctant to give me criticism. The students are only middle school students
and I am sure there are no other times when a teacher would ask them to
critique his or her work. I attribute this to the fact that the students are
Korean and culturally it considered “out of line” if a student were to critique
his teacher. In spite of this, of course with a little coaxing and
encouragement, both students were able to overcome this cultural barrier
and provide me with some valuable feedback. There really were no issues
that came up while gathering information from Heidi. I was able to talk to her
without appointment when I was free and she was able to offer solid
feedback on what she liked and didn’t like.

Data Collected and Implications:

From Heidi: The first time I approached Heidi in regards to getting her
feedback on my work she accepted and became curious about the project.
After explaining to her my hopes of making a TOEFL speaking tutorial, her
immediate response provided valuable feedback. She said, “Aren’t there
already a lot of other websites out there that do that?” Of course I have
thought about this before but haven’t found a whole lot in the way of useful
sites to bounce my ideas from, at least not in the way of TOEFL speaking
tutorials. Anyway, this prompted me ask her to inform me of some that she
has come by. She provided me with a few good sites to review including:
www.toeflblog.com , http://goodlucktoefl.com/ , and a weekly podcast
channel called http://privateenglishportal.com . By looking over these
websites, I was better able to see what was out there. Also, by evaluating
these other sites, it helped me see what works and what doesn’t and what’s
interesting and what isn’t. The private English portal site basically just has a
guy that does English tutorial podcasts each week, in an effort to answer
student questions. What a great idea. I’ve thought about offering my own
feedback to my students’ responses in writing and even in recorded form but
for some reason, I never considered using video podcasting as a way to give
them feedback. One of the things I liked about this guy’s podcasts was that
he had subtitles written on the screen for everything he said. That seems
very time consuming but for students trying to make the connection
between reading, listening, writing and speaking, having a video podcast
with subtitles could certainly help. I like this idea and I will consider adding
this as part of my edweb.

Some other things that Heidi pointed out were: There isn’t enough student
interaction provided for while students are watching the tutorials. She said
that it would be better to have students be able to record themselves as
soon as they get done watching the tutorials. By limiting student interaction
to just pressing a pause, play, or replay button, students may find the
tutorials boring. This is one of my concerns with my edweb as well.
Originally, I had hoped to find a way to develop an edweb that would simply
allow me to enter sample test questions into a template, have the students
listen and read the questions, hear a beep, then begin recording directly
onto my website, which would store their recordings. However, I think this is
outside the scope of what Flash can do. Moreover, this is also outside the
scope of my own abilities using Flash. Still, this is something to consider
strongly in the future. I think I could at some point, get funding to have
someone actually build this type of interaction into the site. For now, my
students will be reading sample questions and posting responses to their
personal blogs.

She also mentioned that because I’m trying to design a TOEFL tutorial, I’m
going to be competing with a number of other professionals in the industry.
She offered that I may be better off developing a different twist, a sort of
creative angle to teaching the TOEFL. We never got really far with this one. I
mean there are hundreds of thousands of things that I could do with my
edweb, millions of creative twists and turns to make. But, that’s not to say
this was bad feedback by any means. Part of what I could do is put a game
or two into my edweb. I haven’t yet thought of any games that may go
along with TOEFL but one thing I could try is adapting some of the phrases
associated with high level English responses into some sort of matching
game. I’m not very creative so I’ll need to explore this much further to make
it a viable possibility and not just a distraction.

She went on to say that content should be tailored more to the student’s
abilities, not just the abilities required of them to take the test. This is
something I’ve been thinking a lot about. In teaching TOEFL, it is important
not to baby the students. What I mean is that, if I’m going to prepare
students for the TOEFL test, I need to get them understanding how stressful
it can be. The test is not only testing their English ability but it is testing their
English ability in a limited amount of time. For example, to answer question
one, students get 15 seconds to take notes and then they hear a beep and
the recording starts. Whether they are ready or not, the recording starts.
Having scored the test for ETS before, it was always heartbreaking to hear
students get to the recording and not say anything. My point is that if I’m
going to prepare students for the test, I need to prepare them to be timed. I
need to prepare them for the stress that can come with the test. However,
Heidi made me think. How can I tailor this subject matter to a student’s
ability yet still help them understand the importance of being timed? In the
classroom, I’ve used stopwatches, had students stand up in front of the
class, recorded their responses, played them back, all in an attempt to
introduce stress. After thinking about Heidi’s comment one method I thought
of was to have students write a response in 4 minutes. In four minutes
students in my class can write about 80-90 words. This is the roughly the
same number of words that would be spoken in a one minute, oral response
typical of a TOEFL speaking question. I like this activity because it
introduces the stress of being timed, but takes away the stress of being
recorded. After the students finish writing, I take their papers and I read
them out loud. In doing so, I point out grammar or word choice mistakes
along the way. For me, it’s a way to interact with the students and introduce
them what it feels like to be timed. For them, it’s a less stressful way of
learning the timing. It’s also a way for them to hear their thoughts and ideas
spoken fluently in English, which applies to our core learning value
relevance.

My two students evaluated my work and let me know that it needed music if
it were going to get their attention. They said, using a pop song or some
type of catchy tune, perhaps in the background of me narrating would bring
some excitement to the tutorial. I am not sure if this would actually be
something I would like to do because I think it will only serve to be a
distraction for students who are new at studying for the TOEFL test.
However, I do think that if/when I can get a game in place, some background
music will be very appropriate.

The students also said that it would be helpful if I talked slower while I’m
narrating. I did pay attention to this while I was recording but I can certainly
understand the need to hear me speak slowly and I think there could be two
solutions for this. One would be to re-record at a slower pace. The other
would be to possibly add subtitles to the flash tutorials.
All students agreed that the color scheme was adequate and Heidi felt the
same way. However, the students did say that the entire layout was a bit
boring because each screen of the tutorials is roughly the same. Again, I
think this was done intentionally and I don’t feel that I need to change this
part of the edweb.

In regards to content of tutorials, the students said they would like to hear
more examples of how a high level speaker speaks. This is a smart request
and I will work at getting a full response to each sample question along with
subtitles that the students can click on and hopefully download via podcast.
The samples will come from me for now but I can surely get free help from
other teachers answering questions as well to give students a variety of
voices to hear, again in hopes of curbing some of the monotony of hearing
only me.

Lessons Learned:

I think the first lesson learned is something that I’ve heard before from our
instructors: a website is never actually finished. I can see that since the very
early stages of me planning this edweb until the day I put it up for use I will
constantly be making changes to it. With this in mind, I think I will set out
with a little more focus on a set number of questions. While I did write out
my questions and they were specific, I didn’t feel like they were actually
enough to really gather enough feedback. I felt like I had to keep digging to
get more information, particularly from the students. I guess what I’m
saying is that there weren’t enough questions that I prepared before hand to
get all of the help on the evaluation that I needed. I think it would have been
smart to do a little more work up front in developing the questions I wanted
answered. I think when I chose my questions for the formative evaluation
last semester I used somewhat generic questions. These questions certainly
served as a good guide to get things started but they were, by no means, the
end all of my line of questioning. Next time I would be smart to brainstorm
more about what questions I need answered.

Other than that, I think what I’ve learned from this process is that it is hard
to do a formative evaluation on a website that is not yet built and running.
While I can certainly see the benefits of a formative evaluation being cost
effective and saving time and energy in development, I also think there is a
lot of skill involved in going about this. This was the first time I’ve tried a
formative evaluation and I do feel that the more I do them, the better I’ll get
at them. Still, it’s sort of a balance on how much work to put into a pre-
formative-evaluation website vs. a post-formative-evaluation website. All-in-
all, having completed this my first formative evaluation, I am a step closer to
finding that balance.
Summative Evaluation
The purpose of my summative evaluation will be to gather data and
feedback on the level of satisfaction students had after taking this six week
course. The evaluation will be created using the free version of Survey
Monkey and will 10 questions long. I made sure to allow students to post
comments on any or all of the questions asked so as to try and gather as
much information as possible to make adjustments to the edweb. Students
will have access to the survey at any time during the class- but will be
required to fill it out only after the final week of the course. The question will
read as follows:
1. On a scale of one to 5 (1 is the lowest and 5 is highest), how
helpful were the six tutorials to you understanding the
TOEFL iBT Speaking section?

2. What was the most beneficial part of this course?

3. How difficult was it to start a blog using blogger.com?

4. On a scale of 1-5 (1 is the lowest and 5 is the highest), how
well prepared are you to answer the 6 questions in the
TOEFL iBT Speaking section?

5. Which question do you feel is the easiest to answer?

6. Which Question do you feel is the most difficult to answer?

7. What was the most helpful part of the online portion of your
course?

8. When did you start studying for the TOEFL iBT Speaking
section and when will you take the actual test? *Note: If
you don't know when you will take the test fill in
01/01/2050.

9. Did you find the rubric helpful in understanding how you are
scored on the Speaking Section of test?

10. Describe the overall experience you've had in taking this
course.