Module Evaluation 1

Running head: EVALUATING THE MODULE FOR LEARNING SIMPLE PAST TENSE IN
URDU

Evaluating the Module for Learning The Simple Past Tense in Urdu

Nina Gill and Zia Ahmed

California State University – Monterey Bay

IST622 Assessment and Evaluation

Dr. Su

May 30, 2018

 Module Evaluation 2

Table of Contents
I. Introduction-----------------------------------------------------------------------------------------3
II. Methodology--------------------------------------------------------------------------------------3
a. Prototype-----------------------------------------------------------------------------------------3
b. Learners------------------------------------------------------------------------------------------5
c. Tryout Process-------------------------------------------------------------------------------6
1) Pre-Test---------------------------------------------------------------------------------6
2)Post-Test-----------------------------------------------------------------------------------7
3) Observation-------------------------------------------------------------------------------7
4) Survey-------------------------------------------------------------------------------------8
d. Tryout Conditions------------------------------------------------------------------------------8
III. Results---------------------------------------------------------------------------------------------8
a. Entry conditions--------------------------------------------------------------------------------9
b. Instruction---------------------------------------------------------------------------------------9
c. Outcomes----------------------------------------------------------------------------------------9
d. Recommendations----------------------------------------------------------------------------11
IV. Summary and Conclusion---------------------------------------------------------------------12
V. Appendices---------------------------------------------------------------------------------------12
A. Pre-test activity-------------------------------------------------------------------------------13
B. Post-test activity--------------------------------------------------------------------------13
C. Pre- and Post-Test Grading Rubric------------------------------------------------------14
D. Pre- and Post-Test Scores----------------------------------------------------------------14
E. T-test Paired Two Sample for Means---------------------------------------------------15
F. Survey & Response Summary-----------------------------------------------------------15
G. Observation Checklist--------------------------------------------------------------------16
 Module Evaluation 3

I. Introduction

This training would be given as tryout to check a part of the learning module which will

be the future capstone project. The tryout will be given to semester one Urdu students of the

Defense Language Institute Foreign Language Centre (DLIFLC) in Monterey California. Urdu is

a difficult language to learn and students only have forty seven weeks to master this language

and become proficient in listening, reading and speaking the Language. It has been over ten

years that the Urdu Department is struggling to achieve their target level score in the speaking

skill. Learning the Past Tense narration is the most challenging grammar feature for almost every

student till the third semester and that is the reason we see a need to fill the gap with this

training. The training can be used by anyone outside of DLI as well if they wish to learn using

the simple past tense in Urdu.

This tryout will help us to assess the following components of the training by means of

observation, pre and post -test and survey questioners:

1. Are the objectives clear and specific?

2. Are the instructions to the questions clear and easy to follow?

3. Do all the tabs work?

4. Is the content interesting and engaging?

5. Does the module give enough content knowledge and practice to perform the post- test.

II. Methodology

a. Prototype

The prototype used for evaluation at this point in the course is a part of the capstone that
 Module Evaluation 4

will be presented at the end of the course. This training is used to teach the students verb

conjugation in the simple past tense with some common Urdu verbs. This will help the

students to use simple past tense in their speaking more fluently. The trial module will

explain how to conjugate some of the common Transitive Urdu Verbs in simple past

tense. Instructions and information about how to maneuverer the training is given in the

beginning of the training. The main verb that will be used in the sentences to explain the

different constructions is “To Eat”. Since the main verb that shows verb conjugation in

different forms is “To Eat”, students will see different popular restaurants of the Target

Country “Pakistan” in the background. This background adds a cultural element to the

module and is the strength of the module. Adding cultural pictures of restaurants and food

items to the training keeps the students engaged in the training and exposes them to

sociocultural aspect of the language they are learning. DLIFLC believes in sociocultural

linguistics and professors emphasize this aspect to a great extent while teaching the target

language. Another strength of this training is keeping the training personalized and

simple and providing them with the correct feedback on the spot if the answer is

incorrect. At this point of Romanized Urdu is used to make it easy even for the students

who have not learned the sound and script as well. The weakness at this point which will

be improved later on, before the final capstone is the quality of recording. Booking a

professional room for recording which we have at DLI will be helpful and will be used

before submitting the final capstone. As the training begins the students and the

instructors will hear and read the instructions on the slides as they go through the

training. The training begins with the explanation of the Transitive Verb and it’s

conjugation in the simple past tense. Examples of some common verbs are given to
 Module Evaluation 5

explain further. The students will read and hear what is the verb “To Eat” in Urdu and

different forms of its conjugation. After learning the content, the students will attempt

multiple choice questions, fill in the blank, and matching an exercise for practice, to

check their understanding. Once the quiz is finished the score and review for the quiz is

provided and a certificate of completion.

The Captivate module since it is a tryout runs approximately around ten minutes with

students attempting the questions or pausing it for more understanding. But this time will

be increased with the inclusion of more practice activities. Including the pre-and post-test

and if learners do every exercise. An average learner is expected to complete the module

in about 40-50 minutes.

b. Learners

The target audience of the capstone project are the US Armed Forces Military Linguists

who have just completed booth camp and an officers. Most of the students are high

school graduates but sometimes there are senior ranking officers and university graduates

also. These linguist before entering the Urdu program go through a four day refresher of

basic English grammar, language learning strategies and cultural information of the

Target Country. All are native speakers of English and some of them have already learned

a different language. Once they enter the Urdu program, in their first four weeks they are

taught the sound and script. The average age of the students ranges from 18 to 30 years of

age. The distribution of male and female most of the time is fifty percent each. They have

good computer skills as most of them are millennials. These students go through an

extensive medical exam and have no disabilities. They are also given training in stress

management and are provided with several resources where they can seek help in case of
 Module Evaluation 6

stress and anxiety.

For this tryout, in the training the Urdu words are Romanized and they can hear the

pronunciation as the narrator speaks. The test subjects do need to be native speaker of

English, have basic computer skills, they are required to know some basic Urdu verbs and

pronouns for this training. Age requirement is 18 years and above.

We chose seven students from the Urdu first semester class in the DLIFLC. Six male

students were Airman Basic from United States Airforce their average age was from 18 to

26 and one female Unites States Army officer 25 years of age. They did not have any

listening and visual impairments and no other disability. They were all native speakers of

English and had advance computer skills. They also possessed some basic knowledge of

Urdu verbs and pronouns.

c)Tryout and Evaluation Process

The process of evaluation consisted of ;

1) Pre-tests was given to check their prior knowledge of the Simple Past Tense in

Urdu.

2) Post-test was given to check how much they have learned after doing the module.

3) Observation was done to check how well the training went in terms of technicality,

completion by the students and their overall experience and engagement in the

training.

4) Survey in google doc to measure specific feedback from the students.

1.Pre-tests:

Pre-test was prepared keeping in mind that students were refreshed on their basic

knowledge of English grammar as they come to DLIFLC. Before they join the
 Module Evaluation 7

Urdu Department they go through the short training period in the Student Learning

Centre (SLC)at the DLIFLC. Over here they are also provided with some basic

Urdu words and phrases which they can read in Romanized Urdu and they are

basically just memorizing them. These students are also familiarized with some

basic verbs and pronouns in Urdu.

For the pretest ten sentences are given and the students will mark them true or

false. It is basically prepared to check their basic knowledge on the types of verbs,

some basic Urdu verbs and their meaning which they were provided in the (SLC).

2.Post-test:

For Post-test ten sentences to translate from English to Urdu are given. This is

given to check if the students understood the concept of what Transitive verbs are

and how to make sentences in the simple past tense in Urdu. There are some

questions which are used to check the conjugation of verbs if the object is singular,

plural, masculine or feminine.

3.Observation

Students were contacted and scheduled one week prior to the day they were

supposed to take the tryout. A lab was reserved at the DLIFLC for one hour so that

students take the tryout training module. We made sure that the seating

arrangement was comfortable for the students, internet was available and there

were seven instructors to observe the students one on one. The students were

encouraged to take the training tryout independently focusing on the training

module and not to be worried about how well they perform. Most of them did it

with ease with no technical issues.

 Module Evaluation 8

The main thing that the instructors were observing are following (appendix G):

1. Did the students understand and follow the instructions?

2. Did they run into any technical issues?

3. Were they clear on the learning objectives.

4. Did they go through the training completely and not skip any slide or exercise.

The entire training including the pre and post-test was finished in 40-50 minutes. It

was made clear that this training is a prototype to get their feedback which will

help the developer a great deal. The students were also requested to complete the

training but no one was pressured to do anything. Help was provided if it was

requested by any student.

5. Survey

Google Survey Form was used to get the feedback from students. This form was

used to avoid bias of opinion.(appendix F). This survey addressed if the students

enjoyed the training, were the learning objectives and instruction clear and to the

point. Did the students get enough practice to actually produce that they were

expected to.

d. Tryout Conditions

The students and observers were instructed to report to the Lab at one in the afternoon

on June 1st 2018. They were told to bring their Laptops fully charged along with their

chargers. It was made sure that the lab was booked for this tryout with clean comfortable

chairs and tables enough for the students and instructors who were observing. Students

were instructed to bring their headphones. Link was given to access the training.

III. Results
 Module Evaluation 9

a. Entry Conditions

Entry conditions for the students to take the given module were good. The students did

not have the knowledge of using simple past tense in Urdu. The students were aware of

some basic Urdu Verbs and Pronouns had knowledge of using Simple Past tense in

English with Transitive Verbs. Being millennials they had good computer skills so they

could easily maneuver the training module. The students did not have prior knowledge of

using simple past tense in Urdu.

b. Instruction

The learners were clear on how to maneuver the training, they had no problem in doing

the training. In fact they liked it, as it was different from the traditional lecture. Students

were not clear on the multiple choice question and did not answer it correctly and said

that the question should be more specific. One students rushed with training and got two

questions wrong. He was explained not to worry about the score. The learners finished

the module and had no problem in doing the activities. The gap observed by the observers

was that the students had some confusion in answering the multiple choice questions.

And the instruction had to be made more clear for them. In the multiple choice question

they wanted it to be more specific and to mention the Transitive Verb in it. (see appendix

G, Observation Checklist). Over all they liked it but gave feedback to improve it and

make it more enjoyable.

c. Outcome

Analysis of pre- and post-test results (see appendix) indicates that learners achieved

significant learning as they went through the training and test taking. Learners’ pre- and

post-test results were graded based on the grading rubric (appendix) and then paired two
 Module Evaluation 10

sample t-test to see that the training contributed to the students learning skills. The

absolute value of t-stat was significantly greater than the t critical value (|t|=13.747 >

9.210) and it was concluded that the instructional training that was given to the students

resulted in learning gains. The training that the student went through was comprehensive

and made impact the pre and post-test. The pre and post-test was relatively of more or

less same complexity level. The pre and post-test were made carefully as not to test any

concept that is not covered in the training. So, the contents used for the pre- and post-tests

were developed carefully to ensure they were at the same difficulty level, and we as

instructors understand that learning gap in even though it may not be introduced to

enhance complexity of the material, it could lead to improved mean scores from pre-test

to post-test (64 to 79).

While learners considered training to enhance their understanding of the simple past

tense structure in Urdu. They also enjoyed the training as being effective. Learners did

mention need for clearer voice recording for instructions and training.

The survey given to the students showed that 87% of the students agreed that the

objectives of the training were clear and 100% of the students agreed that the training

was easy to maneuver. All students wanted more practice as the survey showed 100%

disagreed that there was enough practice. 50 to 56 % of the students wanted the

instructions on the question to be more clear and specific. There was an equal split of

between the students regarding the enjoyability of the training so 42.9% wanted

improvement in terms of enjoyability and 42.9% thought it was enjoyable, 14.3 % were

neutral. Based on the data we will work to make the training more enjoyable and

implement the suggestions in the training.

 Module Evaluation 11

d. Recommendations

Based on the findings, the following are the recommendations:

Including more multimedia elements:

The students suggested that the recorded voice does not sound enthusiastic rather sad and

monotonous. More pictures of food items or including a video will make it more

enjoyable. As most students taking this training module are very energetic and young

they wanted some good music playing in the background, because currently there are just

two pictures of food items and no music in the background. They also suggested the

avatar could be a cartoon figure that they can relate to such as “Sponge Bob”.

Inclusion of a variety of practice items at varying level of difficulty.

The Module has four practice items, two of them are Multiple choice questions, one item

is fill in the blank and one activity is to match.

The students suggested that they would like to have more multiple choice questions,

using them in a variety of different verbs and their forms will help them to become more

proficient in using the language. The multiple choice questions are also a part of their

final external exam so it gives them more practice. Students also suggested that they

would like to see more challenging questions.

Provide specific instruction on the activities that are embedded in the module:

Confirmed by both observation and survey responses, students experienced confusion

in attempting the multiple choice question. All students suggested to make the questions

and statements in the embedded activities more specific and to use Transitive Verbs in

them. They mentioned that their answers were based on the assumption that it could be

Transitive or Intransitive verbs.

 Module Evaluation 12

IV. Summary and Conclusion

Analysis of pre-& post-tests, observations and surveys support that not only learners

demonstrated significant learning gains. The students demonstrated a very positive impact of the

training in terms of its instruction and content. The feedback from the survey reported quite a

few limitations, but students did benefit from the content of the training and the way it was

delivered. The reason being the Urdu curriculum is very typical paper book style and is very

teacher centered with many errors and typos. The students are very frustrated with the quality of

the current curriculum and enjoyed doing something which was more engaging and interactive.

Although there had many suggestion to make it better in terms of enjoyability and getting more

practice but the students were satisfied with content of the training and the way it was explained.

Students actually got a good understanding of how to use Transitive Verbs in Urdu while using

Simple Past Tense. Based on the feedback of the students by means of survey and observation,

the narration will be rerecorded with some background music, more multiple choice questions

will be incorporated, the instructions will be made more specific, more multimedia effects can be

included to make the training more dynamic so the students could enjoy the training more.

The tryout process was a great learning experience, the shortcomings pointed out by the

students were very helpful, these were just overlooked while developing the training and were

not something major. It also teaches us how important it is to know your learners and their

background while we develop these trainings.

V. Appendices

A. Pre-test activity

Pre-test for the Simple Past Tense in Urdu

Mark if the following are true or false.
 Module Evaluation 13

Transitive verbs require one or more objects.

Nay follows the subject in simple past tense in Urdu.

Transitive verb conjugation in the simple past tense varies with the number of object
in Urdu.

In simple past tense in Urdu if the verb is transitive it conjugates with the object.

Transitive verb conjugation in the simple past tense varies with the gender of object in
Urdu.

Main nay kebab khaya. ( I ate a kebab)

Main nay teen kebab khayai. ( I ate three kebabs)

Main nay roti khaiee. ( I ate a roti)

Main nay teen rotiyaan khaieen. ( I ate three roties)

Main nay pani piya. ( I drank water)

B. Post-test activity

Post-test-Simple Past Using Transitive Verbs

Translate the Present Tense (PT) into Simple Past (SP)
PT I drink coffee. (Main coffee peeta huun)
SP
PT I eat roti. (Main roti khata huun)
SP
PT I eat roties. (Main rotiyaan khata huun)
SP
PT I eat fish. (Main machli khata huun)
 Module Evaluation 14

SP
PT I eat mangoes. ( Main aam khata huun)
SP
PT I drink tea. (Main chai peeta huun)
SP
PT I eat chicken. (Main murgi khata huun)
SP
PT I eat apples. (Main saib khata huun)
SP
PT Babar drinks coffee. (Baber coffee peeta hay)
SP
PT I eat Breakfast. (Main nashta kerta huun)
SP

C. Pre- and Post-Test Grading Rubric

Category
Points Total Score
Pre- 5 Points Each (10 Points x 10
True and False 100
Test Items)
Post- Translate Simple Past into 10 Points (10 Points x 10
100
Test Urdu Items)
D. Pre- and Post-Test Scores

E. T-test Paired Two Sample for Means

 Module Evaluation 15

Variable 1 Variable 2
Mean 63.57142857 78.57142857
Variance 47.61904762 64.28571429
Observations 7 7
Pearson Correlation 0.935970975
Hypothesized Mean Difference 0
df 6
t Stat -13.74772708
P(T<=t) one-tail 4.60517E-06
t Critical one-tail 1.943180281
P(T<=t) two-tail 9.21034E-06
t Critical two-tail 2.446911851

F. Questioner and Survey Responses

Module Evaluation 16
 Module Evaluation 17

G. Observation Checklist

Checklist Yes No Comments

Did the students easily access the training?
Did the learner read and understand the
learning objectives?
Did the learner follow the instruction
correctly?
Did the learner complete each and every
item in the module?
Did the learner have any problem, but
managed to solve it on his/her own without
developer’s intervention?
Did the learner encounter any problem, and
need developer’s intervention to be able to
continue?
 Module Evaluation 18

Did the learner navigate freely without

much problem?