Ardee Jhel Joy S.

Enriquez
Sheela Mae V. Salingay
BSE-ENG
LANGUAGE AND LITERATURE ASSESSMENT
MODULE VI- QUALITATIVE ANALYSIS
Categories and Approaches:
1. Reflection
- Aim at getting an insight into the thinking processes and opinion of the test
taker
2. Verbal Reports
- Offer an insight into the thought process of the informants.
- A number of variables can be distinguished:
o Talk aloud- informants voice their thoughts
o Think aloud- voice their thoughts as well as other information such as
physical movements
o Concurrent- The verbal report is given in real time
o Retrospective- the given report is given afterwards
o Mediated- researcher occassionaly intervenes
o Non-mediated- the researcher does not intervene some pointers when
using verbal reports in test analysis
3. Diary Studies
- Informants keep a diary which allows researchers to get an insight into their
thoughts

TECHNIQUES IN MONITORING STUDENT PROGRESS

1. Make a video and audio recordings of a variety of formal and informal Language
informants
2. Use Checklist as concisemethods of collecting information and rating scales
3. Record anecdotal comments to provide useful data
4. Interview students to determine what they believe they do well or in areas they need
improvement
5. Have students keep portfolios of their dated writing sample and language abilities ,
checklist and records
6. Keep anecdotal records of students reading and writing activities
7. Have students write in reader response journals
8. Confer with students during the writing and reading processes
Self-assessment

Promotes students abilities to assume more responsibility to identify where

they have been successful and where they believe they require assistance.

Peer assessment
-

Allows students to collaborate and learn from others.

TYPES OF QUALITATIVE ANALYSIS

1. Conversation Analysis (CA)
The assumption is that every interaction is contextual and has a stable and
predictable nature.
a. From a transcript, the power relations can be researched by analyzing
The structure of adjacency pairs
Turn-taking
b. Some possible pitfalls to look out for with CA
Data quality
Loss of data
Time
Limited generalizability
2. Discourse analysis (DA)
Da is the analysis of text and talk as social practices and is mainly concerned
with power relations gender inequalities etc.
The effect of examiner
The effect test taker
The effect of task type
Comparing task taker language ability
3. Test language analysis
An analysis of test input or test taker responses for lexical richness, rhetoric,
genre, discourse markers, grammatical complexity, etc.
When using this type of sample analysis, pay attention to:
Time
Data

4. Task characteristics
This type of validation research helps to examine the test tasks and to determine
to which extent they correspond to the test goal.

Bachman & Palmer (1996) suggest a framework of analysis which considers the
ff:
The setting
Rubric
Test input
Expected response
Relationship between expected and actual response
When analyzing test for its task characteristics, consider the following:
-

The framework should be adjusted to each different test.

The judges have to be competent and experienced.

FEEDBACK METHODS
1. Questionnaires
Questionnaires gather data such as opinions and views that can also be gathered
through interviews.
Two kinds of questionnaires:
Closed
Open
Before administering the actual questionnaire, it is useful to run it through the ff.
process:

Consider all possible issues

Write a draft
Eliminate questions that do not address the questionnaire purpose
Group the questions thematically
Format the questionnaire and administer it to a small group of target
respondents for feedback.
Rewrite the questionnaire

Avoid:

Double-barreled questions
Unclear instructions
Questions that do not apply to the respondents
Questions that rely on memory
Hypothetical questions
Biased options

Checklists are a way of determining whether all procedures have gone through,
whether all necessary features are present, etc.
2. Interviews
- They are a flexible of gathering data. There are various kinds of interview
depending on the structure and the number of informants interviewed at the
same time:
1. Unstructured
2. Semi structured
3. Structured
4. One on one
5. Group
REFERENCING
Think about the following before the interview:
a. Interviewers should get the chance to practice their interview skills prior to the
data collection
b. Ideally, the pilot settings resemble the actual conditions as accurately as possible.
c. During the interview, its useful to take note of the interview situation
d. The success of an interview largely depends on the interviewer responded
interaction
e. Interviews are time consuming
Various kinds of referencing:

Norm-referencing
Criterion-referencing
The mastery criterion-referencing approach
The continuum criterion-referencing approach