Click to edit Master title style

The Summary of Chapter 4

Psychometric Approaches
to Fairness: Bias and DIF

T. McNamara, T. & C. Roever, 2006, entitled Language testing: The social

dimension. Malden, M.A.: Blackwell Publishing

1
Click to edit Master title style
Bias and Differential Item Functioning.

“Bias” in assessment research conveys a skewed

and unfair inclination toward one side (group,
population).

2 2
Click to edit Master title style
Bias and Differential Item Functioning.

DIF (Differential Item Functioning) identifies

test items that function differently for two
groups of test takers.

3 3
Click to edit Master title style
Bias and Differential Item Functioning.

In most general DIF studies, gender and

ethnicity were the factors under investigation
and it is important to remember that DIF
findings are based on large groups and can
only be generalized to large groups, not
specific individuals.

4 4
Methods
Click to editfor Detecting
Master title styleDIF

Methods used for detecting DIF fall into four

categories:
1. Analyses based on item difficulty.
2. Nonparametric approaches.
3. Item-response-theory-based approaches.
4. Other approaches (include logistic
regression)

5 5
Click to
Early DIFedit Master title
Techniques: style Item Difficulties for
Different
Different Groups.
• The delta plot was used in some earlier DIF
studies in language testing, but finally the
delta plot (TID) method does not match test
takers by ability and does not take item
discrimination into account.

6 6
Click to edit Master
Contingency title style
Table Methods and Other Nonparametric
Approaches
• Contingency table methods have the advantage that
they do not require in-depth understanding of the
complex mathematics of item response theory. A 2X2
table is created that is used to compute the relative
odds of a correct response for the reference group and
the focal group.

7 7
Click Difficulty
Item to edit Master title style
and Matched Groups: The Standardization
Procedure
• It’s another nonparametric approach and it’s
also known as the conditional p-value. The
standardization procedure compares the
proportion of test takers who answered an
item correctly for the reference and focal
groups at each score level.

8 8
Clickand
DIF to edit Master titleTheory
Item-Response style

• Item-response theory (IRT) is an approach to

DIF analysis radically different from the simple
comparison of difficulty levels. It facilitates the
matching of test takers on ability.

9 9
Clickand
DIF to edit Master titleTheory
Item-Response style

Item-response theory is radically different from

classical test theory because it describes the
behavior of an item, rather than the behavior of
a population of test takers.

1010
Click to edit
Problems withMaster title style
DIF Analyses

1. They are quite complex, not readily done with

standard spreadsheet or statistical software,
and not easy to interpret.
2. Categorization used in most studies are
rough and not theoretically motivated.

1111
Conclusion
Click to edit Master title style

Fairness plays an important role in traditional

psychometric work on testing, and a variety
of approaches have been developed to
detect unfair items and investigate unwanted
influences of test-taker background factors.

1212
Click to edit Master title style

Thank You

13