CONTENT VALIDITY RATIO AND INDEX

A content validity index is obtained for each item. The experts independently answer
the following questions about each item:
 Is this item directly related to job performance? yes or no
 To what extent is the content of this item essential to job performance?
Definitely essential
Maybe essential
Useful, not essential
Unnecessary
 What is the quality of the item in terms of the stem, the options, and the key?
Excellent quality
Good quality
Fair quality
Poor quality

Consensus is not required, so the experts do not have to be brought together. Item
security should be a concern if experts will not be rating the items in a secure setting.

Once the data has been collected, you will need to examine the content expert judgment
for each item. A sample table of data for Expert 1 is shown below.

Expert 1 Ratings
Item Linked Essential to job performance Quality of Item
Useful, not
Yes No Not Maybe Definitely Poor Fair Good Excellent
essential
1 X X X
2 X X X
3 X X X
4 X X X
5 X X X

The data from Expert 1 indicates that there may be a problem with Item 2 and Item 5.
You would need to examine the ratings of the other experts in order to decide how to
handle the situation.

The next step is to calculate the content validity ratio for each item. Only the
essentiality rating is used to perform this calculation.
(ne - N2 )
CVR 
N 2
ne = total number of experts divided by the number of experts saying item essential
N = total number of experts on the panel

Test Design and Delivery Page 1 of 2

Summary data for Item 2
Expert Essential to job performance
Useful, not
Not Maybe Definitely
essential
1 X
2 X
3 X
4 X
5 X

The CVR for Item 2 is zero and should not be included on the test.
5
2.5 
2 0
52
CONTENT VALIDITY INDEX (CVI). There will be one content validity index for each
form of a test. The content validity index is the mean of the content validity ratios for
each item on the form. The formula for calculating the CVI is shown below.

CVI 
 CVR
NCVR

PRACTICE
Use the summary data below to calculate the content validity index for a test. This
activity should take no more than 10 minutes.

Expert Ratings
Item Essential Not Essential
1 5 0
2 5 0
3 4 1
4 5 0
5 4 1

Answer to Optional Activity 2.5 is CVI = .84

Test Design and Delivery Page 2 of 2