DEVELOPMENT OF ASSESSMENT TOOLS

IN THE AFFECTIVE DOMAIN

Assessment tools in the affective domain, in particular, those which are used to assess attitudes,
interests, motivations, and self-efficacy, have been developed. We consider a few of the standard
assessment tools in the affective domain.

Self-Report

The most common measurement tool in the affective domain. It essentially requires an individual
to provide an account of his attitude or feelings toward a concept or idea or people. Self-reports are also
sometimes called written reflections. The teacher ensures that the students write something, which would
demonstrate the various levels of the taxonomy e.g. lowest level of receiving up to characterization.

Rating Scales

It is a set of categories designed to elicit information about a quantitative attribute. e.g. Likert
Scale and 1-10 rating scale for which a person selects the number which is considered to reflect the
perceived quality of a product. The basic feature of any rating scale is that it consists of a number of
categories. These are usually assigned integers.

Semantic Differential Scale

Semantic differential is a rating scale to measure the connotative meaning towards ideas,
concepts, items, people, and events (sometimes referred to as an attitudinal study). The
connotations are used to derive the attitudes towards the given object, event or concept.
Respondent is asked to choose where his or her position lies, on a scale between 2 bipolar
adjectives (e.g. adequate – inadequate; good – evil; or valuable – worthless). It is the most accurate
take on respondents’ emotional feedback. Semantic Differential questions were developed by C.E.
Osgood in 1957.

Two Aspects of Meaning

1. Denotation - what a name or concept refers to (denote - to mark out plainly, to indicate)
2. Connotation - the suggestive significance of a word, apart from its explicit and recognized
meaning

Example:

denotation = eagle, bird

connotation = strength, power

Creating the scale

The scale is set up using polar adjectives (opposite-meaning terms) at each end.

Three Major Dimensions of Meaning (by Charles Osgood and his associates)
1. Strength/potency (pairs such as ‘powerful-weak’)
2. Value/evaluation (pairs like ‘good-bad’)
3. Activity (includes pairs like ‘active-passive’)
 The first two examples below fit the theme of strength. The second two represent
value, and the last two illustrate activity.

Strong____:____:____:____:____:____:____Weak
Decisive____:____:____:____:____:____:____Indecisive
Good____:____:____:____:____:____:____Bad
Cheap____:____:____:____:____:____:____Expensive
Active____:____:____:____:____:____:____Passive
Lazy____:____:____:____:____:____:____Industrious

The respondent is asked to rate an object, person or any concept, by putting a mark
on one of the 7 spaces along each dimension.

Layout and length

It is not necessary to use these particular sets of adjectives, or cover all three themes.
Any set can be substituted, depending on the purpose of the research.
Use 10-12 adjective pairs (not more than 20) to avoid fatigue or boring the
respondent (fewer is acceptable)
Location of the positive attributes should be varied; do not put all the "good"
adjectives on one side, as it might bias the responses.

Provide clear instructions for the respondent to put their marks in the right place.

Examples:

Instructions: Make your ratings by checking the appropriate space. For example,

Rate the current Secretary-General of the United Nations on each of the following
dimensions:

Strong ____:____:____:____:____:____:____ Weak

Decisive ____:____:____:____:____:____:____ Indecisive
Good ____:____:____:____:____:____:____ Bad
Cheap ____:____:____:____:____:____:____ Expensive
Active ____:____:____:____:____:____:____ Passive
Lazy ____:____:____:____:____:____:____ Industrious

Although a 7-point scale is common, it is acceptable to provide fewer choices, 5

or even 3. For example,

Rate the park on the following dimensions:

Safe____:____:____Dangerous
Dirty____:____:____Clean
Quiet____:____:____Noisy

Scoring

The blanks are numbered from 1 to 7, get the average for each dimension. The
average is plotted on the form and provides a profile of the connotation of the target
concept.

Here is an example of semantic differential results comparing the connotative

meaning of a university and state college system. The differences on each dimension can
be statistically analyzed.
 State College

University

If respondents, despite the instructions, circle the colon, you can score the
dimension using a midpoint. For example, if they circle the colon between the first and
second space, score it as 1.5.

Results can be presented in a figure, as above. Or you can list the average scores on
each dimension, and then draw conclusions.

Limitations
The semantic differential is good for respondents who are intelligent/cooperative;
who have good knowledge of language; who are willing and able to make fine distinctions;
therefore, would not for children, unless simplified

Likert Scale

The Likert Scale is named after its originator, Rensis Likert. It is also called the summative
scale, as the result of a questionnaire is often achieved by summing numerical assignments to the
responses given. It is an ordered, one-dimensional scale from which respondents choose one option
that best aligns with their view. There are typically between four and seven options. Five is very
common.

Basic steps in developing a Likert Scale

1. Defining the Focus.

The first step is to define what it is you are trying to measure.

2. Generating the Items.

The next is to create the set of potential scale items. These should be items that can be
rated on a 1-to-5 or 1-to-7 Disagree - Agree response scale.

3. Rating the Items.

The next step is to have a group of judges rate the items. Usually you would use a 1-
to-5 rating scale where:
= strongly unfavorable to the concept
= somewhat unfavorable to the concept
= undecided
= somewhat favorable to the concept
= strongly favorable to the concept
 Notice that, as in other scaling methods, the judges are not telling what they believe -
- they are judging how favorable each item is with respect to the construct of interest.

4. Selecting the Items.

The next step is to compute the inter-correlations between all pairs of items, based
on the ratings of the judges. In making judgements about which items to retain for the final
scale there are several analyses you can do:
• Throw out any items that have a low correlation with the total (summed) score across
all items
In most statistics packages it is relatively easy to compute this type of Item-
Total correlation. First, you create a new variable which is the sum of all of the
individual items for each respondent. Then, you include this variable in the correlation
matrix computation (if you include it as the last variable in the list, the resulting Item-
Total correlations will all be the last line of the correlation matrix and will be easy to
spot). How low should the correlation be for you to throw out the item? There is no
fixed rule here -- you might eliminate all items with a correlation with the total score
less than .6, for example.
• For each item, get the average rating for the top quarter of judges and the bottom
quarter. Then, do a t-test of the differences between the mean value for the item for the
top and bottom quarter judges.
Higher t-values mean that there is a greater difference between the highest and
lowest judges. In more practical terms, items with higher t-values are better
discriminators, so you want to keep these items. In the end, you will have to use your
judgement about which items are most sensibly retained. You want a relatively small
number of items on your final scale (e.g., 10-15) and you want them to have high Item-
Total correlations and high discrimination (e.g., high t-values).

5. Administering the Scale.

Each respondent is asked to rate each item on some response scale. For instance, they
could rate each item on a 1-to-5 response scale where:
1 = strongly disagree
2 = disagree
3 = undecided
4 = agree
5 = strongly agree

There are a variety possible response scales (1-to-7, 1-to-9, 0-to-4). All of these odd-
numbered scales have a middle value is often labeled Neutral or Undecided. It is also possible
to use a forced-choice response scale with an even number of responses and no middle neutral
or undecided choice. In this situation, the respondent is forced to decide whether they lean
more towards the agree or disagree end of the scale for each item.

The final score for the respondent on the scale is the sum of their ratings for all of the
items (this is why this is sometimes called a "summated" scale). On some scales, you will
have items that are reversed in meaning from the overall direction of the scale. These are
called reversal items. You will need to reverse the response value for each of these items
before summing for the total. That is, if the respondent gave a 1, you make it a 5; if they gave
a 2 you make it a 4; 3 = 3; 4 = 2; and, 5 = 1.

Example No. 1:

Instructions: Please rate how strongly you agree or disagree with the statement below by
placing a check mark in the appropriate box (SA - strongly agree; D –
disagree, U – undecided; A – agree; and SD – strongly disagree)

SD D U A SA
1 2 3 4 5
I do not like to solve algebraic equation.
From the item, 50 respondents answered.

The following are the answers: 10 respondents answered Strongly Agree, 20

answered Agree, 10 answered Undecided, 5 answered Disagree, and 5 on Strongly
Disagree. Computation for the weighted mean is as follows:

Scale No. of Respondents Total

5 10 50
4 20 80
3 10 30
2 5 10
1 5 5
TOTAL 50 175 / 50 = 3.5 (weighted mean)

To get the interval for the interpretation of the result: 5 – 1 = 4 (range) / 5 (steps) = .8
Therefore:
Scale Description
4.20 – 5.00 SA
3.40 – 4.19 A = 3.5
2.60 – 3.39 U
1.80 – 2.59 D
1.00 – 1.79 SD

Conclusion: The respondent ‘Agree’ that they do not like to solve algebraic equation.

Example No. 2:

The Employment Self Esteem Scale, a ten-item Likert Scale that attempts to
estimate the level of self-esteem a person has on the job.

Notice that this instrument has no center or neutral point -- the respondent has to
declare whether he/she is in agreement or disagreement with the item.

Instructions: Please rate how strongly you agree or disagree with each of the following
statements by placing a check mark in the appropriate box.

Strongly Somewhat Somewhat Strongly

Disagree Disagree Agree Agree Statements
1. I feel good about my work on the job.
2. On the whole, I get along well with others at
work.
3. I am proud of my ability to cope with
difficulties at work.
4. When I feel uncomfortable at work, I know
how to handle it.
5. I can tell that other people at work are glad to
have me there.
6. I know I'll be able to cope with work for as
long as I want.
7. I am proud of my relationship with my
supervisor at work.
8. I am confident that I can handle my job without
constant assistance.
9. I feel like I make a useful contribution at work.
10. I can tell that my coworkers respect me.
Other examples:

5-point traditional Likert scale:

Neither
Strongly Tend to agree Tend to Strongly
agree agree nor disagree disagree
disagree
I like going to Chinese restaurants [ ] [ ] [ ] [ ] [ ]

5-point Likert-type scale, not all labeled:

Good Neutral Bad

When I think about Chinese restaurants I feel [ ] [ ] [ ] [ ] [ ]

6-point Likert-type scale:

Never Infrequently Sometimes Frequently Always

I feel happy when
entering a Chinese O O O O O O
Restaurant

Semantic Differential Versus Likert Scale

Example: conducting an employee evaluation of their managers

 Or you could use this:

Which do you think looks easier for the respondent?

When using the Likert scale questions, if you say, “I feel my manager is
decisive,” the way you are using “decisive” could influence the survey respondent and
give you less than accurate feedback.

You could go the other direction and say, “I feel my manager is indecisive,” but
then you would have the same problem.

Thurstone Scale

It is a unidimensional scale to measure attitudes towards people.

Thurstone was one of the first and most productive scaling theorists. He actually invented
three different methods for developing a unidimensional scale: the method of equal-appearing
intervals; the method of successive intervals; and, the method of paired comparisons. The three
methods differed in how the scale values for items were constructed, but in all three cases, the
resulting scale was rated the same way by respondents. Among the three, the method of equal-
appearing intervals is the easiest to implement.

The Method of Equal-Appearing Intervals

1. Developing the focus

- generate statements that describe specific attitudes that people might have towards a
specific topic (ex. AIDS)

2. Generating potential statements (create statements)

- have an item pool of 80-100
- worded similarly, don’t differ in grammar /structure (stated in question form; answerable
with agree or dis agree)

For our example focus on developing an AIDS attitude scale, might generate
statements like the following (these statements came from a class exercise in Spring 1997
undergrad class):

1. people get AIDS by engaging in immoral behavior

2. you can get AIDS from toilet seats
3. AIDS is the wrath of God
4. anybody with AIDS is either gay or a junkie
5. AIDS is an epidemic that affects us all
6. people with AIDS are bad
7. people with AIDS are real people
8. AIDS is a cure, not a disease
9. you can get AIDS from heterosexual sex
10. people with AIDS are like my parents
 11. you can get AIDS from public toilets
12. women don’t get AIDS
13. I treat everyone the same, regardless of whether or not they have AIDS
14. AIDS costs the public too much
15. AIDS is something the other guy gets
16. living with AIDS is impossible
17. children cannot catch AIDS
18. AIDS is a death sentence
19. because AIDS is preventable, we should focus our resources on prevention instead of curing
20. People who contract AIDS deserve it
21. AIDS doesn't have a preference, anyone can get it.
22. AIDS is the worst thing that could happen to you.
23. AIDS is good because it will help control the population.
24. If you have AIDS, you can still live a normal life.
25. People with AIDS do not need or deserve our help
26. By the time I would get sick from AIDS, there will be a cure
27. AIDS will never happen to me
28. you can't get AIDS from oral sex
29. AIDS is spread the same way colds are
30. AIDS does not discriminate
31. You can get AIDS from kissing
32. AIDS is spread through the air
33. Condoms will always prevent the spread of AIDS
34. People with AIDS deserve what they got
35. If you get AIDS you will die within a year
36. Bad people get AIDS and since I am a good person I will never get AIDS
37. I don't care if I get AIDS because researchers will soon find a cure for it.
38. AIDS distracts from other diseases that deserve our attention more
39. bringing AIDS into my family would be the worst thing I could do
40. very few people have AIDS, so it's unlikely that I'll ever come into contact with a sufferer
41. if my brother caught AIDS I'd never talk to him again
42. People with AIDS deserve our understanding, but not necessarily special treatment
43. AIDS is a omnipresent, ruthless killer that lurks around dark alleys, silently waiting for naive
victims to wander passed so that it might pounce.
44. I can't get AIDS if I'm in a monogamous relationship
45. the nation's blood supply is safe
46. universal precautions are infallible
47. people with AIDS should be quarantined to protect the rest of society
48. because I don't live in a big city, the threat of AIDS is very small
49. I know enough about the spread of the disease that I would have no problem working in a health
care setting with patients with AIDS
50. the AIDS virus will not ever affect me
51. Everyone affected with AIDS deserves it due to their lifestyle
52. Someone with AIDS could be just like me
53. People infected with AIDS did not have safe sex
54. Aids affects us all.
55. People with AIDS should be treated just like everybody else.
56. AIDS is a disease that anyone can get if there are not careful.
57. It's easy to get AIDS.
58. The likelihood of contracting AIDS is very low.
59. The AIDS quilt is an emotional reminder to remember those who did not deserve to die painfully
or in vain
60. The number of individuals with AIDS in Hollywood is higher than the general public thinks
61. It is not the AIDS virus that kills people, it is complications from other illnesses (because the
immune system isn't functioning) that cause death
62. AIDS is becoming more a problem for heterosexual women and their offsprings than IV drug
users or homosexuals
63. A cure for AIDS is on the horizon
64. A cure for AIDS is on the horizon
65. Mandatory HIV testing should be established for all pregnant women

3. Rating the statements (on favorability of statement)

- participants rate from 1 to 11 (not to rate attitude on aids but to rate the statement, where 1
= ‘extremely unfavorable’ and 11 = ‘extremely favorable’ over the statement)
4. Computing scale score values for each item
- get the median & interquartile range (Q3 – Q1)
Example:

Statement Interquartile Statement Interquartile

Median Q1 Q3 Median Q1 Q3
Number Range Number Range
23 1 1 2.5 1.5 6 3 1.5 4 2.5
8 1 1 2 1 21 3 1.5 4 2.5
12 1 1 2 1 32 3 2 4.5 2.5
34 1 1 2 1 9 3 2 3.5 1.5
39 1 1 2 1 1 4 3 7 4
54 1 1 2 1 26 4 1 5 4
56 1 1 2 1 47 4 1 5 4
57 1 1 2 1 30 4 1.5 5 3.5
18 1 1 1 0 13 4 2 5 3
25 1 1 1 0 11 4 2 4.5 2.5
51 1 1 1 0 15 4 3 5 2
27 2 1 5 4 40 5 4.5 8 3.5
45 2 1 4 3 2 5 4 6.5 2.5
16 2 1 3.5 2.5 14 5 4 6 2
42 2 1 3.5 2.5 17 5.5 4 8 4
24 2 1 3 2 49 6 5 9.75 4.75
44 2 2 4 2 50 8 5.5 11 5.5
36 2 1 2.5 1.5 35 8 6.25 10 3.75
43 2 1 2.5 1.5 29 9 5.5 11 5.5
33 3 1 5 4 38 9 5.5 10.5 5
48 3 1 5 4 3 9 6 10 4
20 3 1.5 5 3.5 55 9 7 11 4
28 3 1.5 5 3.5 10 10 6 10.5 4.5
31 3 1.5 5 3.5 7 10 7.5 11 3.5
19 3 1 4 3 46 10 8 11 3
22 3 1 4 3 5 10 8.5 11 2.5
37 3 1 4 3 53 11 9.5 11 1.5
41 3 2 5 3 4 11 10 11 1

5. Selecting the Final Scale System

- select items that are at equal interval across the range of medians
- select the item with the smallest interquartile range
- statement with the least amount of variability across judges

Came up with the following set of items for the scale:

• People with AIDS are like my parents (6)
• Because AIDS is preventable, we should focus our resources on prevention instead of curing (5)
• People with AIDS deserve what they got. (1)
 • Aids affects us all (10)
• People with AIDS should be treated just like everybody else. (11)
• AIDS will never happen to me. (3)
• It's easy to get AIDS (5)
• AIDS doesn't have a preference; anyone can get it (9)
• AIDS is a disease that anyone can get if they are not careful (9)
• If you have AIDS, you can still lead a normal life (8)
• AIDS is good because it helps control the population. (2)
• I can't get AIDS if I'm in a monogamous relationship. (4)

Scale value – value in parentheses after each item (higher value – more favorable attitude
towards people with aids)
6. Administering the Scale (no more values)

Agree Disagree Statements

People with AIDS are like my parents.
Because AIDS is preventable, we should focus our resources on
prevention instead of curing.
People with AIDS deserve what they got.
Aids affects us all.
People with AIDS should be treated just like everybody else.
AIDS will never happen to me.
It's easy to get AIDS.
AIDS doesn't have a preference, anyone can get it.
AIDS is a disease that anyone can get if they are not careful.
If you have AIDS, you can still lead a normal life.
AIDS is good because it helps control the population.
I can't get AIDS if I'm in a monogamous relationship.

Prepared by:

JANE D. NAVALTA
CHERRY ANN P. NAVALTA
Professors

REFERENCES:

Beasley, Terry. (2015). Retrieved from

http://psc.dss.ucdavis.edu/sommerb/sommerdemo/scaling/semdiff.htm

Navalta, Jane D. (2017). Teaching guide in ASL 2

Trochim, William M.K. (2006). Retrieved from

https://www.socialresearchmethods.net/kb/scallik.php

http://changingminds.org/explanations/research/measurement/likert_scale.htm

http://psc.dss.ucdavis.edu/sommerb/sommerdemo/scaling/enrich/thurstone.htm

https://www.socialresearchmethods.net/kb/scalthur.php

http://www.statisticshowto.com/thurstone-scale