Running head: REVISING SURVEY ITEMS 1

Jordan Glover
Hi Ed 830: Designing Institutional Research Studies
Penn State World Campus
Revising Survey Items
REVISING SURVEY ITEMS 2

1. Do you feel welcomed by staff and other students in the Center for Student

Organizations?

According to Guideline 4.9 (Dillman, Smyth & Christian, 2014), it is important that

“yes” and “no” questions should clearly mean “yes” or “no”. In the case of this question,

students may have a hard time answering this question on such a broad topic (the welcoming or

unwelcoming environment of the center) with either a yes or a no. Students may feel welcome

sometimes, and not others. It may be a staff member that makes them feel welcome or

unwelcome, a fellow student, or specific situations that cannot be conveyed in just a yes or no. I

would rewrite the question as a scale, and ask the student to rate how welcome they feel on

average, or typically. Here is how I would organize the question:

Please rate (on a scale of 1-10, with 1 being very unwelcome, and 10 being very welcome) how

welcome you feel on an everyday basis when visiting the Center for Student Organizations.

2. Organizing student groups is very hard. Have the leadership trainings helped you feel

better prepared for leading student organizations?

 Much more prepared  Somewhat more prepared

 Slightly more prepared  Not more prepared

This question is confusing to read and understand. I am presuming that students will pick one

of these four options to describe their experience with the leadership trainings. According to

Guideline 4.7 (Dillman, Smyth & Christian, 2014), when writing a survey, there should be as

few words as possible used. This question has an entire extra sentence as well as a wordy

question. I would rewrite the question to only one sentence, and have a clearer question:

After the leadership trainings, do you feel more or less prepared to lead student organizations?
Running head: REVISING SURVEY ITEMS 3

 Much more prepared  Somewhat more prepared

 Slightly more prepared  Not more prepared

3. Do you favor the Board of Trustees to raise student fees without approval of 60% of the

student organizations on campus?

 Favor  Oppose

When writing closed-ended questions, it’s important to keep in mind that respondents will

need to be able to answer the question clearly by picking one of the options as it relates to their

own opinion. The phrasing of this question is problematic as it includes a potentially leading

statement. There are a few other ways to phrase this question depending on the information that

you wanted to collect. If you wanted to see if student organization leaders were opposed or in

favor of the fee raise, then just that question should be asked. Or, if you wanted to see if student

organization leaders were opposed or in favor of the action taken by the board, then just that

question should be asked. According to Guideline 5.5, when asking either/or questions, both the

positive and negative options should be presented (Dillman, Smyth & Christian, 2014). I would

argue that this question is only stating the negative side of the question and then asking

respondents if they favor or oppose the action. Again, depending on the information you wanted

to get, there are multiple ways to organize the question. To ask specifically about the actions of

the board, I would structure the question like this:

Do you favor or oppose the Board of Trustees changing the amount students pay in fees without

the approval of all student organizations on campus?

REVISING SURVEY ITEMS 4

To ask specifically about the respondents’ opinions on the student fee raise, I would structure the

question like this:

Do you favor or oppose the raise in student fees?

By asking the question in either of these two ways, the survey will collect more accurate data. If

necessary, both questions could be included.

4. How satisfied are you with the variety of student organizations at the university?

 Very satisfied  Neither satisfied  Somewhat

 Somewhat nor dissatisfied dissatisfied

satisfied  Strongly

dissatisfied

When writing surveys, according to Guideline 4.6 (Dillman, Smyth & Christian, 2014), it is

important to use specific and concrete words. I don’t feel that this question is utilize specific

words. “Variety” is a vague word that could mean many different things to different people.

When I first read this question, I was thinking specifically about diversity in student

organizations. Are there options for all students? I believe that this is what the question is asking,

but I can imagine that students might also think that variety could mean options for different

academic interest, professional organizations versus organizations that are just for fun, or have a

special interest like community service. I think that having students rank whether or not they are

satisfied with the “variety” of offerings could lead to far too much variance in the results

depending on how each respondent interpreted the question. To rewrite the question, I would

keep the same scale that students need to pick from, but I would word the question like this:

How would you rank your satisfaction in the diversity of student organizations at the university?
REVISING SURVEY ITEMS 5

5. How often do you visit the Center for Student Organizations?

 Very often  Sometimes

 Often  Rarely

I think this question is good, the problem actually lies in the way that respondents need to

answer the question and think about their responses. There are a few changes that could be made

in order to make this question better, but I think that considering Guideline 5.8 would allow for

the most accurate data to be collected. Guideline 5.8 asks survey writers to consider what types

of answer spaces are most appropriate for the measurement they want to see in their results

(Dillman, Smyth & Christian, 2014).

Instead of asking “how often”, the person creating this survey should consider what

measurement of time they want to look at. Do they want to know how frequently students are

visiting the center weekly, monthly, semesterly, annually? If I were writing the survey, I would

want to know semesterly, but I think that would depend. So, to rework this question, here is what

I would propose:

Each semester, how frequently do you visit the Center for Student Organizations?

 1-3 times  4-6 times  7+ times

6. Which of the following factors influenced you to apply to the university? Please check all

that apply.

 Reputation of  Location of  Majors offered

university university  Diversity of

 Cost to attend student body

REVISING SURVEY ITEMS 6

 Availability of

student

organizations

Guideline 5.12 states that survey writers should utilize forced-choice questions instead of this

style, the check-all-that-apply option (Dillman, Smyth & Christian, 2014). While I like the

premise of this question, I almost feel that splitting this into two questions might be more

beneficial. I’m assuming based on what I know about this survey that the creator is looking to

see if student organizations were a reason that students opted to attend this institution. To see if it

is, and to see the level of importance that students placed on it during the decision process, I

would probably opt to create a series of questions on this topic:

1. Please indicate the primary reason you chose to apply to the university by selecting one

choice from the list below.

 Include all the same options, including a none of the above choice

2. Please indicate any of reasons that were in your top five when considering applying to the

university (please check all that apply)?

 Include all the same options, including a none of the above choice

I believe that by asking students to indicate their number one reason, and then any other things

that were in their top five things to consider the center could get better data on where student

organizations stand in the comparison against other factors.

7. How would you rate the overall quality of your experience participating in student

organizations?
Running head: REVISING SURVEY ITEMS 7

 Excellent  Good  Poor

 Very Good  Fair

In order to have an accurate answer from respondents in regard to their rating of the quality

of their experience, they need to be able to utilize a scale to rate their answer. Guideline 5.15

asks survey writers to choose direct labels to improve cognition. In order to have the best

possible results on this question, I believe that the question could remain the same, if instead the

answers were able to be rated on a scale from “poor” to “excellent”.

References

Dillman, D., Smyth, J., & Christian, L. (2014). Internet, phone, mail, and mixed-mode surveys :

The tailored design method (4th ed., pp. 126, 167-168). John Wiley & Sons,

Incorporated.

(Dillman, Smyth & Christian, 2014)