Designing and conducting

questionnaires
Types of questionnaire item (closed-ended)
• Likert scales

• Semantic differential scales

• Numerical rating scales

• True-false items

• Multiple choice items

• Rank order items

• Checklists
Example Likert Scale item
Learning English is important for
economic development

Strongly Agree Neither Disagree Strongly

agree agree nor disagree
disagree
 ‘Smilegram’ response

  
  

Example semantic differential scales

Listening comprehension tasks are:

difficult _ : _ : _ : _ : _ : _ : _ easy
useless _ : _ : _ : _ : _ : _ : _ useful
enjoyable _ : _ : _ : _ : _ : _ : _ not enjoyable
 Example numerical rating scale
In the following section we would like you to answer some questions by giving
marks from 1 to 5.
1= not at all 2= not really 3= so-so 4= quite a lot 5= very much

_ How much do you like learning English?

Example multiple choice item

During English class, I would like:
1. To have a combination of English & Chinese spoken 
2. To have as much Chinese as possible spoken 
3. To have only English spoken 
 Example rank order item
Number the following language learning aids in order of importance to
you ( 1= most important, 5= least important)

_ Textbook
_ Dictionary
_ Video
_ Internet
_ Authentic materials
Example checklist item

_ Fairness _ Praises the students

_ Sense of humour _ Corrects all mistakes
_ Enthusiasm _ Patience
_ Native speaker _ Good at explaining
pronunciation
_ Understands students’
_ Good grammar problems
knowledge
_ Other: ……………………..
_ Well organised
 Types of open-ended question
• Specific open questions, e.g. “What languages have you studied in the past?”

• Clarification questions, e.g. “If you rated the course book as ‘poor’ or ‘very
poor’, please explain why”

• Sentence completion items, e.g. “One thing I liked about this activity is:…….”

• Short-answer questions
 Evaluating a survey
• Are your survey participants able to answer all of your questions?

• Will they interpret the questions in the way you expect? Are there any
ambiguous or confusing questions?

• Are you able to get the data you expected? Will there be any surprises?

Piloting is the key!

• How can you analyse your data?

• How can you present your data?

 Issues in questionnaire design
• Deciding on questionnaire themes

• Deciding on questionnaire length and format

• Choosing question types

• Facilitating later analysis

• Paper or web based delivery

• Ethical considerations
 What you can do?
• Not always a need to ‘re-invent the wheel’

• Not always a good idea to start from scratch

• Start from relevant literature

• Understanding the main ‘constructs’/themes

• Understanding your research context

 What to avoid?
• Avoid ambiguity and imprecision: On average, how much time do you spend
studying?

• Avoid challenging respondents’ memory/knowledge: How does your

secondary school teacher teach English?

• Avoid double-barreled question: How would you rate the gym facility and
gym staff’s professionalism?

• Avoid hypothetical questions: If we don't have penalty system, will you try to
submit your assignment on time?

• Avoid offensive questions: age (?), relationships (?), salary (?)

 Disadvantages
The reliability of survey data may depend on the following factors:
• Respondents may not feel encouraged to provide accurate answers

• Respondents may not be fully aware of their reasons for any given answer

• Surveys with closed-ended questions may have a lower validity than

other question types.

• Survey question options could lead to unclear data because certain

answer options may be interpreted differently by respondents.

• For example, the answer option “somewhat agree”

 Advantages (features) of survey
• Allows collection of both subjective and objective (reliable) data in a large
sample of the study population

• Can measure both qualitative and quantitative data

• A good tool for the protection of the privacy of the participants

• The validity of data and information depends on the honesty of the

respondent.

• Have the potential of reaching a large sample –> generalisability

Activity: Discuss with your neighbour:

• Would questionnaire be a good option for your proposed study?

• Give at least two reasons.

15
 Survey examples
• Foreign Language Classroom Anxiety Questionnaire (Horwitz et al 1986)

• Language Disposition Questionnaire (Dornyei et al 2006)

• Language Strategy Use Inventory (Cohen et al 2005 based on Oxford 2000)

• Questionnaire on pedagogic preferences (web based: Levine & Grace 2002)

Motivation questionnaires
https://www.zoltandornyei.co.uk/motivation-questionnaires
 References
Cohen, A, Oxford, R and Chi, J C 2005 Language Strategy Use
Inventory. University of Minnesota.

Dornyei, Z, Csizer, K and Nemeth, N 2006 Motivation, Language

Attitudes and Globalisation: A Hungarian perspective. Multilingual
Matters.

Horwitz, E K, Horwitz, M B and Cope, J 1986 Foreign language

classroom anxiety. Modern Language Journal 70 (2): 125-132