This action might not be possible to undo. Are you sure you want to continue?
As a mechanism for obtaining information and opinion, questionnaires have a number of advantages and disadvantages when compared with other evaluation tools. The key strengths and weaknesses of questionnaires are summarised in bullet points below. In general, questionnaires are effective mechanisms for efficient collection of certain kinds of information. They are not, however, a comprehensive means of evaluation and should be used to support and supplement other procedures for evaluating and improving teaching.
Advantages of questionnaires
• • •
They permit respondents time to consider their responses carefully without interference from, for example, an interviewer. Cost. It is possible to provide questionnaires to large numbers of people simultaneously. niformity. !ach respondent receives the identical set of questions. "ith closed# form questions, responses are standardised, which can assist in interpreting from large numbers of respondents. Can address a large number of issues and questions of concern in a relatively efficient way, with the possibility of a high response rate. $ften, questionnaires are designed so that answers to questions are scored and scores summed to obtain an overall measure of the attitudes and opinions of the respondent. They may be mailed to respondents %although this approach may lower the response rate&. They permit anonymity. It is usually argued that anonymity increases the rate of response and may increase the likelihood that responses reflect genuinely held opinions.
Disadvantages of questionnaires
• • • • •
It may be difficult to obtain a good response rate. $ften there is no strong motivation for respondents to respond. They are complex instruments and, if badly designed, can be misleading. They are an unsuitable method of evaluation if probing is required ' there is usually no real possibility for follow#up on answers. (uality of data is probably not as high as with alternative methods of data collection, such as personal interviewing. They can be misused ' a mistake is to try to read too much into questionnaire results.
Getting started )efore you start to design a questionnaire. 1lease tick one box for each of the questions. Avoid words of more than three or four syllables and over#long sentences.ere are some useful rules of thumb to follow• • • • . Use of open and closed questions +ost questionnaires contain both types of question and this is advisable. Avoid ambiguous. Avoid overly sensitive questions ' you are unlikely to get a /true0 response. which. In closed questions.& by a series of five closed questions. They are developed in simple headers and bullet points. double#barrelled questions being ones that ask two questions in one. will make this material more accessible and of practical benefit to potential users. Closed and open questions are appropriate in different contexts and provide different kinds of information.elp us measure the success of the programme. This helps the respondent and avoids difficulties later in interpreting questions that have no responses.& Exa ple 1 . +ore specifically. . In the following example. double#barrelled and hypothetical questions.eep questions simple. leading. identify its ob*ectives. students were asked to evaluate the quality of programme materials %handouts. identify what kind of information you want to obtain. I hope. allow the respondent the option of answering with /not appropriate0. %The questionnaire is not well designed but illustrates clearly the nature of closed questions. Then brainstorm ' write down all possible questions for incorporating in the questionnaire. /don0t know0 or /have no strong feelings0. . etc. Closed questions Closed questions are questions in which all possible answers are identified and the respondent is asked to choose one of the answers.Designing questionnaires This section contains extensive guidelines on how to design a questionnaire. Constructing questions This is the most difficult part of developing a questionnaire.
They avoid problems of interpreting respondents0 handwriting. providing a high level of control to the questioner. >3.<=&. This type of question is often used in evaluation to uncover respondents0 attitudes and opinions. 7ometimes this is factual information but closed questions are also used for obtaining data on attitudes and opinions %see ranked closed questions below&. Disadvantages of closed questions • • Closed questions are appropriate only when the set of possible answers are known and clear#cut. . p. so they are potentially easier for evaluating the opinion of the sample group as a whole. and this allows the questionnaire to ask more questions. inappropriate questions and questions that have answers other than those listed. They can provide better information than open#ended questions. They provide uniformity of questions and student responses. Typical problems are poorly designed questions. closed questions may be misleading and frustrate respondents. They oblige the respondent to answer particular questions. They involve minimal effort on the part of the respondent.8it9#:ibbon and +orris %2. Closed questions are less time consuming for respondents to complete. Closed questions %ith ran&ed ans%ers • These are closed questions for which answers are located on a scale of alternatives. If poorly designed. particularly where respondents are not highly motivated. Advantages of closed questions • • • • • • • Closed questions are an appropriate means of asking questions that have a finite set of answers of a clear#cut nature. The scale often represents degrees of satisfaction with a particular service or degrees of agreement with a statement.!rogra e aterials Excellent Good "air !oor Una#le to $udge 2& the availability of the materials 3& the quality of the materials 4& the durability of the materials 5& the quantity of the materials 6& the suitability of the materials for students 7ource. They save time.
7ummary statistics are powerful and if based on poorly designed questionnaires can be damaging. please ring your answer.• Always balance scales around a mid#point in the response answer. 5 A Bery :ood. The module was . The module stimulated my interest Cisagree 2 3 4 5 6 Agree 3. Advantages of ran&ed questions • • • The advantages of using closed questions apply ' see above. agree. The module as a whole 2. respondent may choose from the following alternatives. 3 A 8air. Disadvantages of ran&ed questions • • • Attitudes and opinions are complex and not readily summarised in a scale. 4 A 7atisfactory. Answers can be pooled across students to derive summary statistics that potentially measure an overall degree of satisfaction?agreement. @anked questions do not provide means for students to elaborate on or explain reasons behind the stated degree of satisfaction.strongly agree. Exa ple ' 8ill in one response for each question. have no strong feelings. disagree. strongly disagree. The following are examples of ranked closed questions drawn from questionnaires used to evaluate teaching in anonymous economics departments. 7ummary scores can also be obtained from pooling across different questions related to some overriding issue. 2 A 1oor 7kill of the instructor 2& Instructor0s effectiveness as a lecturer 3& Clarity of instructor0s presentations 2 3 4 5 6 2 3 4 5 6 4& Instructor0s ability to stimulate interest in the sub*ect 2 3 4 5 6 Exa ple 3 8or each of the following questions. 6 A !xcellent. 8or example.
"e have to be sure that every question used in computing the index indeed reveals information about a student0s level of enthusiasm. They may be better means of eliciting true opinions?attitudes and identifying how strongly attitudes are held or not. The module ob*ectives were fulfilled Cisagree 2 3 4 5 6 Agree Exa ple ( This is an example of how ranked questions may be pooled to generate an overall index %from . 5 and 6 were summed to obtain /an enthusiasm index0 for each child.<=&Teachers in a new experienced#based science programme filled out a questionnaire about each of several children in their classes. . of course.Too easy 2 3 4 5 6 Too hard 4. The alternative approach here would be to ask teachers to rate the enthusiasm of students. . )pen questions $pen questions are questions that allow the respondent to answer in any way they wish. but it is difficult to know where to fix the thresholds. students might be asked to respond to the following question. The respondent can answer in any way he?she wishes.. a point on a scale of 5'3D. 2. 4. and that the scales of the questions are consistent./"hat do you feel is the best thing%s& about the courseE0 Advantages of open questions • • 8lexibility.enerson et al. i.ere is a portion of the questionnaire- The scores for questions 3. The greatest difficulty lies in interpretation of the final scores ' usually researchers consider scores above or beneath threshold levels as revealing something definite about behaviour and attitudes.e. 8or example. that high enthusiasm is always indicated by scores close to or equal to 6. There are difficulties designing and interpreting these results.
"here possible. are limited in number and are clear#cut. year of study. @espondents may answer in unhelpful ways. degree programme. +ost questionnaires are /mixed0.>. avoiding an overly restrictive questionnaire and one that is too open and difficult to analyse. containing both open and closed questions. se of such questions makes it less likely that the respondent will disengage from the questionnaire. +uestionnaires should #e attractive • • • • • "arm#up questions are recommended. A good questionnaire has a coherent structure. $pen#ended questions can be used by students to elaborate on the reasons underlying their answers to the closed#form questions. or ask questions that require answers to be completed on subsequent pages. At the end of the questionnaire it is a nice touch to thank the respondent for his?her time and consideration. . This should contain the following features• • • • The purpose of the questionnaire should be communicated clearly to potential respondents. 2. the confidentiality of responses should be assured. It is more difficult to pool opinion across the sample when questionnaires use open questions. The supporting text should contain simple instructions on how to complete. These are questions that are simple to answer. )pen versus closed questions /. such as questions on the age of student. . This dramatically reduces the number of questions that the questionnaire can realistically ask. Co not overcrowd the questionnaire with questions and text. and where a process is being explored0 %7tacey. "here deemed appropriate. Co not leave important questions to the end of questionnaire. collect questions under definable sub*ect areas and develop a logical order of questionnaires. etc. $pen#ended questions are used where the issue is complex.Disadvantages of open questions • • • They require more thought and time on the part of respondent and analyst. Co not split questions over pages.&. *upporting text All questionnaires must be supported with some text. closed questions should be used where alternative replies are known. where relevant dimensions are not known. This is often the best approach. .
which. so you can have more of them. the appropriate length does depend upon the purpose of the questionnaire. o 8or the validity of the questionnaire. Directness. This criterion relates to the practicalities of implementing and using a questionnaire. is a sample of students. Cesigners need to identify at the outset what kinds of knowledge they are trying to obtain from the questionnaire. o Appropriately chosen and designed closed questions are easy to answer. it is often appropriate to include a number of questions relating to one broad issue. o "here questionnaires are administered at the beginning of the lecture. insufficient thought is given to the purpose of a questionnaire. the sample is biased towards those students who attend lectures ' of course. questionnaires should continue to be evaluated as an ongoing process. the type of respondents targeted and the type of questions. score and interpretE o "hat resources are involved in implementing the questionnaireE Reliability. In evaluating a questionnaire.<3& identify five key criteria that may be used in evaluating the quality of a questionnaire ' these are listed and discussed below.ength of the questionnaire • It is advised that questionnaires should not be too long %for obvious reasons&. o ..e. The criteria to use in evaluating a questionnaire are• • • • • Purpose. they should ask questions that address as directly as possible the issues you want to evaluate. much depends on the quality of the questioning. To evaluate a questionnaire effectively. Utility. (uestionnaires should be as direct as possible. one has to be absolutely clear about the purpose.ere.owever. this bias may raise the quality of responsesF Validity. Test results are analysed and any changes to the questionnaire made. )loom and 8ischer %2. o $ften. After initial implementation. . -esting questionnaires It is essential that questionnaires are thoroughly tested prior to use. (uestions to consider includeo Is the questionnaire easy to administer. in our case. A study is reliable if similar results would be obtained by others using the same questions and using the same sampling criteria.A study is valid if it actually measures what it sets out to measure. i. . it should be tested on an appropriate sample.
This action might not be possible to undo. Are you sure you want to continue?