Professional Documents
Culture Documents
The self-administered questionnaire is pre- The main advantage of the mailed question-
sented to the respondents by the researcher or by naire is its cheapness. Since it does not require a
someone in an official position such as schools' trained staff or the field workers, the payments
headteachers. Holroyd and Harlen (9: p. 326) to them and their travel expenses do not exisL
say that "Headteachers distributed the question- Furthermore, all it requires the cost of planning
naires to teachers of pupils. . ." The purpose of and piloting, printing expenses, sampling, and
the questionnaire is explained, and then the mailing, providing stamped, self-addressed
respondents is left alone to complete the ques- envelops for the retums. However, there is a
tionnaire, which may be sent or collected later. danger of low response rate in mail question-
This method assures a high response rate, accu- naire that it can make the survey expensive.
rate sampling, and minimum of bias, providing "...the response to a mail survey may be so low
necessary explanations and giying the benefit of that the cost per completed questionnaire is
personal contact (6). higher than with an interview sample." (10:
p.257). Another advantage of the questionnaires
is that their data processing and analysis is
2. THE COMPARISON OF INTER- cheaper than that of interviews.
VIEWS AND QUESTIONNAIRES
2.2. Time
2.1. Cost
The main disadvantage of interview is that it
Interviews cost more money than question- requires a great amount of time to collect infor-
naires. Moser and Kalton (10: p.256) tell us mation. Because it takes several interviews with
A Comparison ofTwo Data Collecting Methods: lnterviews and Questionnaires
3
different people before asimilar, a set of ques- requires less than 30 minutes to complete and,
tions has been asked to individual respondent in preferably, less than 15 or 20 minutes.
the survey. The actual interview session changes
"Questionnaires were completed in about 20
in length and time. Any interview under half an minutes and no pupils refused to take part." (15:
hour is unlikely to valuable; and an interview p. 309). A long and time-consuming question-
taking more than an hour may be making unrea- naıre, as pointed out by Huse'n and
sonable demands on busy interviews. So it can Postlethwaite (14: pA883) "... may cause a
effect of reducing the number of persons to par- respondent to cease to cooperate after a period
ticipate, which may in turn lead biases in the of answering questions. At best, one will receive
sample. an incomplete questionnaire and, at worst, the
Sharp (12: p.IO) admits that "Staff feel it is questionnaire will not be returned ." Therefore,
important to allow time for students to feel at these factors influence the response rate, relia-
ease during the interview, so interviews are bility and validity of the research.
timed to last at least 30 minutes." AIso she quot- AIso, self-completed questionnaires, which
ed from Howard Jones "Students who have been
respondents fill in for themselves, are very effi-
for interview at other colleges are often sur-
cient in terms of researchers time and effort.
prised at being gi yen this amount of time, but we Copies of the questionnaire could be distributed
feel that it is important to give staff and students to all pupils in a school, or to the hundreds of
time to find out about each other", Additionally, employees in firm, can be completed by them
Mouton et.al (13) say that "Interviews took 45 and returned to the researcher about the same
minutes to i hour and were completely volun-
amount of time that it takes to complete a single
tary ," it seems that choosing the voluntary sam-
interview.
ples is good solution for time-consuming in
interview. 2.3. Schedule
All interviews require careful preparation Interviews and questionnaires involve ques-
that means time and effort. "Arrangements to tions to ask to the respondents. There are main-
visit; securing necessary permission; confirming ly two types of questions, which are close-ended
arrangements; rescheduIing appointments to and open-ended. The close-ended type where
cover absences and crises; these need more time. the respondents choose from two or more alter-
Notes to be writing up; tapes if used require native. The most frequently used is the dichoto-
whole or partial transcription (one hour of tape mous question, which offers to a respondent
time requires ten hours to transcribe fully)" (7: only two alternatives: 'yes-no' or 'agree-dis-
p. 230). AIso, the analyzing and transcribing
agree'. According to Kerlinger the chief advan-
requires time. tage of this type question is that it has greater
Regarding the time for the completion a uniformity of measurement and so has higher
questionnaire, Huse'n and Postlethwaite (14) reliability; making respondents answer in a man-
suggest that for adults 30 minutes is about the ner fitting the response category; and being
upper limit that can be expected in the way of more easily coded. Their superficiality, the pos-
answering time when questionnaires are admin- sibility of irritating respondents who find none
istered in group setting. of the alternatives suitable, and the possibility of
" When questionnaires
are mail ed to respondents, about 15 minutes foreing responses that are inappropriate can be
appears to be the limit to the respondent time. found their disadvantages [cited in (2)],
Questionnaires administered to students may
The open-ended type which supply a frame
need to be shorter and require less time." (14:
of referenee for the respondents' answers but put
pA883). Therefore, the full questionnaire should a minimum control on the answers and their
4 Burcu Akbayrak
[ 1. of
Ed ]8
tion which includes impIicit knowledge. One of faH to obey instructions; she may show sur-
the aims of interview may be to bring as much as prise or boredom in tone or emphasis or in
other ways unconsciously communicate her
possible of this implicit knowledge out into the
own attitudes and her expectations of the
open, and to make it explicit."
responden!' s attitudes. Questions con-
Apart from that, interviewer is able to cerned with 'delicate' issues raise special
answer questions conceming the purpose of the problems."
interview, and the interviewee may be put at in
a way that is possible with questionnaire. This
may result accessing to more reliable informa- The main sources of bias are the characteris-
tion. On the other hand, in a questionnaire there tics of the interviewer and the characteristics of
is no way of determining how the respondent the respondents, and the content of the ques-
filled the questionnaire. It is also possible that tions. Studies have also show n that race, reli-
the questionnaire may be have been filled by gion, age and social class can be potential
individual other than the real respondent. sources of bias. There are some ways of reduc-
ing bias, which are the careful formulation of
2.6. Bias questions so that the meaning is very clear; thor-
ough training procedures so that an interviewer
Questionnaires and interviews are prone to
is more aware of possible problems; probabiIity
distortion of data because of the presence of
sampIing of respondents; and sometimes by
bias. One of the disadvantages of close-ended
matching interviewer characteristics with those
questionnaire is the source of bias. Since the
of the sample being interviewed (2).
researcher force respondents to choose one or
some predetermined altematives and to make Some of the problems mentioned above can
them consider on alternatives that maybe be extended by suitable selection and training of
respondents never think about them. interviewers, by careful preparation of the inter-
view schedule control, and by supervision of the
it seems that interviews are prone more bias
interview process. However, some other biases
than questionnaires because of the interviewer
may exist and they can influence the results of
factor. As Selltiz et .al (1962) point out, "inter-
the interview to an unknown degree.
viewers are human beings and not machines"
[cited in (18): p.95)]. Bias can be present in
2.7. Anonymity
wording questions and in the manner that they
are expressed. As Oppenheim (20: p.31-32) Anonymity is one of the features sought in
points out that the interview has several sources questionnaires. As Psacharopoulos (19: p.162)
of biases. indicates that "it is more often that question-
naires start by asking the name of the respon-
dents and his address. On the contrary, anony-
"The interviewer may give an inkling mous questionnaires generate a higher and bet-
of her opinion or expectations by her tone
ter quality response, especially on the income
of voice, the way she reads questions, or
simply by her appearance, dress and accent. and second job question.". This is very impor-
She may unwittingly influence the respon- tant in case of asking questions about one's pri-
dent by pausing expectantly at certain vate life.
points, by probing with leading questions,
Data obtained from the interviews and ques-
and by agreeing with the respondent in an
tionnaires should always be regarded as confi-
effort to maintain rapport. Her own expec-
tations and her selective understandings dential as possible, in the sense that no respons-
and recording of the answers may produce es or results should ever be published which
bias. An interviewer may misunderstand of could identify certain respondents. To get this
6 Burcu Akbayrak
[ J. of
Ed 18
result respondents should be given assurance to every researcher should satisfy himself that all
this effect and a guarantee of anonymity. For personal information, such as place ofresidence,
example, Boulton (23: p.440) to facilitate honest occupation, and the like, which could lead to an
answers and to comply with ethical standards identification, and which is not absolutely essen-
from a sample of 8-10 year-olds, informed the tial to the topic of inquiry, has either been
children that ".. .their responses would not be removed or effectively disguised." For example,
communicated to anybody from their schooL." Holroyd and Harlen (9: p.326) have assured the
and he continues by "At the outset of the inter- confidentiality of the questionnaires to the sam-
view, it was pointed out that it was not a test, as ple group. "Teachers were fully informed of the
there were no right or wrong answers, and that it aims of the research and the procedure that was
was acceptable for the children to say whatever adopted for the return of completed question-
they wanted in response to the questions." naires ensured confidentiality."
This is very important aspect of the inter- During the interview respondents tells the
views and questionnaires. The anonymous mail interviewer things that they never intended to
questionnaires give a greater portion of socially tell. Interviews can become confessions, partic-
unacceptable responses than face-to-face inter- ularly under the promise of confidentiality.
views. If possible the names of the respondents Therefore, the power of interviewing can put the
should not be asked to write or requested to sign interviewees at risk. As Patton (8: p.355)
the questionnaires. emphasizes that "The interviewer needs to have
an ethical framework for dealing with such
2.8. Confidentiality issues ."
response." (25: p.289). Interestingly, Scott (26) rates to mail surveys (mail questionnaires). They
has found no significant evidence to this view. worked on a random sample of 1, 000 retired
Also he supported his findings with two other public employees in a large western city. A four-
experiments (Brown,1960 and Sletto,1940). page questionnaire booklet of Dillman (1978)
Sletto sent questionnaires to three different design was mailed to the sample. They conclud-
group of 100 graduate students. Questionnaires ed that"... age, within elderly population, does
were 10 pages, 25 pages and 35 pages. have a significant impact on response rate to a
Surprisingly, the response rates were respective- mail questionnaire Mail surveys can be
ly 68 per cent, 60 per cent, and 63 per cent. administered successfully to elderly populations
Hence, there was no clear evidence of a clear and a reasonable overall response ratemay be
decline in response regarding the questionnaire obtained-but only if appropriate precautions are
length (cited in (26): p. 168)]. Unfortunately, taken." (27: p.75-76).
there was not recent research mentioning this
issue available. 2.10. Validity
Scott (26) has found in his research that the "High validity dem ands high reliability, but
official sponsorship is one of the main factors the reverse is not true. In practice, high reliabil-
affecting the high response rate. The similar evi- ity is found to give comfort to research workers
dence has found in Holroyd and Harlen' s even where validity is clearly the crucial
research in 1996 (9). They explain that "With
issue..." (26: p.179).
the valuable cooperation of the directors of AAP
science project, questionnaires were mailed to Although there are different type of validity,
the 119 schools...Replies were received 99 inferences about the validity of interviews are
schools, that is, 83 per cent of the AAP sample. made on the basis of face validity, that is,
lt is not possible to give a precise .figure for the whether the questions asked look as if theyare
return from the teacher population targeted since measuring what they claim to measure. One way
the exact number of possible respondents in of validating interview measures is to compare
each school is not known. Thanks to the strate- the interview measure with another measure that
gy of cooperation with AAP these r'esponse has already been shown to be valid. If two mea-
rates are unusually high for a questionnaire sur- sures agree, it can be concluded that the validity
vey." (9: p.326). of the interview is comparable with the reliable
validity of the other measure.
The refusal rate for personal interviews is
typically very much smailer than non-response Ross bases an interview's validity on the
rate for postal questionnaires. Although a good oral proficiency of an interviewee. "The validity
planning of the questionnaires can increase the of the interview as a legitimate interactional
response rate for the questionnaires, this remains process depends crucially on the interface
a major problem. The basic point is that the between the way language is used for the pur-
researcher has little or no knowledge about the pose of communication in the interview and how
views and characteristics of those who do not this use represents the extent and quality of lin-
response, which can seriously reduce any claims guistic accommodation in native-non-native dis-
to generalizability (7: p.237). course." (28: p.175).
Another aspect of the response rate is the Perhaps the most practical way of increasing
relationship between respondent age and the validity of the interviews is to minimize the
response. Arecent study was carried out by amount of bias as much as possible.
Kaldenberg, Koenig, and Becker (27) to exam- 1 The Scottish Office Education Department funds the
ine the relationship between age and response Assesment of Achievement Program (AAP)
8 Burcu Akbayrak
[ J. of
Ed 18
The validity of a questionnaire concems the and Postlethwaite concluded that "Correlations
extent to which it is actually capable of provid- of other socioeconomic variables were equally
ing information, which it claims to provideo In law, suggesting that there is a considerable
anather words, data obtained by a questionnaire amount of error in response to questionnaire
surveyare valid if they measure what theyare items. it is important that research workers or
supposed to measure. Typical examples of non- plan to use questionnaires investigate for mea-
validity refer to high sampling error, the age and surement error so that the results obtained from
productivity questions (19). questionnaires can be used with confidence or
correction for unreliability made in analysis."
The important point is that, general ques-
tion s about the validity of a questionnaire are not In order to assess the reliability, a smail
really meaningful when theyare considered number of children were re-interviewed by
without purpose of the questionnaire. The Boulton (23: pA41) between 3 and 5 days after
researcher needs to know which types of validi- the initial interview. "For each individual child,
ty is being applied as a criterion, and what pur- a high Kappa value was obtained, and the mean
pose the data obtained from the questionnaire across all 15 children was 0.89. These results
are intended to serve. support the view that children's nominations..."
Mareaver, oral proficiency is powerful fac-
2.11. Reliability tar for reliability and validity of an interview.
One of the characteristics sought in any Because oral proficiency has be en on the relia-
measuring device is the reliability. The reliabili- bility of the ratings, perhaps at the expense of
ty of a questionnaire is concemed with the the validity of the interview format (28).
extent to which procedure is capable of return-
ing an accurate result despite the presence of 2.12. Data Analysis
same factors, which may influence the outcome When data are collected through either inter-
in one directian or anather. A reliable question- views or questionnaires, the task of the analysis
naire would be one, which is capable of giying begins. The analysis of quantitative data is rela-
the same result when given to the same individ- tively easy and can be accomplished through the
uals on different time. In anather words, a reli- use of descriptive and inferential statistics.
able questionnaire or even any measuring tool Qualitative data, obtained from open-ended
should be free from the measurement errors. questions is rather more difficult to code and
Reliability of a questionnaire can be accom- analyze, since it is difficult to categorize the
plished in two ways. One approach is to ask the responses.
same question that was presented early in the One of the main problem s of interpreting
questionnaire in the same or slightly different qualitative data is that it rapidly seems to
form in the questionnaire. This approach gives a become very large. Researchers indicate the dif-
modest consistency check but does not take into ficulty of transcribing of an interview. These are
account variations in time. The second approach examples, "Half a dozen interviews can produce
is to re-administer a questionnaire to the same hours of transcriptian work and document col-
group of respondents several days later and
lection rapidly fill several files" (29: p.251).
compare the results that were obtained. Munck
"One hour tape recording means seven or ten
(1991) made a study of 9-year aıds in Sweden.
hour transcribing" (30).
Munck's results present a striking example of
the amount of unreliability inherent in question- Data obtained from interviews are difficult
naires. For the three items used in the question- to bring them together and analyze. if the inter-
naire, she found the "kappa carrelation"0.98, views are systematized and standardized such as
0.77, and0041. [citedin (14) p. 4886)]. Huse'n stmctured interviews, this facilitates coding data
A Comparison ofTwo Data Collecting Methods: lnterviews and Questionnaires 9
and analysis of data through computers. elosed end questions, responses have to limited
However, this provides less flexibilityand less into predetermined boxes which may or may not
sensitiye to individual differences. be appropriate.
if the questionnaire has been well designed
The relative advantages and disadvantages
and constructed, the time needed to code and
of interviews and questionnaires are summa-
analyze responses can be short, partieularly if
the computer cmıing or analysis is available. rized in Table 1. It seems that one advantage of
The data are necessarily superficial. There little one techniques is the disadvantage of the other
or no check on the honesty or seriousness of technique. it cannot be elaimed that one tech-
responses. In the questionnaires consist of nique is better than the other.
Table ı. A Summary of Advantages and Disadvantages of Interviews and Questionnaires
4. Foddy, W. "Constructing Questions for Interviews and 18.Bell,]. Doing Your Research Project, Buckingham,
Questionnaires", Public Opinion Quarterly, Vol. 58, Open University Press (1993).
pp. 639-42 (1994). 19.Psacharopoulos, G. "Questionnaire Surveys in
5. Fontana, A. and Frey, J.H. "Interviewing: The Art of Educational Planning", Comparative Education, Vol.
Science". In N.K. Denzin and Y.S. Lincoln (Eds.), 16. No. 2, pp.159-165 (1980).
Handbook of Qualitative Research, London, Sage 20.Oppenheim, A.N. Questionnaire Design and
Publications (1994). Attitude Measurement, London, Heinemann
6. Oppenheim, A. N. Questionnaire Design, Educational Books Ltd. (1972).
Interviewing and Attitude Measurement, London, 21.Schenk, P. "The Role of Drawing in Graphic Design
Pinter Publishers (1992). and The Implications for Curriculum Planning",
7. Robson, C. Real World Research, Oxford, Blackwell Journal of Art and Design, Vol. 16, No. I, pp.73-82
(1995). (1997) .
8. Patton, M.Q. Qualitative Evaiuation and Research 22. Walford, G. (ed) Doing Educational Research,
Methods, Califomia. Sage Publication (1990). London, Routledge (1991).
9. Holroyd, C. and Harlen, W. "Primary Teachers' 23.Boulton, MJ. "Bullying in Mixed Sex Groups of
Confidence About Teaching Science and Technology", Children", Educational Psychology, Vol. 16, No. 4.
Research Papers in Education, ii (3), pp.323-335 pp.439-443 (1996).
(1996).
24. Krysan, M., Schuman, H., Scott, L.J., Beatty, P.
IO.Moser C.A. and Kalton G. Survey Methods in Response Rates and Response Content in Mail Versus
Educational Social Investigation, Hants, Gower Face-to-Face Surveys, Public Opinion Quarterly,
(1989).
Vol. 58, p.381-9 (194).
Il.Granström, K. "Private Communication Between
25. Diliman, D.A., Sinclair, MD. and Clark, J.R. "Effects
Students in the Classroom in Relation to Different
of Questionnaire Length, Respondent-Friendly Design,
Classroom Features", Educational Psychology, Vol. and Difficult Question on Response Rates for
16, No. 4, p.349-6 (1996).
Occupant-Addressed Census Mail Surveys, Public
12.Sharp, C. DeveIoping the Arts in Primary Schools, Opinion Quarterly, Vol.57 , pp.289-304 (1993).
Berkshire, Nfer (1990).
26. Scott, C. "Research on Mail Surveys", Journal of the
13.Mouton, S.G., Hawkins, J. McPherson, R.H. and Royal Statistical Society, Series A, (124), pp. 143-205
Copley, J. School Attachment: Perspectives of Low- (196).
Attached High School Students, Educational
27. Kaldenberg,D.O., Koenig, H.F. and Becker, BW.
Psychology, Vol. 16, No. 3, p.297-30 (1987).
"Mail Survey Response Rate Patterns in Population of
14.Huse'n, T. and Post1ethwaite T.N. "Interviewing for the Elderly", Public Opinion Quarterly, Vol.58,
Selection and Clinical Research", and "Questionnaires" pp.68-76 (194).
The International EncycIopedia of Research, Second
28.Ross, S. "Accommodative Questions in Oral
Edition, Vo1.5. and 8, pp.3003-07 and pp. 4882-87.
Proficiency Interviews", Language Testing, VoJ..9,
Oxford, Pergamon Press (1994).
No.l, pp.173-184 (1992).
15.Cury, F., Biddle, S.. Famose, J.P., Goudas, M.,
Sarrazin, P. and Durand, M. "Personal and Situational 29. Winter, R., "Interviews, Interviewees and the Exercİse
Factors Influencing Intrinsic Interest of Adolescent of Power", British Educational Research Journal,
Vol. 17, No. 3, pp. 251-262 (1991).
Girls in School Physical Education: a Structural
Equation Modelling Analysis", Educational 30. Wragg, E.C. Conducting and Analysing Interviews,
Psychology, Vol. 16, No. 3, p. 305-314 (196). Oxford, TRC (1980).