Qualitative Research Methodology

Reliability, Trustworthiness/ Validity and

Reflexivity in Qualitative Research
Session 25 and 26

By
Dr. Qudsia Kalsoom
School of Education
Recap of Some Concepts
Triangulation between methods involves
combining qualitative and quantitative
research methods. For example, in a case
study of a child regarding second
language vocabulary development at the
age 6, the researcher might collect data
through observation, interviews and
student’s vocabulary test scores.
Research paradigm is constructionism/
Interpretivism
 Mixed-Methods Research
• Research paradigm is Pragmatism
• Mixed methods research is combination of
qualitative and quantitative approaches in the
methodology of a study (Tashakkori & Teddlie,
1998).
• Mixed methods research is a research design with
philosophical assumptions as well as methods of
inquiry. As a methodology, it involves
philosophical assumptions that guide the
direction of the collection and analysis and the
mixture of qualitative and quantitative approaches
in many phases of the research process.
 Mixed-Methods Research
As a method, it focuses on collecting,
analyzing, and mixing both quantitative and
qualitative data in a single study or series of
studies. Its central premise is that the use of
quantitative and qualitative approaches, in
combination, provides a better understanding
of research problems than either approach
alone (Creswell & Plano Clark, 2007, p. 5).
 Types of Mixed-Method Studies
1. Convergent Design (QUALITATIVE +
QUANTITATIVE)
2. Explanatory Design (QUANTITATIVE +
Qualitative)
3. Exploratory Design (QUALITATIVE +
Quantitative)
Fig: Core Mixed Method Designs (Creswell, 2018)
Validity & Reliability in
Research
 Common Description of Validity and
Reliability in Research

Validity refers to measuring or investigating

exactly what the researcher intends to measure
or investigate.
Reliability refers to reaching to same
conclusions consistently or repeatedly.
 Reliability and Validity
Reliability is a necessary precondition of
validity.
Validity may be a sufficient but not necessary
condition for reliability.
 Can Validity be really Achieved?
It is IMPOSSIBLE for research to be 100 per cent
valid; that is the optimism of perfection.
Quantitative research possesses a measure of
standard error which is inbuilt and which has to be
acknowledged. In qualitative data the subjectivity of
respondents, their opinions, attitudes and
perspectives together contribute to a degree of bias.
Validity, then, should be seen as a matter of degree
rather than as an absolute state (Gronlund 1981).
 Kinds of Validity
1. content validity
2. criterion-related validity
3. construct validity
4. internal validity
5. external validity
6. concurrent validity
7. face validity
8. jury validity
9. predictive validity
10.consequential validity
11.systemic validity
12.catalytic validity
13.ecological validity
14.cultural validity
15.interpretive validity
16.theoretical validity
17.evaluative validity
 To understand validity in
qualitative research, principles of
qualitative research should be
taken into account.
Principles of Qualitative Research
1. The natural setting is the principal source of data.
2. Context-boundedness and ‘thick description’ are
important.
3. Data are socially situated, and socially and culturally
saturated.
4. The researcher is a part of the researched world.
5. The researcher – rather than a research tool – is the key
instrument of research.
6. There is a concern for processes rather than simply the
outcomes.
7. Data are presented in terms of the respondents rather
than researchers.
 Content Validity
To demonstrate this form of validity the instrument must
show that it fairly and comprehensively covers the domain
or items that it purports to cover. For example measuring
students vocabulary of 100 words (5-8 letters). A test
cannot have less than 100 items. Similarly, if the
researcher intends to capture the essence of online
learning experiences then the instrument should be
focused on online learning experiences and NOT on
comparison of face to face and online learning
experiences.
Another example:
The researcher needs to investigate experiences NOT
perceptions or beliefs.
 Content Validity
• Research intends to understand university
teachers’ conceptions/ understandings about
sustainability education.
• A valid question could be “How do you define/
describe sustainability education?” What are
the elements of sustainability education?
• An INVALID question would be how do you
perceive/ view sustainability education?
 Construct Validity
Construct validity refers to the validity of
measuring/ investigating a construct accurately
(as far as possible).
A construct is abstract. Construct validity is
different from content validity. Content validity
deals with actualities – defined content.
In construct validity agreement is sought on the
‘operationalized’ forms of a construct, clarifying
what we mean when we use this construct.
Operationalization of a Construct
 Operationalizing Sustainability Education

• Literature Review
In this study, sustainability education refers to an
education that includes content related to
environmental issues, social issues and
economic issues and activities that promote
systems thinking, normative competencies,
problem-solving skills and the disposition of
critical thinking.
 The Construct of Learning Outcome in
Science at grade V
The construct of learning outcomes in science may be
operationalized as:
• Knowledge of the concepts of …….
• Ability to analyze …..
• Ability to classify…
• Ability to hypothesize, experiment, predict, record,
infer, communicate
• Raising questions
• Observing things
 The construct of Social Justice Teacher
Education
Teacher education that includes:
• The term social justice appears in the mission
statement and goals of all key teacher education
documents (curricula, standards and ploicies)
• The programme includes a dedicated course on
some aspect of social justice e.g.
multiculturalism, citizenship, gender studies etc.
• Employs critical dialogue as a teaching and
learning method in all of courses.
Reliability in Qualitative Research
Quantitative research assumes the possibility of
replication; if the same methods are used with the
same sample then the results should be the same.
Typically quantitative methods require a degree of
control. This distorts the natural occurrence of
phenomena (ecological validity). Indeed the
premises of naturalistic studies include the
uniqueness of situations, such that the study
cannot be replicated – that is their strength rather
than their weakness.
NOTE: This is not to say that qualitative research
need not strive for replication in generating,
refining, comparing and validating constructs.
Criteria of reliability in quantitative
methodologies differ from those in
qualitative methodologies. In qualitative
methodologies reliability includes fidelity
(faithfulness) to real life, context- and
situation-specificity, authenticity,
comprehensiveness, detail, honesty, depth
of response and meaningfulness TO THE
RESPONDENTS.
 Suitability of the term for qualitative
research
The suitability of the term for qualitative
research is contested (e.g. Winter 2000;
Stenbacka 2001; Golafshani 2003).
Lincoln and Guba (1985) prefer to replace
‘reliability’ with terms such as ‘credibility’,
‘neutrality’, ‘confirmability’, ‘dependability’,
‘consistency’, ‘applicability’, and ‘transferability’,
in particular the notion of ‘dependability’.
Denzin and Lincoln (1994) suggest that
reliability as replicability in qualitative research
can be addressed in several ways:
stability of observations: whether the researcher
would have made the same observations and
interpretations if observations had been done
at a different time or in a different
place
inter-rater reliability: whether another observer
with the same theoretical framework and
observing the same phenomena would have
interpreted them in the same way.
 What leads to invalidity in Qualitative
Research?
• Interviewers and interviewees alike bring their own,
often unconscious, experiential and biographical
baggage (race, religion, gender, sexual orientation,
status, social class and Age) with them into the
interview situation.
• Hitchcock and Hughes (1989) argue that because
interviews are interpersonal, humans interacting
with humans, it is inevitable that the researcher will
have some influence. Interviewer neutrality is a
chimera/ fantasy (Denscombe 1995).
 Enhancing Reliability and Validity in
Qualitative Research
Minimize the amount of bias as much as possible
by avoiding:
• To SEE the respondent in their own image.
• To seek answers that support preconceived
notions
• Misperceptions (on the part of the interviewer)
of what the respondent is saying
• Misunderstandings (on the part of the
respondent) of what is being asked.
 Validity and Interviews
• Interview is not simply a data collection
situation but a social and frequently a political
situation.
• Literally the word ‘inter-view’ is a view
between people, mutually, not the interviewer
extracting data, one-way, from the
interviewee. Power can reside with
interviewer and interviewee alike (Thapar-
Bj¨orkert and Henry,2004)
 Power and Interviews
It is generally assumed that more power resides
with the interviewer as:
• the interviewer generates the questions and
the interviewee answers them;
• the interviewee is under scrutiny while the
interviewer is not.
• the interviewer tends to define the situation,
the topics, and the course of the interview.
 Power and Interviews
Neal (1995) draws attention to the feelings of powerlessness
and anxieties about physical presentation and status on the
part of interviewers when interviewing powerful people.
This is particularly so for frequently low-status research
students interviewing powerful people; a low-status female
research student might find that an interview with a male in
a position of power (e.g. a university Vice-chancellor or a
senior manager) might turn out to be very different from an
interview with the same person if conducted by a male
university professor where it is perceived by the interviewee
to be more of a dialogue between equals (see also Gewirtz
and Ozga 1993, 1994).
 Power and Interviews
• The effect of power can be felt even before the
interview commences where the researcher
(she) instances being kept waiting, and
subsequently being interrupted, being
patronized, and being interviewed by the
interviewee (Walford 1994).
• Scheurich (1995) maintains that many
powerful interviewees will rephrase or not
answer the question.
 Validity & Reliability: Addressing the issue
Power
One way of overcoming this is to have two
interviewers conducting each interview (Walford,
1994).
Hitchcock and Hughes (1989) observe that if the
researchers are known to the interviewees and
they are peers, however powerful, then a degree
of reciprocity might be taking place, with
interviewees giving answers that they think the
researchers might want to hear.
 Issues of Reliability and Validity
• The issues of reliability reside in the preparations for
and conduct of the interview and the ways
interviews are analysed.
• Lee (1993) and Kvale (1996) talk about the issue of
‘transcriber selectivity’. Even very detailed transcripts
of interviews remain selective, since they are
interpretations of social situations. They become
decontextualized, abstracted, even if they record
silences, intonation, non-verbal behaviour etc.
• The issue, then, is how useful they are to researchers
overall rather than whether they are completely
reliable.
The issues of reliability and validity
in qualitative research are
addressed through triangulation.
Reflexivity in Qualitative Research
REFLEXIVITY
Reflexivity refers to an awareness of
the researcher’s contribution to the
construction of meanings throughout
the research process, and an
acknowledgement of the impossibility
of remaining ‘outside of’ one’s study
matter while conducting research
(Nightingale & Cromby, 1999).
Suppose the research topic is :
Learning Mathematics: Experiences of grade X students
in public-sector schools of Pakistan.
Suppose the major finding of the research is:
Most of the students of grade X, in public-sector
schools of Pakistan, have terrible experiences regarding
learning the concepts of Mathematics.
The reader of the research will interpret this finding in
terms of the researcher’s reflexive statement such as:
Researcher’s own daughter studies in the public sector
school. OR
Researcher herself is a Mathematics teacher.
Reflexivity is extremely important in qualitative
research as the reality or truth is CONSTRUCTED by the
researcher.