Professional Documents
Culture Documents
3 (2021) 1
RESEARCH
UNDERSTANDING VALIDITY AND RELIABILITY
FROM QUALITATIVE AND QUANTITATIVE RESEARCH
TRADITIONS
Abstract: Educational constructs change over time to reflect developments in research and
educational approaches. To illustrate the process, this article aims to examine validity and reliability,
which are important concepts to justify research quality. Originally, validity and reliability were applied
to quantitative research. However, these criteria can not be equally applied to qualitative research studies
which differ in terms of their theoretical foundations and research aims. The unclear use of these
concepts might lead to inappropriate research design or evaluation. This paper, therefore, first examines
two different theoretical foundations underlying these two research traditions. It then analyses the subtle
variations to clarify the notions of reliability and validity. Some implications are made for researchers
to flexibly employ these criteria to enhance their research rigor.
Key words: validity, reliability, qualitative research, quantitative research
1
*
Corresponding author. This research is funded by Vietnam National
University, Hanoi (VNU) under project number
Email address: nhavtt@vnu.edu.vn
QG.20.04.
https://doi.org/10.25073/2525-2445/vnufs.4672
VNU JOURNAL OF FOREIGN STUDIES, VOL. 37, NO. 3 (2021) 2
research under the constructivist paradigm habit when some researchers who have
has come into practice. Instead of trying to worked in quantitative research for a long
explain a phenomenon through a verification time before they are introduced to qualitative
or falsification process, qualitative research research. Dörnyei (2007), for example,
aims to “understand, interpret, explain admits that he is “more naturally inclined”
complex and highly textualized social to quantitative research (p. 47), given his
phenomena” (Kamberelis & Dimitriadis, past training and experience in quantitative
2005, p. 17). methodology. He needs collaboration with
While such purist authors contrast qualitative researchers to complement his
the two paradigms, situationalist and quantitative orientation.
pragmatist researchers see the shared values Another source of complications is
of both paradigms (Donyei, 2007). For the lack of clear-cut boundaries between
example, Merriam (2009) supports the view sound and unsound research practices in
that qualitative research is best defined from mixed-method research. On the one hand,
its philosophical underpinnings, and at more researchers intentionally adopt some
micro levels, they may overlap. She states: unsound scientific practices to cope with
I think it is helpful to philosophically publishing criteria (Świątkowski &
position qualitative research among Dompnier, 2017). For instance, HARKing,
other forms of research. Such a is a practice of quantitative researchers who
change their hypothesis after the results are
positioning entails what one believes
known. They start their research with a
about the nature of reality (also
hypothesis which can not be positively
called ontology) and the nature of confirmed due to some unexpected findings.
knowledge (epistemology). (p. 8) Hence, they change their hypothesis to make
The author explicitly outlines what it confirmable with the collected data. On the
she means by “philosophical foundation”, other hand, researchers are encouraged to
which comprises ontology and adopt mixed method approaches to optimise
epistemology. She also briefly defines their research benefits (Riazi & Candlin,
qualitative research, sometimes 2014). For example, exploratory studies
interchangeably used with naturalistic, provide inputs to construct questionnaires
interpretive inquiry, by looking at the for the hypothesis confirmatory research to
purpose of qualitative researchers who are follow. Post-positivism also acknowledges
“interested in understanding the meaning the existence of multiple realities that can be
people have constructed, that is, how people captured through objective scientific
make sense of their world and the procedures. Yin (2014), for example,
experiences they have in the world” indicates that a case study can take either
(Merriam, 2009, p. 13) (emphasis in the theoretical foundation: “a realist perspective,
original). which assumes the existence of a single
The latter group of authors, including reality that is independent of any observer”
Dörnyei (2007) and Marriam (2009), tend to or “a relativist perspective—acknowledging
value the co-existence and contribution of multiple realities having multiple meanings,
both qualitative and quantitative research with findings that are observer dependent”
paradigms as legitimate ways to pursue (p. 91). The use of validity and reliability in
knowledge. However, complications occur mixed-method studies requires subtle
when these concepts cross the paradigm understanding from researchers.
lines and are uncritically applied in some In short, quantitative and qualitative
research. Merriam (2009) explains this as a traditions are established on two different
VNU JOURNAL OF FOREIGN STUDIES, VOL. 37, NO. 3 (2021) 4
Bailey, 2009; Schwandt, 2001; Silverman, image of the researcher as a scholar with
2005). For example, Schwandt (2001) argues: principled standards and integrity, which is
In social science.... validity is an called “craftsmanship” (Kvale, 2002, p. 321).
epistemic criterion: to say that the Some specific strategies to ensure this include:
findings are in fact (or must be) true • Contextualization and thick
and certain. Here “true” means that description which requires
the findings accurately represent the researchers to present detailed
phenomena to which they refer and accounts of the places and the
“certain” means that the findings are phenomena under investigation,
readers to benefit from deep
backed by evidence -or warranted.
understanding and allowing
(p. 267) transferability of the research
This definition, of course, causes findings to other contexts (Aguinis &
outright rejection from qualitative Solarino, 2019; Merriam, 2009);
researchers who hold different positions • Identifying potential researcher bias
about truth (Schwandt, 2001). Kvale (2002) which could be referred to as
explains that the rejection occurs because the positioning the researcher or
concept of validity-as-truth indicates that reflexibility mentioned by Merriam
there is a “firm boundary between truth and (2009) in the earlier section;
non-truth” (p. 302), an obvious threat to • Examining outliers, extreme or
constructivist beliefs of multiple truths. negative cases and alternative
In a response, qualitative researchers explanations which aims to identify
employ different concepts of validity such as and discuss aspects of the study not
trustworthiness (Lincoln & Guba, 1985), supportive of the conclusion to
worthwhileness (Bradbury & Reason, 2001) increase the result’s persuasiveness.
or credibility (Maxwell, 2005; Silverman,
ii) validity/reliability check
2006) which can be achieved by multiple
specific strategies. Dörnyei (2007) believes This group includes specific steps
that these offer useful frameworks to think deliberately taken during the research to
about “the threats to validity and the possible improve validity:
ways that specific threats might be • Respondent feedback (or respondent
addressed” (p. 59). Other authors such as validation/member checking): This
Kvale (2002) and Merriam (2009) still use involves inviting the participants to
the term validity, but they also suggest comment on the study conclusion via
strategies for improving validity. follow-up interviews;
In the following section, I describe • Peer checking: This technique has
some strategies based on Dörnyei’s (2007) been described in the previous
grouping: i) strategies to build up an image section of reliability.
of researcher’s integrity; ii) validity/
iii) research-design based strategies
reliability check; and iii) research-design-
based strategies. Under this heading, there are three
strategies: method and data triangulation;
i) strategies to build up an image of
prolonged engagement and persistent
researcher’s integrity
observation; and longitudinal research
Dörnyei (2007) asserts that the most design. However, Dörnyei (2007, p. 61)
important strategy to ensure the indicates that these strategies could be most
trustworthiness of a project is to create an
VNU JOURNAL OF FOREIGN STUDIES, VOL. 37, NO. 3 (2021) 8
effective when they are organic parts of the reliability and validity are treated differently
research rather than being “add-ons”. It in qualitative and quantitative traditions.
could be inferred that these techniques While quantitative research emphasizes the
should be well combined to contribute to the importance of the consistency of research
overall purposes of the research. results which can be replicated in other
• Method and data triangulation: as contexts, qualitative research aims at
discussed earlier, triangulation research transparency and transferability.
provides different angles of looking Validity in quantitative research focuses on
at the research problem (Merriam, the meaningful fit of the tool with the
2009). It helps reduce “the chance of observed object and the congruence of the
systematic bias in qualitative study” results with reality. However, valid
(Dörnyei, 2007, p. 61). qualitative research requires evidence and
• Prolonged engagement and trustworthiness. Because of this difference,
persistent observation: it is assumed alternative terms are used for reliability and
that the longer the researchers are validity in qualitative research such as
engaged in the project, the more credibility, dependability, trustworthiness,
convincing their results will be. transparency, and transferability. Thirdly,
• Longitudinal research design: the each study can take one or many quality
advantage of longitudinal study is the assurance measures to improve its
increased opportunities for robustness during the research process.
researchers to collect different data Quantitative research seems to strictly
sets and thick description of the require reliability and validity. Qualitative
phenomenon/individual. It also allows research, however, adopts a more flexible
tracing developmental change over approach. Some exemplar strategies include
time. Therefore, longitudinal design triangulation, member check, audit trail,
helps researchers to arrive at a “valid reflexibility, respondent validation,
conclusion” (Duff, 2008, p. 41). contextualization, and thick description.
These strategies are “cumulative” (Aguinis
Clearly, validity can be the & Solarino, 2019, p. 1296) rather than
generalisability of quantitative results or the exclusive. Being aware of these subtle
trustworthiness of qualitative findings from variations will definitely support researchers
the collected data. in selecting appropriate strategies that are
5. Conclusion and Implications aligned with their research purposes
(Dörnyei, 2007) and beneficial to their
In summary, this paper has examined pursuit of knowledge.
the concepts of reliability and validity to References
illustrate the developments of educational
constructs. Although the debates on these Aguinis, H., & Solarino, A. M. (2019). Transparency
and replicability in qualitative research: The
concepts are not settled, there are certain case of interviews with elite informants.
consensus achieved in the literature. Firstly, Strategic Management Journal, 40(8),
reliability and validity, which have been 1291-1315. https://doi.org/10.1002/smj.3015
analysed from two different theoretical Bradbury, H., & Reason, P. (2001). Conclusion:
foundations, are important quality assurance Broadening the bandwidth of validity: Issues
criteria for both qualitative and quantitative and choice-point for improving quality of
action research. In H. Bradbury & P. Reason
research. To ensure the robustness and rigor (Eds.), The handbook of action research
of research, researchers have to take actions (pp. 447-455). Sage.
to adhere to these criteria. Secondly,
VNU JOURNAL OF FOREIGN STUDIES, VOL. 37, NO. 3 (2021) 9
Burns, A. (2010). Action research. In B. Paltridge & Nunan, D., & Bailey, K. M. (2009). Exploring second
A. Phakiti (Eds.), Continuum companion to language classroom research: A
research methods in applied linguistics comprehensive guide. Heinle, Cengage
(pp. 80-97). Continuum. Learning.
Chalhoub-Deville, M., Chapelle, C. A., & Duff, P. A. Onwuegbuzie, A. J., & Leech, N. G. (2005). On
(Eds.). (2006). Inference and becoming a pragmatic researcher: The
generalizability in applied linguistics: importance of combining quantitative and
Multiple perspectives (Vol. 12). John qualitative research methodologies.
Benjamins Publishing. International Journal of Social Research
Dörnyei, Z. (2007). Research methods in applied Methodology, 8(5), 375-387.
linguistics: Quantitative, qualitative, and mixed Qureshi, M. A. (2020). Grammaticality judgment
methodologies. Oxford University Press. task: Reliability and scope. The Journal of
Duff, P. A. (2008). Case study research in applied Asia TEFL, 17(2), 349-362.
linguistics. Lawrence Erlbaum Associates. Riazi, A. M., & Candlin, C. N. (2014). Mixed-
Gass, S. (2010). Experimental research. In B. methods research in language teaching and
Paltridge & A. Phakiti (Eds.), Continuum learning: Opportunities, issues and
companion to research methods in applied challenges. Language Teaching, 47(02),
linguistics (pp. 7-21). Continuum. 135-173.
Guba, E. G., & Lincoln, Y. S. (1989). Fourth Schwandt, T. (2001). Dictionary of qualitative
generation evaluation. Sage. inquiry (2nd ed.). Sage.
Hammersley, M. (1992). What's wrong with Silverman, D. (2001). Interpreting qualitative data:
ethnography: Methodological explorations. Methods for analysing talk, text, and
Routledge. interaction (2nd ed.). Sage.
Kamberelis, G., & Dimitriadis, G. (2005). On Silverman, D. (2005). Doing qualitative research
qualitative inquiry: Approaches to language (2nd ed.). Sage.
and literacy research. Teachers College Press. Silverman, D. (2006). Interpreting qualitative data:
Kuhn, T. (1970). The structure of scientific Methods for analyzing talk, text, and
revolutions. University of Chicago Press. interaction (3rd ed.). Sage.
Kvale, S. (2002). The social construction of validity. Świątkowski, W., & Dompnier, B. (2017).
In N. Denzin & Y. Lincoln (Eds.), The Replicability crisis in social psychology:
qualitative inquiry reader (pp. 299-328). Looking at the past to find new pathways for
Sage. the future. International Review of Social
Psychology, 30(1), 111-124.
Lincoln, Y., & Guba, E. (1985). Naturalistic inquiry.
Sage. Wolcott, H. F. (2005). The art of fieldwork. Alta Mira
Press.
Maxwell, J. A. (2005). Qualitative research design:
An interactive approach. Sage. Yin, R. K. (2014). Case study research: Design and
methods. Sage.
Merriam, S. B. (2009). Qualitative research: A guide
to design and implementation (2nd ed.).
Jossey-Bass.
VNU JOURNAL OF FOREIGN STUDIES, VOL. 37, NO. 3 (2021) 10
Tóm tắt: Các khái niệm giáo dục thay đổi theo thời gian và thể hiện các mốc phát triển trong
nghiên cứu hoặc đường hướng giáo dục. Để minh hoạ cho quá trình này, bài báo tìm hiểu ý nghĩa của
hai khái niệm độ chính xác và độ tin cậy vốn là những khái niệm quan trọng dùng để đánh giá chất lượng
nghiên cứu. Ban đầu, hai khái niệm này được dùng trong các nghiên cứu định lượng. Tuy nhiên, việc áp
dụng hai tiêu chuẩn này cho việc đánh giá nghiên cứu định tính cần phải thay đổi vì hai loại nghiên cứu
này khác nhau về nền tảng lí luận và mục tiêu nghiên cứu. Việc áp dụng không rõ ràng có thể dẫn đến
việc áp dụng phương pháp nghiên cứu hoặc đánh giá nghiên cứu không phù hợp. Bài báo này sẽ làm rõ
nền tảng lí luận của hai loại nghiên cứu định lượng và định tính sau đó phân tích những điểm khác biệt
để hiểu rõ về khái niệm độ chính xác và độ tin cậy. Phần cuối của bài sẽ đưa ra một số đề xuất cho các
nhà nghiên cứu có thể áp dụng linh hoạt hai tiêu chuẩn này để tăng giá trị và ảnh hưởng của nghiên cứu.
Từ khoá: độ chính xác, độ tin cậy, nghiên cứu định tính, nghiên cứu định lượng