Quantitative and Qualitative Research

On Language Learning

By
Mohammad Adnan Latief
adnanlatiefs@yahoo.com

University of Pittsburgh
State University of Malang
2009

1
 Validity of Research Data on Language Learning
By Mohammad Adnan Latief

Abstract: This article presents the differences in the paradigms between

quantitative and qualitative research. The discussion covers (a) verifying vs.
generating theory, (b) formulating research problems, (c) numerical vs.
verbal data, (d) sample vs. informant, (e) one vs. many kinds of data
sources, (f) one vs. many techniques in data collection, (g) natural vs. non-
natural setting, (h) deductive vs. inductive analysis, (i) linear vs. circular
procedure, (j) snow balling technique, (k) process vs. product, (l) human vs.
non-human instrument. This article does not claim that one approach is
more scientific than the other. Each approach has its own scientific value if
used for the right kind of data. A researcher does not have to bother about
contrasting or combining the two approaches. A researcher, though, may use
the two approaches at the same time for different kind of data, not
combining the two approaches for the same kind of data

Key Words: quantitative, qualitative, deductive, inductive, generating,

verifying, linear, circular.

We can not expect every body to have the same way of thinking nor the same way
of solving problems. The difference naturally happens because we are exposed to
different experience. Those who share the same experience, though, may agree on the
same way of thinking. In research, the different way of thinking can be classified, among
others, based on quantitative and qualitative paradigms. This article discusses different
ways of thinking especially in research.

EXAMPLES OF TWO DIFFERENT WAYS OF THINKING

Everyday we always make decisions to make choices in our life. From the time we
wake up from bed in the morning, to the time we go to bed again; we make decisions to
choose what time to wake up, what to eat for breakfast, to go to school (for students), to
work (for those who have a job) or to stay at home, to teach what subject and how (for
teachers), to spend weekend at home or to go somewhere, etc. Each decision of choices
leads to different results. The best choice for us is of course decided based on our
knowledge and experience. We will not make a decision to choose to do something that

2
we know will not lead us to our success. Following are some examples of different
decisions in choosing an action.
In developing students’ good behavior, some teachers believe in behaviorist’s
theory of stimulus-response, which requires teachers to give rewards for students' good
behavior and punishment for bad behavior. They believe that students will have good
behavior in their life if they get used to good habits and are never allowed to experience
bad habits in their school. Other teachers believe in constructivist’s theory which
requires the teachers to develop students’ self awareness of the need to have good
behavior. Students will behave well if they know that it is good for themselves to do so.
Students who misbehave do not realize that what they are doing is wrong (Degeng,
2000:5).
In teaching English skills to students, some teachers believe in structural theory that
drilling the students is the best way, because drilling develops students’ accuracy of
language use. They emphasize accuracy of the students’use of English. Others believe in
communicative theory that practicing communication in English is the best way because
English can be learned by actually using the language in the real communication. They
emphasize fluency of students’ use of English.
In some organizations, leaders are elected based on voting, those who get the
majority of the votes are considered to be the best leaders for them. In other organizations
leaders are selected based on some criteria. Those who get the highest score on the
criteria-based selection are considered to be the best leaders for them.
In deciding students’ scores at the end of a semester, some teachers use norm-
referencing technique, which require the teachers to give high scores to few of the
students, middle scores to the majority of the students, and low scores to another few of
them. Proportion of students who get high, mid, and low scores is always of interest to
the teachers. Other teachers use criterion-referencing, which requires the teachers to use a
certain standard of scoring. In this scoring technique, the percentage of students who get
high, mid, and low scores is not of interest.
There are still many other different ways in solving the same problems. What is
important is that each approach is selected with certain reasoning and is believed to be
the best choice.

3
QUANTITATIVE AND QUALITATIVE RESEARCH
As each research is trying to discover the underlying system of the research objects,
a researcher has to decide the appropriate strategy for that purpose. The strategy is
decided based on either quantitative or qualitative approach. The different approaches
dictate different ways in the process of research; in defining data, in selecting data
collection instrument, in selecting the right sources of data, in collecting data, and in data
analysis. Following are differences between quantitative and qualitative research
paradigms.

Verifying vs. Generating Theory

Research questions ask the underlying system of the objects of the study. In
quantitative research, a researcher predicts the answer to the research problems based on
theory. This theoretical prediction is called research hypothesis. The goal of the
quantitative research is to verify this theoretical hypothesis using empirical data. The
empirical data are analyzed to verify the theoretical hypothesis. So the goal of the
quantitative research is to verify theory. In qualitative research, researchers try to
understand a research object without making any theoretical prediction. The existing
theory does not dictate what a researcher should do to understand a research object.
Anything resulted from the observation and analysis will be accepted as the findings. Or
the theory is generated freely from the result of observation of the research object and
analysis of the result of observation (Bogdan & Biklen 1998:38).

Fixed vs. Flexible Research Problems

In quantitative research, research problems are usually developed cautiously based
on the purpose, the design to be used, the sources of data to be selected, the instruments
to be developed, and the hypothesis to predict the finding. So it usually takes a lot of
thinking before a research problem can finally be formulated. But once the research
problem is formulated, the research problem becomes fixed and the researcher has to
stick to that research problem. The only thing the researcher needs to do is to proceed
with the process of research to get the answer to the problem. The researcher does not

4
need to worry about changing or modifying the research problem that has been well
developed. In qualitative research, research problems are usually defined in a general
way. The more focused research problems are usually made after the research has been
started, when some collected data have been analyzed (Bogdan & Biklen, 1998:2). So
while research problems in quantitative research are made fixed from the beginning,
research problems in qualitative research are developing into more focused during the
process of research.

Numerical Data vs. Verbal Data

In quantitative research, data collected are represented numerically. Data
representing achievement, for example, are recorded in scores. Even data representing
sex are recorded in numbers, 1 for male and 2 for female. The numbers are needed for
statistical analysis. Charles, C.M. (1995:118) states that: “Quantitative data are
analyzed mathematically, and the results are expressed in statistical terminology.” In
qualitative research, the data analysis does not use statistics that requires numerical data.
Therefore, the data are collected and recorded in description, not symbols or numbers
(Bogdan & Biklen, 1998:5). Data representing students’ language achievement, for
example would be the description of what language performance students can show to
represent their language competence. Charles, C.M (1995:118) states that: “Qualitative
data are analyzed through logical-inductive analysis, a process of grouping, regrouping,
and matching data with research questions. The results are expressed as verbal
statements”.

Representative vs. Authoritative

In quantitative research, sources of data are assumed to be heterogeneous, having
different variations, each of which has equal right to represent the group. Since the
population of the sources of data is usually big and so only a sample can be taken, then
the best sample is the one that is most representative to the population. The sample is
usually called respondents or the research subjects (when the source is human beings). In
qualitative research, sources of data are assumed to be homogeneous, having no
variation. Therefore, the trustworthy source of data does not come from representation of

5
different groups of the source, but selected based on certain criteria to find the most
authoritative one. The source in qualitative research is usually called informats (of course
when the source is human being)

The Same Source vs. Many Different Sources of Data

In quantitative research, data are collected from the same group of sources. When
data on personality are collected from students of the English Department of State
University of Malang, for example, only scores resulted from personality tests of those
students are collected. The interpretation of the students’ personality is mainly based on
the scores resulted from the personality test done by the students. In qualitative research,
when personality is involved as one of the variables, the data on personality are assessed
from as many different sources as possible; from their parents, from their neighborhood,
from their diaries, etc. In short, in quantitative research, only one kind of sources is used
while in qualitative research, many different kinds of sources are used.

One Technique vs. Many Different Techniques of Data Collection

In quantitative research, data are collected using one data collection technique.
When data on personality are collected from students of the English Department of State
University of Malang, for example, only scores resulted from personality tests of those
students are collected. Another data collection technique is not needed. The interpretation
of the students’ personality is mainly based on the scores resulted from the personality
test done by the students. In qualitative research, when personality is involved as one of
the variables, the data on personality are assessed using as many as possible different data
collection techniques, like interview to the students themselves, interview to their parents,
examining their neighborhood, examining their historical biography, etc. In short, in
quantitative research, only one technique of data collection is used, while in qualitative
research, various techniques of data collection are used.

One Data Collection Instrument vs. Many Different Instruments

Since the sources of data representing population in quantitative research tend to be
in big number, or otherwise not representative, only one data collection instrument is

6
used. This one instrument has to be developed correctly because the validity of the data
collected are mainly relied on the quality of this instrument. If the data collected suffers
from validity problems, it is the instrument which is to blame. The quantitative
researchers believe, though, that if data collection instrument is correctly developed, then
researchers can trust the well-developed instrument to result in data with strong validity.
So, it is important that the data collection instrument be prepared with high caution.
In qualitative research, data are taken from many different sources, but each source
is in a small number because it is not the representativeness of a sample that counts, but
the authoritativeness that comes from the source selected based on certain criteria.
Therefore, different data collection instruments are used. To collect data on personality of
a certain student, for example, an interview guide to the student, an interview guide to
their parents, an interview guide to their friends and relatives, an observation sheet to
observe their behavior, a note to write anything related the explanation of their
personality, etc. are used as the data collection instruments. To collect data on the correct
use of high level Javanese Structure, to cite another example, a tape recorder to record the
speech of an authoritative Javanese speaker is used and an interview guide to interview
the Javanese speakers and a guide to examine Javanese literature in Javanese are used.

Non-Human Instrument vs. Human Instruments

Since quantitative researchers assume that well-developed instruments can be used
to collect valid data, they can rely so much on their instruments. They have to develop
their instruments in their best possible way to meet their purpose so that the instruments
can be administered by anybody. That’s why this instrument is called non-human
instrument, an instrument that can run by itself to collect data.
In qualitative research, the researcher collects and anlyzes data simulatneously to
draw a temporary conclusion and repeats the cycles several times, deciding what data
needs to be collected again to verify their temporary conclusion. Therefore, the researcher
has to be involved himself in the process of data collection using all kinds of necessary
instruments. The instruments used by the researchers themselves are called human
instruments.

7
 The qualitative researchers believe that many advantages can be obtained more in
the process of data collection with the human instruments than using non-human
instruments. When the subjects do not understand the questions raised by the researcher,
an explanation can be offered by the researcher to clarify the questions. Qualitative
researchers can dig more information from the subjects by rephrasing the questions or by
asking other additional related questions. Qualitative researchers can decide the right time
when to ask, to stop, to continue, or to repeat asking questions to the subjects.
Qualitative researchers who are involved in the life or in the community of the subjects
can understand what is not said by the subjects, or when the subjects are not telling the
truths. The researchers can recheck the information to confirm the reliability of the
information that has been obtained, and to cross check the information with some other
evidence to verify the validity of the information collected.
Qualitative researchers have to be very closed to the subjects’ life and or the
subjects’ community so that they can see the information objectively but are also
reminded not to be too closed to the life of the subjects so that they are not carried away
and do not become biased.

Linear vs. Circular

In quantitative research, all steps of research are done in a linear order, one step is
followed with another step, or one step becomes the pre-requisite for the following step.
The whole process of research is planned from the very beginning to the end. Quantitative
researchers assume that when a plan is well prepared then the whole project will run
successfully. Therefore, once the plan is launched, the researcher has to stick to the plan,
the rest of the activities are done following the plan. There is no need to question the plan
or to go back and forth from one step to the previous steps. In qualitative research, the
research plan does not dictate the research steps as rigidly as the plan in quantitative
research does. The plan may be modified during the process of research. The assumption
is that many things may show up to be better than what we have planned. And this has to
be accommodated in the research. Therefore, the process in qualitative research is not
linear but circular, going back and forth from one step to the previous step in a circular
way.

8
Manipulated Setting vs. Natural Settings.
Quantatitative research data are usually resulted from a formal assessment, like a
test, questionaire, structured interview, etc. This kind of assessment is usually done in a
special maniputed setting where data can be collected efficiently. An achievement test,
for example, is administered when the students are not learning. An interview to an
English teacher can be done in an office when he/she is not teaching. In an experimental
research, settings can even be manipulated, assigning one group of students to an
experimental group to be given an experimental treatment, and another group into a
control group and are given a control conventional treatmnet. This is very different from
the way qualitative researchers collect data. Qualitative researchers collect data using
naturalistic observation or authentic assessment. Qualitative research data on students’
language achievement, for example, is collected through authentic classroom assessment
at the time when students are using the language while studying in a classroom or at the
play ground when they are playing using the language we are assessing. Research data
on language teachers’ teaching behavior is observed when the language teacher is in class
teaching the language. Charles, C.M. (1995) states that a study on patterns of personal
interaction, dominance, and submission among children requires a naturalistic
observation done at the play ground while the children are playing and interacting. A
study on changes, over time, in language patterns by young learners learning English
requires naturalistic observation.

Deductive vs. Inductive Way of Drawing Conclusion

In quantitative research, the most valid conclusion of research is based on research
data collected from all the population, or thesample representing all the population of the
sources. A conclusion based on data from the majority of the population can be applied to
any member of the population. This is called a deductive way of drawing a conclusion,
which is often simplified into the phrase from general to specific, which is of course
meaningles. In qualitative research, the most valid conclusion of research is based on
research data collected from the most authoritative member of the source of data. A

9
conclusion based on data obtained from the members of a community with the highest
authority can be applied to any other member of the community. This is called an
inductive way of drawing a conclusion, which is often simplified into the phrase from
specific to general, which is of course meaningles.

Snowballing Technique
We learn a lot of things in our life but we can not learn everything all at once. We
keep on learning as we go along and improve what we are learning. If we are better today
than tomorrow, it must be because we are learning from our experience that we have got
from yesterday, in the same way if tomorrow we will be better than what we are today, it
must be because we are learning from our experience that we get today. When children
speaking English state hisself and theirself at a certain age, then change the reflexive
pronouns into himself and themself at the following age, and later than they state correctly
himself and themselves, we can see that children learn the rules of reflexive pronouns, one
rule at one time. At their first step, children may just learn from the reflexive pronouns
myself and apply the rule into the reflexive pronouns hisself and theirself, which is of
course wrong. Later then they improve their knowledge of pronouns from hisself into
himself, which is already correct, and from theirself into themself, which is still wrong.
Only much later than they improve their reflexive pronoun from themself to themselves.
This learning process shows the snowballing procedure in learning.
Qualitative researchers follow the process of drawing the rules operating in a
research object gradually from one cycle to another cycle, each cycle involving the steps
of collecting data, analyzing data, and drawing conclusion. Finding from one cycle is
reconstructed based on the finding from the following cycles and on until the finding
does not need to be reconstructed and becomes the final conclusion. This cyclic process
is called snowballing.

Product vs. Process

In quantitative research, the study is always focused on the relationship between
variables, like the causal relationship between sex and students’ achievement, the co-
relation between students’ reading skills and their writing skill, or on a survey like

10
opinions of students towards the Policy of their Rector. In Qualitative research, the focus
is not the condition or the results of a process, but the process itself, like how effective
teachers behave differently from ineffective teachers, or how a writer becomes a skilled
writer, how students fail in their final exams. Rather than focusing the question on which
school is the most effective, qualitative researchers focus their study on what makes
schools effective. Rather than focusing the question on the causal relationship between
students’ sex and their achievement at school, qualitative researchers focus their question
on how female students behave differently in their learning process than male students.

Generalization vs. Transferability

In quantitative research, the significance of the finding as resulted from the analysis
of data collected from the sample depends on how much it can be generalized to the
population represented by the sample. That is why it is important that the sample be
representative to the population. Otherwise, the finding from the sample cannot be
generalized to the population or over-generalization may happen to the finding. Bogdan
and Biklen (2003) state that:
“Generalization refers to whether the findings of a particular study hold up beyond
the specific research subjects and setting involved. If you study a classroom, for
example, people want to know whether other classrooms are like the one you
studied.” (Bogdan & Biklen, 2003:32).

In qualitative research, the source selected is the most authoritative, so the concern
is not the generalization of the finding but the transferability of the finding to another
setting or to other subjects. Bogdan and Biklen (2003) state that:
“Qualitative researchers concern themselves not with the question of whether their
findings can be generalized, but rather with the question of to which other settings
and subjects they can be generalized.” (Bogdan & Biklen, 2003:32).

COMMON QUESTIONS ON QUALITATIVE RESEARCH

Some questions often raised about qualitative research are objectivity of qualitative
research, the choice of quantitative or qualitative research design, and the possibility of
using both at the same time.

Is Qualitative Research Objective?

11
 Very often objectivity is related to numerical data, collected with an instrument
which requires people to choose one alternative answer which can be scored and then
analyzed statistically. So since qualitative research uses verbal data, which involves a lot
of personal judgment, it is considered subjective. This is of course not true because the
subjectivity in qualitative research can be minimized and evidence to support the
objectivity of qualitative research can be provided. The repeated process of data
collection, analysis, conclusion drawing, and verifying the conclusion with data, data
analysis, then reconstructing the temporary conclusion, again in several cycles until the
data get saturated shows the objectivity of qualitative research. Bogdan and Biklen state
that:
”Qualitative Studies are not impressionistic essays made after a quick visit to a
setting or after some conversation with a few subjects. The researcher spends a
considerable time in the empirical world laboriously collecting and reviewing piles
of data. The researcher’s primary goal is to add to knowledge, not to pass
judgment on a setting.” (Bogdan &Biklen, 2003:33).

Which approach is better?

The discussion on differences between quantitative and qualitative research invites
a curious question on which approach is better between the two. Some researchers may
believe strongly that quantitative research is the best way to answer their research
questions as the variables of their interest have to be measured objectively to obtain
numerical data that can be analyzed statistically. Other researchers may believe that
qualitative research is better for them not because they don’t like statistics, but because
the variables involved in their study have to be observed and recorded verbally to be
analyzed inductively. So, each approach is good when used appropriately. Bogdan and
Biklen (2003) state that:
“There is no best method. It all depends on what you are studying. If you want to
find out what the majority of American people think about a particular issue,
survey research which relies heavily on quantitative design in picking your sample,
designing and pretesting your instrument, and analyzing the data is best. If you
want to know about the process of change and how the various school members
experience a change, qualitative method will do a better job”. (Bogdan & biklen,
2003:41)

Can both approaches be used at the same research?

12
 Quantitative and qualitative research is based on different assumptions. The
assumptions on what is the trustworthy source of data, what kind of instrument is
trustworthy to collect data, what is the correct way in analyzing data, etc. are different. In
selecting the sources of data, for example, quantitative researchers rely on
representativeness of the samples, while qualitative researchers rely on authoritativeness
of the sources of data. So, they are different in almost every step of the research. They
are different in choosing the sources of data, in developing instruments to collect data, in
the process data collection, and in the process of data analysis to draw conclusions. So
using two opposing approaches, involving different assumptions, different process would
be a very difficult job. Bogdan and Biklen (2003) state that:
“While it is possible, and in some cases desirable, to use the new approaches
together (Fielding & Fielding, 1986 in Bogdan & Biklen, 2003: 37) attempting to
carry out a sophisticated quantitative study while doing an in depth qualitative
study simultaneously is very difficult. The two approaches are based on different
assumptions” (Smith & Heshusus, 1986 in Bogdan & Biklen, 2003:37).

REFERENCES

Bogdan, R.C., Biklen, S. K. 2003 Qualitative Research for Education An Introduction to

Theory and Methods Fourth Edition Boston, New York, San Fransisco, Mexico
City, Montreal, Toronto, London, Munich, Paris, Hongkong, Singapore, Tokyo,
Cape Town, Sydney: Pearson Education Group, Inc
Charles, C.M. 1995 Introduction to Educational Research. Second Edition New York:
Longman Group Ltd.
Degeng, N.S. 2000 New Paradigms in Education in Decentralization and
Democratization Era. A paper presented in a regional Seminar in Graduate
Program of Educational Technology, PGRi University Adi Buana Surabaya,
Wednesday, April 19, 2000
Marshall, C. R., Gretcher, B. 1995 Designing Qualitative Research Second Edition
London: SAGE Publication.

13